170 Commits

Include renames

Author	SHA1	Message	Date
Chris LuandGitHub	4a2879abad	admin: show a copyable S3 object URL in the bucket file browser (#10933) ... * admin: offer copyable S3 object URLs in the bucket file browser * admin: hide object urls when the bucket type lookup fails * admin: ignore an s3.public_endpoint that is not an absolute http url * mini: build the seeded s3 endpoint with JoinHostPort for ipv6 * admin: reject a query or fragment in s3.public_endpoint * mini: drop the seeded s3 endpoint when a later run disables s3 * admin: reject userinfo and bare delimiters in s3.public_endpoint, redact the warning * mini: pass its s3 endpoint as an admin option instead of mutating viper * admin: keep the rejected s3.public_endpoint value out of the log	2026-08-24 19:29:01 -07:00
Chris LuandGitHub	46ce2c45a2	mini: reserve the admin gRPC port instead of binding it late (#10928) ... * mini: reserve the admin gRPC port instead of binding it late Port selection probes every port with a throwaway listener and closes it. Master, filer, volume and S3 bind a moment later, but the admin waits for all of them first and only then binds its worker gRPC port, roughly two seconds in. That port defaults to the admin http port + 10000, which lands inside the Linux ephemeral range, so one of the cluster's own outgoing gRPC dials can take it during the gap and the admin dies on bind, taking the worker with it. Keep the listener from the availability check and hand it to the admin. * mini: clear the admin gRPC reservation before retaking it A rerun inside one process would otherwise inherit the closed listener of the previous run whenever the reservation fails, and the admin would accept it and only find out inside Serve. * mini: snapshot the admin options for the startup goroutine The cleanup path read the package-level options long after the goroutine started, so a later in-process run could have its reserved listener closed by the previous run.	2026-08-24 14:55:53 -07:00
Chris LuandGitHub	69cc2869ad	Fixes from the review of the admin bucket policy UI (#10907) ... * admin: treat a missing S3 Tables policy as an empty load, not an error The bucket/table policy GET relayed the backend's 404 NoSuchPolicy to the dialog, whose loader treats any non-OK response as a load failure and keeps Save and Delete blocked. A bucket or table without a policy could never be given one. Return policy null instead, the same contract ShowBucketPolicy uses for classic buckets. * admin: reject policy documents the structured editor would misread A top-level JSON array passed the object guard (typeof [] is 'object') and loaded as a zero-statement policy, which the next commit would rewrite to an empty document. Object elements in Action/Resource were coerced to '[object Object]' and saved that way on the s3tables surface, which stores policies verbatim. Both now throw, which routes the document to the JSON tab like other unrepresentable shapes. * admin: let the JSON tab save documents the structured editor can't model Save with the JSON tab active required a round-trip through policyDocToEditorState, so exactly the documents the dialogs shunt to 'JSON tab only' mode (unrepresentable Effect, Resource+NotResource, and the like) could never be saved - Delete was the only mutation left. Invalid JSON still blocks; an unrepresentable document now saves and the editor state stays marked unparsed. * admin: pin the policy editor to what each consumer's backend supports The s3tables evaluator has no NotResource/NotPrincipal fields - it silently drops them, turning Allow+NotResource into allow-everything and making Deny+NotPrincipal inert - and it only matches s3tables: actions against s3tables ARNs, while the editor suggested s3: actions and arn:aws:s3::: resources. New registerPolicyEditor knobs: allowNegation hides the Not* modes and routes documents using them to the JSON tab; resourceSuggestions pins the Resource autocomplete to the open resource's ARN; the S3 Tables dialogs get an s3tables-only action datalist. requirePrincipal now also hides NotPrincipal, which policy_engine.ValidateBucketPolicy always rejects, and the client-side check requires Principal specifically to match that server rule. * admin: save S3 Tables policies from a button, not form submission The multi-input structured editor sits inside a form whose Save button was type=submit, so Enter in any single-line editor input - accepting an autocomplete suggestion, say - implicitly submitted whatever half-built statement the editor held, and the backend stores the document verbatim. A lone statement with no Principal matches nobody, locking out every non-owner. Save is now an ordinary button and the form ignores submission. * admin: block zero-statement policy saves Committing the active tab before the emptiness check made 'Policy JSON is required' dead code: an empty editor serializes to {"Statement":[]}, which the s3tables backend stores verbatim - evaluated default-deny for every non-owner, while the statement-count column keeps showing 'Not configured'. All three policy dialogs now refuse a save with no statements and point at Delete instead. The classic bucket modal only gained a clearer message; the server already rejected the document. * admin: guard S3 Tables policy mutations against stale and overlapping requests The save/delete completions ran against whatever resource the shared modal happened to show by then: a slow PUT for one bucket would hide the modal mid-edit of another and misattribute its alerts, a late DELETE cleared the shared textarea over the newly opened resource with its loaded flag set, and nothing stopped a double-click from firing two overlapping mutations. Ported the classic modal's pattern: capture the target on start, flag the mutation in flight with the buttons disabled, and only touch the UI when the completion still matches the open resource. Success now reloads the page, which also keeps the Policy column's statement count honest. * admin: confirm before deleting an S3 Tables policy Delete Policy sat next to Save and fired on a single click; with default-allow enabled one stray click silently dropped the resource policy and left the bucket open to every principal. Same confirmation the classic bucket modal already has. * admin: let a corrupt stored bucket policy be shown, fixed, and deleted A stored document the decoder rejects made the policy GET 500, and with the loaded flag never set the modal blocked both Save and Delete - the one policy an operator most needs to remove was the one they couldn't, even though the delete path never reads the document. The GET now returns the raw bytes alongside a null policy; the dialog hands them to the JSON tab and unblocks the buttons. * admin: url-encode the bucket name in the policy API calls The filer lists any directory under the buckets path, names S3 would never allow included; one carrying '#' or '%' broke the fetch URL or addressed a different name than the modal shows. * admin: drop stale edit-policy responses on the IAM policies page The same race the bucket and S3 Tables dialogs already guard against: open one policy's editor while its GET stalls, open another, and the late response populates the editor under the second policy's name - Update then saves the first policy's statements over the second. * admin: warn before a bucket policy save drops unsupported fields The editor tracks unmodeled top-level keys precisely so confirmPolicyFieldDiscard can warn before the server's Version+Statement decode discards them, but only the IAM page called it; the bucket modal saved a pasted document with e.g. a console-generated Id without a word while the editor kept displaying the field. * s3: enforce the bucket policy size cap on both surfaces The 20KB cap lived only in the admin UI, so a larger policy stored via the S3 API displayed there but could never be re-saved, desyncing the two writers the cap comment claimed could not desync. The constant now lives in policy_engine next to the shared validator and PutBucketPolicy rejects oversized documents with PolicyTooLarge, matching AWS. * admin: ship the policy editor's fieldset styles with the editor The .policy-stmt-* rules that undo Bootstrap's full-width legend reset stayed behind in policies.templ when the editor markup moved to the shared script, so the bucket and S3 Tables dialogs rendered Actions/ Resource/Principal as full-width jumbo headings. PolicyDatalists is the component every consumer already renders once; the styles live there now. * s3: mirror bucket policy changes into the IAM store from the metadata subscription The advanced-IAM path appends the bucket-policy:<bucket> document to every STS/session evaluation, but only this gateway's own PutBucketPolicy maintained that mirror - a policy tightened or created through the admin UI (or another gateway) never reached it, so revoked access stayed live indefinitely, and the delete side was an unimplemented TODO in any case. The metadata subscription now diffs the stored policy on every bucket entry change and updates or removes the mirror, covering all writers and deletion with one mechanism; IAMManager gains the missing RemoveBucketPolicy. * admin: deduplicate the bucket policy write path Set and Delete carried line-for-line identical filer closures; bucketPolicyMutation already treats nil as clear-the-key. The shared helper sits below Set's validation, since ValidatePolicy cannot take the nil document Delete passes. * s3: drop ValidateBucketPolicy's re-checks of ValidatePolicy rules Both callers run ValidatePolicy first, which already enforces the version and at-least-one-statement rules; the duplicates were dead code with drifted error text. * admin: seed a new statement's Resource from the pinned suggestions A fresh statement on the S3 Tables dialogs started with no resource row at all; seed it with the broadest pinned ARN the same way cfg.bucket already seeds the classic modal. * admin: refuse to save Not* fields the backend would silently drop Hiding the NotResource/NotPrincipal modes was not enough where negation is disallowed: the JSON tab accepts any valid document (that is its job), and a statement's Advanced-fields box can reintroduce the keys, so an s3tables save could still store fields the evaluator drops - turning Allow+NotResource into allow-everything. commitPolicyActiveTab now runs a final document-level check over what would actually be saved; Delete stays available for cleanup. * s3: move the IAM bucket policy mirror on a bucket rename A same-directory rename delivers one event carrying both entries, and the byte-equality short-circuit skipped the new name's mirror when the policy was unchanged - while the replayed delete for the old name removed its mirror, leaving the renamed bucket unmirrored. The mirror decision is now a pure function that removes the old name and writes the new one regardless of byte equality, with the rename cases unit tested. * s3: backfill the IAM bucket policy mirror on lazy bucket loads The metadata subscription only mirrors changes, so a policy that predates the IAM integration never reached the bucket-policy:<bucket> mirror and its grants did not bind on the IAM path until the policy was next modified. The gateway is deliberately lazy at startup (nothing lists all buckets), so the backfill hooks the same place a bucket's policy first becomes known: the cold bucket-config load. EnsureBucketPolicy writes only when no mirror is stored, so repeat loads cost one cached read. * s3: reconcile the bucket policy backfill against concurrent changes The backfill's check-then-write could race an event-driven mirror update or removal and re-store bytes that were already stale, with no later event to heal it. EnsureBucketPolicy now reports whether it wrote, and a write is reconciled against a fresh authoritative entry read: a changed policy is re-mirrored, a removed one is removed. Anything changing after that read fires its own event, which finds the backfill's write already present and supersedes it. The backfill also carries the entry's raw bytes rather than a re-marshaled document, so the reconcile can byte-compare. * s3: prime the bucket policy mirror before advanced-IAM authorization The backfill ran from the lazy bucket-config load, but IAM authorization evaluates the bucket-policy:<bucket> mirror before any handler runs - a grant carried only by a not-yet-mirrored policy denied forever, and the denied request never reached the code that would have loaded the bucket. authorizeWithIAM now primes the bucket config first (an in-memory cache hit once warm), and the backfill runs synchronously on the cold load so the very first authorization already sees the mirror.	2026-08-24 00:52:01 -07:00
Chris LuandGitHub	68ec8ca655	admin: honor a persisted or admin.toml maintenance enabled=false (#10909) ... * admin: honor a persisted or admin.toml maintenance enabled=false The startup path discarded an operator's enabled=false twice over: ApplyDefaultsToProtobuf treated the bool zero value as unset and applied the schema default of true, and a force-enable migration block flipped any survivor. With the legacy /maintenance UI routes gone, nothing could write the config either, so the maintenance system ran unconditionally. Keep the persisted enabled flag across schema-default application in LoadMaintenanceConfig, drop the force-enable block, and add a top-level [maintenance] enabled key to admin.toml as the config surface, persisted through SaveMaintenanceConfig like the per-task settings. Absent config still defaults to enabled. * admin: track presence on the maintenance enabled flag A plain proto3 bool cannot distinguish an operator's persisted false from a legacy file that simply omits the field, so honoring false would have silently switched maintenance off for configs written before the toggle could be persisted. Make the field optional: files that predate presence tracking keep the enabled default, while a file that explicitly persists the toggle is honored either way.	2026-08-24 00:01:48 -07:00
Mathieu ArnoldandGitHub	e931cccc7b	Manage bucket policies via the admin ui (#10895) ... * admin: manage S3 bucket policies from the admin UI Bucket policies were only manageable through the S3 PutBucketPolicy API; the admin UI had no equivalent to the quota/owner/lifecycle editors it already offers. Add GET/PUT/DELETE for a bucket's policy, sharing the exact validation the S3 gateway uses. - Extract validateBucketPolicy/validateResourceForBucket out of s3api_bucket_policy_handlers.go into policy_engine.ValidateBucketPolicy / ResourceMatchesBucket so both the S3 API and the admin UI enforce identical rules. - weed/admin/dash/bucket_policy.go: Get/Set/DeleteBucketPolicy, writing through ObjectTransaction + PATCH_EXTENDED (the lifecycle pattern) so a concurrent owner/quota/lifecycle change on the same bucket entry isn't clobbered. Propagation to every S3 gateway is automatic via the existing filer metadata log subscription. The S3 gateway's IAM policy mirror is deliberately not replicated here (its delete path is already an unimplemented TODO on the S3 side). - New GET/PUT/DELETE /api/s3/buckets/{bucket}/policy routes, CSRF-guarded on writes. - Bucket list and details modal now show a statement-count badge, read from the entry already fetched (no extra RPC). - UI: a JSON-textarea policy editor modal, matching the lifecycle modal's structure. * admin: reuse the visual policy editor for bucket policies Extract the structured policy editor (add/remove statement, action/ resource/principal rows with autocomplete, JSON tab kept in sync) out of policies.templ's inline script into a shared weed/admin/static/js/policy_editor.js, and wire the bucket policy modal in s3_buckets.templ up to it instead of a bare JSON textarea. - registerPolicyEditor(which, config) replaces the hardcoded create/edit id derivation with a per-instance config (textarea/tab/body ids, datalist ids, requirePrincipal, bucket). The IAM policies page keeps its exact pre-extraction ids via two registerPolicyEditor calls, so its markup is unchanged. - New policy_datalists.templ exposes the three shared <datalist>s (actions/resources/principals) as @PolicyDatalists(), now rendered by both policies.templ and s3_buckets.templ. - requirePrincipal seeds new bucket-policy statements with Principal: "*" and adds a client-side check before save (the server, via policy_engine.ValidateBucketPolicy, remains the actual authority); the bucket config pins the Resource autocomplete to the open bucket instead of fetching every bucket in the cluster. - layout.templ loads policy_editor.js globally, after admin.js/ modal-alerts.js (basePath/escapeHtml/showAlert) which it depends on. 3a (the extraction) is a byte-preserving move verified against the unchanged policies.templ behavior before layering 3b's parameterization and the bucket-policy wiring on top. * admin: migrate S3 Tables bucket/table policy editors to the shared editor Third consumer of the shared visual policy editor: the S3 Tables bucket and table policy modals (a bare JSON textarea each) now get the same structured Editor/JSON tabs as the bucket policy and IAM policy pages, via registerPolicyEditor('s3tablesBucketPolicy'/'s3tablesTablePolicy', { textareaId: ... }). Storage and validation are untouched - S3 Tables policies still go through their own s3tables.PolicyDocument type and the s3tables.policy extended attribute, unrelated to policy_engine and s3-bucket-policy; only the editor UI is shared. Fix a real bug surfaced by adding this second load path: the bucket policy modal (and the naive first draft of this s3tables port) called commitPolicyTextareaToEditor() right after a GET and then force-switched to the Editor tab. commitPolicyTextareaToEditor() is designed to leave the current tab in place and the editor state untouched when a document fails to parse (so an in-progress edit survives a bad tab switch), so forcing the Editor tab afterwards could show empty/stale editor state that a careless Save would then serialize over a perfectly valid but structurally-unusual stored policy. Add loadPolicyTextareaIntoEditor(which) to policy_editor.js, which has no "current tab" to defer to and instead falls back to the JSON tab with an alert on a document the structured editor can't represent - the same safety editPolicy already had in policies.templ - and use it at all three "populate the editor right after a GET" call sites (bucket policy, S3 Tables bucket policy, S3 Tables table policy). * admin: show policy statement count on the S3 Tables buckets page Mirrors the "Policy" column already added to the classic S3 buckets list: a clickable badge with the statement count when the table bucket has a resource policy, "Not configured" otherwise. S3 Tables policies are a separate mechanism (s3tables.PolicyDocument under the s3tables.policy extended attribute) from the S3 bucket policy work elsewhere in this branch (policy_engine.PolicyDocument / s3-bucket-policy), so this is a parallel implementation of the same pattern rather than shared code. - S3TablesBucketSummary gains PolicyStatementCount, populated in GetS3TablesBucketsData from entry.Entry.Extended[s3tables.ExtendedKeyPolicy] via the new extractS3TablesPolicyStatementCountFromEntry - no extra RPC, the entry is already fetched for ExtendedKeyMetadata. - The badge reuses the existing .s3tables-bucket-policy-btn class, so it opens the same policy modal as the row's action button with no JS changes. * admin: don't let a failed policy GET open the door to an empty overwrite loadS3TablesBucketPolicy/loadS3TablesTablePolicy cleared the textarea, then unconditionally called loadPolicyTextareaIntoEditor() regardless of whether the GET actually succeeded - including when fetch() rejected or the response was not ok, silently logged to console only. That leaves the structured editor holding a legitimate-looking empty policy ({version, statements: []}), with the Editor tab active by default. If Save is then clicked, commitPolicyActiveTab() serializes that empty state into the textarea as `{"Version":"2012-10-17","Statement":[]}` - a non-empty string - before the "Policy JSON is required" guard ever sees it, so the guard passes and the transient load failure gets written over whatever policy was actually stored. Add s3tablesBucketPolicyLoaded/s3tablesTablePolicyLoaded, set true only once a GET has actually completed (ok, including a genuinely empty policy) and false on any failure path (fetch rejection or a non-ok response, which previously fell through silently). Both submit handlers now check the flag before touching the editor at all, and a failed load surfaces via alert() instead of only a console.error - the user previously had no visible indication the load had failed. Verified with a jsdom simulation driving the real rendered page against a stubbed fetch: a failed GET followed by Save now sends no PUT at all (previously it sent Statement: []); a successful GET followed by Save still PUTs the loaded policy unchanged. * admin: address code review findings on the policy editor 1. policy_editor.js: policyEditors is only pre-populated for 'create'/ 'edit'; every other `which` (bucket, s3tablesBucket, s3tablesTable) stays undefined until its first successful async load. Nothing in this file enforces that a page hide its Editor/JSON tabs and Add-statement button until that load completes - the S3 Tables policy modals don't - so a click in that window (e.g. Add statement, or switching to the JSON tab) threw "Cannot read properties of undefined (reading 'unparsed')". Add policyEditorState(which), which lazily initializes a default state, and route addPolicyStatement, the jsonTabBtn 'show.bs.tab' handler, commitPolicyActiveTab, and renderPolicyEditor through it. Verified with a jsdom simulation against a never-resolving fetch: the exact click threw on the pre-fix code and no longer does. 2. s3_buckets.templ: the bucket-policy Save handler checked the textarea for emptiness before calling commitPolicyActiveTab(), which is what actually serializes the structured Editor tab's fields into that textarea. A policy entered entirely through the Editor tab (the primary path - never touching the JSON tab) left the textarea at whatever it was at load time, so creating a new policy this way hit "Enter a policy document" and Save silently did nothing. Move the commit before the emptiness check, preserving the existing alert and early-return. Verified with a jsdom simulation: Add-statement then Save (no tab switch) now PUTs the entered statement; before the fix the same sequence never reached fetch(). 3. s3tables_buckets.templ / s3tables_tables.templ: the policy Editor/ JSON nav-tabs were missing the ARIA roles Bootstrap's own tab pattern expects (role="tab"/"tabpanel", aria-selected, aria-controls, aria-labelledby) - screen readers had no way to tell these were tabs or which pane went with which button. Added the standard Bootstrap 5 tab markup to both. * admin: guard policy load/save flows against overlapping requests 1. s3tables.js: loadS3TablesBucketPolicy/loadS3TablesTablePolicy had no protection against overlapping loads. Opening one bucket's (or table's) policy dialog and then another's before the first GET resolved let the late response write its document into the shared textarea and mark the dialog "loaded" while it was now targeting the second resource - a subsequent Save would then push the first resource's policy onto the second. Add a per-load monotonic sequence number (s3tablesBucketPolicyRequestSeq / s3tablesTablePolicyRequestSeq, the same pattern already used for the classic bucket-policy load in s3_buckets.templ); a response is only applied - textarea, loaded flag, editor state - if its captured sequence still matches the latest one issued. Verified with a jsdom simulation: bucket A's policy load (artificially slow) followed immediately by bucket B's (fast) previously left A's policy in the textarea once A's late response landed; it now correctly keeps B's. 2. s3_buckets.templ: the bucket-policy Save button lives outside the (initially hidden) editor wrapper, so it stays clickable while a load is still in flight - the existing policyRequestSeq guard only protects the *load* from a stale response, not Save from firing before any load for the current bucket has completed. Add bucketPolicyLoaded, reset before each GET and set only once the matching response lands, and check it at the top of the Save handler. Verified with a jsdom simulation: clicking Save immediately after opening the dialog, before a (deliberately never-resolving) GET settles, now sends no PUT; a normal load-then-save sequence still PUTs the loaded policy unchanged. * admin: address further code review findings on the policy editor 1. s3tables.js: loadS3TablesBucketPolicy/loadS3TablesTablePolicy only reset the JSON textarea when a new load starts; the structured editor kept showing the previously loaded resource's statements (Editor tab is the default active one) until the new fetch resolved. Call loadPolicyTextareaIntoEditor() against the now-cleared textarea immediately, so switching resources visibly resets the editor right away instead of only once its own load completes. Verified with jsdom: opening bucket A (loads fully) then bucket B (GET never resolves) no longer leaves A's statements visible in B's editor. 2. s3tables.js: deleteS3TablesBucketPolicy/deleteS3TablesTablePolicy had no loaded-state check, so a failed GET (which already blocks Save) left Delete fully able to remove the resource's stored policy sight unseen. Add the same s3tablesBucketPolicyLoaded/s3tablesTablePolicyLoaded guard Save already uses. Verified with jsdom: delete after a failed load now sends no DELETE; delete after a successful load is unaffected. 3. s3_buckets.templ: the bucket-policy Editor/JSON nav-tabs were missing the same ARIA roles already added to the S3 Tables policy tabs in an earlier round (role="tab"/"tabpanel", aria-selected, aria-controls, aria-labelledby) - this instance was out of scope for that review comment but is the same gap. Bootstrap's own tab.js already manages aria-selected on tab switch once the attribute exists, so no extra JS was needed. 4. s3_buckets.templ: neither the bucket-policy Save nor Delete handler guarded against a double-click, or against firing while the other was still in flight - two overlapping PUT/DELETE requests for the same bucket could land in either order. Add a shared bucketPolicyMutationInFlight flag: set (and both buttons disabled) before each fetch, cleared (and buttons re-enabled) on failure so the user can retry, left set through the existing success hide-and-reload path, and also reset when a new bucket's dialog opens so an abandoned in-flight request from a closed dialog can't leave the buttons stuck disabled. Verified with jsdom: double-clicking Save now sends exactly one PUT, and a Delete click while that PUT is still pending sends no DELETE. * admin: scope bucket-policy mutation completions to the bucket that started them 1. The previous round's fix reset bucketPolicyMutationInFlight whenever a new bucket's policy dialog opened, to avoid leaving Save/Delete stuck disabled if the modal was closed mid-request. That traded one bug for a worse one: if bucket A's PUT/DELETE was still in flight when the user opened bucket B's dialog, the reset let B's Save/Delete fire immediately, and A's completion handler - unaware anything had changed - would still hide the (now B's) modal and reload the page out from under whatever the user was doing with B, on success, or alert a message with no bucket context, on failure. Stop resetting on reopen, so a pending mutation for a previous bucket keeps this bucket's Save/Delete blocked until it settles (matches the "preventing overlapping mutations" the review comment describes). Instead, capture policyEditorBucket as targetBucket right before each fetch and compare it against policyEditorBucket again in the completion handler: the in-flight flag is always released so the buttons never get stuck, but the modal-hide/reload/alert only fire if this bucket is still the one showing; a stale completion for an abandoned bucket just logs to the console instead. Verified with a jsdom simulation: opening bucket B while bucket A's Save is still pending leaves B's Save button disabled and a click on it a no-op; once A's PUT resolves, B's button re-enables but no modal.hide()/reload() fires (previously both fired unconditionally). 2. bucketPolicyDeleteBtn had no bucketPolicyLoaded check, unlike Save - a failed GET blocked Save but left Delete free to remove a policy the client never actually saw (the same gap already fixed for the S3 Tables policy modals in an earlier round). Added the same guard, ahead of the confirm() dialog. Verified with jsdom: Delete after a failed load now sends no DELETE request. * admin: fix spelling mistake	2026-08-23 22:11:18 -07:00
Chris Lu	3b8931c2f6	admin: address review feedback on the maintenance scanner fix	2026-08-23 00:26:38 -07:00
Junker der ProvinzandGitHub	8d2c0273bd	admin: stop the maintenance scanner pinning itself to one scan per second after a transient failure (#10887) ... * admin: honour persisted task configs when building the maintenance policy buildPolicyFromTaskConfigs passed a literal nil to vacuum, erasure_coding and balance LoadConfigFromPersistence. Those functions look for their LoadXTaskPolicy() accessor via a type assertion, which a nil interface can never satisfy, so every call fell through to NewDefaultConfig() and the policy came back with the compiled-in defaults - Enabled: true among them. A task disabled on disk was therefore still scheduled, and the only trace was a glog.V(1) "Using default ... configuration" line. Thread the real ConfigPersistence through instead. There are two copies of this function: the one in weed/admin/dash builds config.Policy on the normal admin startup path and can simply take cp as its receiver, and the one in weed/admin/maintenance is the fallback used when the config carries no policy yet, which now receives the store from NewMaintenanceManager. weed/admin/dash already imports weed/admin/maintenance, so the maintenance side has to keep the duck-typed interface{} parameter that the task loaders already use rather than importing the concrete type back. The store is only handed over when a data directory is configured: an unconfigured one has nothing to read, and a typed nil pointer would pass the loaders' type assertion and then panic on first use. Fixes #10874 * admin: restore the maintenance scan cadence after an error backoff scanLoop shortens its ticker to the error backoff delay after a failed scan, but it decided whether to replace the ticker by comparing the target interval against the configured scan interval instead of against the interval the ticker was actually running at. Once the errors stopped, getScanInterval returned the configured interval again, the comparison came out false, and the ticker was left at the backoff delay - so a single transient scan failure pinned the scanner to one scan per second for the rest of the process lifetime. That is the ~1/second cadence in issue #10874: 658 KB/s of "Cancelled N stale pending balance tasks before re-detection" and 193k orphaned task files over two days. Track the interval the ticker is running at and compare against that, so both entering the backoff and returning to the normal cadence replace the ticker. While in here: - defer ticker.Stop() bound the ticker that was current when the defer was registered, so every replacement ticker leaked on return. Wrap it in a closure. - running was written by Start/Stop and read by all three background loops without synchronisation. Guard it with the existing mutex, fold the running check in triggerScanInternal into the lock it already takes, and make Stop a no-op when not running so a second call cannot close the stop channel twice. Refs #10874 * admin: make the maintenance policy actually reach the task detectors Loading the persisted task configs into the maintenance policy only matters if something reads that policy, and nothing did. MaintenanceIntegration pushes the policy into every registered detector and scheduler through interface{ SetEnabled(bool) } and interface{ SetMaxConcurrent(int) } type assertions. Every task registered through base.RegisterTask is backed by base.GenericDetector and base.GenericScheduler, and neither implemented either method, so all four assertions failed silently for every task on every startup. The policy's enabled flag reached nothing: ScanWithTaskDetectors gates on detector.IsEnabled(), and the queue's policy lookups for max concurrent and repeat interval are fallbacks that only fire when the scheduler reports zero, which the generic scheduler never does. Add the setters, delegating to the TaskConfig.SetEnabled the interface already declares and to TaskDefinition.MaxConcurrent, which is what GetMaxConcurrent returns. Applying the policy required three more fixes, because with the assertions working the policy could now do damage as well as good: - IsTaskEnabled reports false for a task type the policy has no entry for, so applying it unconditionally would have disabled every task the policy does not list. Skip task types with no policy entry: no entry means no opinion, not disabled. - ec_balance was exactly such a task. It is registered like the other three but had no entry in the policy builder and no accessor on ConfigPersistence at all, so its configuration could never be persisted. Add SaveEcBalanceTaskPolicy/LoadEcBalanceTaskPolicy, the task_ec_balance.pb file, the SaveTaskPolicy dispatcher case, and the policy entry. - InitMaintenanceManager ran before loadTaskConfigurationsFromPersistence, which replaces each task's whole config object, so the policy was applied and then immediately thrown away. Swap the order. Both read the same files, so the policy is now the last writer and stays authoritative. MaintenanceManager.UpdateConfig also updated the queue's and the scanner's policy but not the integration's, so a policy changed at runtime never reached the detectors. Add MaintenanceIntegration.SetPolicy and call it. While building the policy, stop hand-copying each task's fields and use the task's own ToTaskPolicy(). The hand-written version was a second definition of every task's policy and had already lost the erasure coding preferred tags and replica placement and the balance IO rate limit. For the same reason, the "nothing persisted yet" branches of LoadVacuumTaskPolicy, LoadErasureCodingTaskPolicy and LoadBalanceTaskPolicy now derive from each task's NewDefaultConfig() instead of a third hand-written copy. Those copies had drifted, so with a data directory but no config file on disk the effective defaults differed from what the task and the admin UI schema both advertise: vacuum scan interval 24h -> 2h balance scan interval 6h -> 30m balance imbalance 0.1 -> 0.2 erasure coding scan interval 168h -> 1h erasure coding fullness 0.90 -> 0.95 erasure coding min volume 1024MB -> 30MB Finally, weed/admin/dash and weed/admin/maintenance each carried a copy of the policy builder and they had already diverged. Export the maintenance one as BuildPolicyFromTaskConfigs and have dash call it. Refs #10874 * worker: warn when a config store cannot supply a task's persisted config LoadConfigFromPersistence logged a single glog.V(1) "Using default X configuration" for every way of not loading anything, so the bug in issue #10874 - a store handed in that the type assertion rejects, leaving a task running on compiled-in defaults - looked exactly like the normal "no data directory configured" case. The reporter had to read the source to work out why their disabled task kept running, and asked for this specifically. Separate the cases. A non-nil store that does not provide the accessor is always a wiring bug and is now logged at warning level, naming the type and the missing method. A read error or a policy that will not apply is also a warning. No persistence configured, and a store with nothing saved yet, stay at V(1): those are normal. Refs #10874 * admin: stop GetTaskPolicy panicking on a maintenance policy that is nil GetTaskPolicy dereferenced its MaintenancePolicy argument to look at TaskPolicies, so IsTaskEnabled, GetMaxConcurrent and GetRepeatInterval all took the admin process down when handed a nil policy. A nil policy is not a programming error here: MaintenanceConfig.Policy is unset until something builds one, DefaultMaintenanceConfig returns a config with no policy at all, and UpdateConfig installs whatever config it is given. Found by calling IsTaskEnabled with the policy from a freshly defaulted MaintenanceConfig. Treat a nil policy as "no entry": no task enabled, the safe concurrency default of 1, and a repeat interval of 0 so callers fall back to their own default instead of reading DefaultRepeatIntervalSeconds off nil. Also add the startup test this was found with. It walks the admin server's startup sequence over a data directory that has balance saved as disabled and checks the state that decides whether issue #10874 happens: the balance detector reports disabled, vacuum stays enabled, and tasks whose config was never saved keep their compiled-in default. Refs #10874 * admin: document the synchronisation SetPolicy would need beyond startup ConfigureTasksFromPolicy now really writes TaskDefinition.Config and TaskDefinition.MaxConcurrent, which the scan loop reads through detector.IsEnabled() with nothing synchronising the two. Every caller runs during admin server startup today, before the scan loop exists, so there is no live race - but the next caller has to add the locking, and the same already applies to UpdateAllConfigs replacing the whole config object. Write it down at the seam instead of leaving it to be rediscovered. Refs #10874	2026-08-22 23:41:45 -07:00
Junker der ProvinzandGitHub	f3caf6e7da	admin: count plugin-runtime workers in worker metrics (#10884) ... * admin: count plugin-runtime workers in worker metrics The admin server keeps two worker registries: the legacy maintenance-worker map, filled by workers registering over the worker gRPC stream, and the plugin worker registry, filled by workers started as `weed worker`. Both the SeaweedFS_admin_workers_connected / SeaweedFS_admin_worker_slots gauges and the dashboard's Workers card read only the legacy map, so a cluster that runs the admin and its workers as separate components reported 0 workers even while its workers showed up on the plugin pages and ran scheduled jobs. Aggregate both registries instead. The two are merged by worker ID: `weed mini` starts both runtimes out of one working directory, so they share the persisted worker ID and must not be counted twice. For such a worker the slot numbers still come from the legacy registry, which keeps mini's existing readings. Plugin workers report their slots in the heartbeat, so detection and execution slots are summed from there; a worker that has connected but not yet sent a heartbeat counts as connected with zero slots. Fixes #10525 * admin: clamp negative worker-reported slot values in metrics merge A plugin worker's self-reported heartbeat slot counts are untrusted input; clamp them to 0 before summing so a stale or misbehaving worker can't drive the aggregate gauge negative, matching the same defensiveness already used in registry.go's own slot arithmetic.	2026-08-22 22:54:39 -07:00
4 people	641fc8b031	admin: add visual iam policy editor (#10878) ... * admin: add visual iam policy editor Add a structured, tabbed editor (Editor / JSON) for creating and editing IAM policies in the admin dashboard, alongside the existing raw-JSON textarea: - policies.templ: per-statement cards for Sid, Effect, Action, and Resource, with unmanaged fields (Principal, NotPrincipal, NotResource, Condition, or anything else) preserved verbatim in a per-statement "advanced fields" JSON box so nothing is lost on round-trip. Switching tabs commits and reparses in both directions. Restored the "Use Sample Policy" button, now filling both the structured editor and the JSON tab. The "Validate" button now calls the existing but previously unused POST /api/object-store/policies/validate endpoint instead of doing JS-only checks. - Progressive Resource ARN autocomplete: suggests bucket names first, then once "bucket/" is typed, suggests bucket/* plus the bucket's direct subfolders, drilling down one path segment at a time as the user types further "/" characters. - New GET /api/files/list-folders endpoint (file_browser_handlers.go) backing the folder autocomplete: wraps the existing file browser data function and returns just the subdirectory names as JSON, scoped to paths under /buckets. - Action-name suggestions (datalist) for the Action field, sourced from the existing s3_constants.S3_ACTION_* constants plus new s3_constants.S3TABLES_ACTION_* constants (extracted from the s3tables operation dispatch switch) so the suggestion list can't drift from the strings the engines actually understand. - policy_handlers.go: ValidatePolicy now accepts a statement with only NotResource set (previously required Resource), matching policy_engine.validateStatement and the fact the new editor makes such statements reachable from the UI. - Tests: ValidatePolicy behavior, route registration for the policy API and the new list-folders endpoint, list-folders path scoping, and the action-suggestion list's shape. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * admin: fix XSS, cache poisoning, and cap overshoot in policy editor Address code review findings on the IAM policy editor added in the previous commit: - policies.templ (displayPolicyDetails): escape every interpolated policy value (Sid, Effect, Action, Resource, policy name, and the raw JSON document) before assigning to innerHTML. Policy documents can come from other admins or an import, so an unescaped field could execute script when the "View" modal renders it. - policies.templ (policyEditorStateToDoc): reject JSON arrays in a statement's "advanced fields" box, not just invalid JSON. `typeof [] === 'object'` was true, so a JSON array was assigned to the statement; subsequent property assignments (Sid, Effect, ...) landed on the array object but JSON.stringify of an array only serializes numeric indices, silently dropping them. - policies.templ (loadPolicyFolderNames): on a failed folder lookup, remove the cache entry instead of permanently caching the empty fallback, so a transient network/server error doesn't block retries for the rest of the page's lifetime. - file_browser_handlers.go (ListFolders): stop appending directory names as soon as the running count reaches maxListFoldersEntries, instead of only checking the cap after a full page is processed, so the returned list never exceeds the configured cap. Regenerated policies_templ.go with the already-stamped templ v0.3.1001 to keep the diff scoped to this file. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * admin: stop policy editor from clobbering the active tab and dropping malformed advanced fields Address two review findings on the IAM policy editor (Issue 3, stored-XSS in displayPolicyDetails, was already fixed by the previous commit and is unchanged here): - createPolicy, updatePolicy, and validatePolicyDocument always committed the structured editor's (possibly stale) state into the JSON textarea before submitting, even when the user had just edited the JSON tab directly. That silently discarded the user's JSON edits and validated/saved the old structured-editor state instead, which could leave broader permissions in force than intended. Added commitPolicyActiveTab(which), which commits whichever tab is currently visible into the other side instead of unconditionally overwriting the JSON tab from the editor: if the JSON tab is active it parses that JSON back into the structured editor (without touching the textarea itself), otherwise it serializes the structured editor into the textarea as before. All three call sites, plus the JSON-tab "show.bs.tab" handler, now use this and abort with an alert if the currently active tab's content can't be committed. - policyEditorStateToDoc silently continued with an empty object when a statement's "advanced fields" box held invalid JSON, so switching tabs, validating, or saving would drop Principal/NotResource/Condition from that statement without telling the user. It now throws (with the statement number and parse error) on invalid or non-object JSON there, and callers surface that via showAlert and abort instead of proceeding. Regenerated policies_templ.go with the already-stamped templ v0.3.1001. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * admin: keep unmanaged top-level policy fields across editor tab switches policyDocToEditorState only carried Version and Statement into editor state, so any other top-level key (e.g. Id) present in the JSON tab was silently rewritten away as soon as the user switched to the Editor tab and back. Capture those keys in state.otherFields and merge them back in policyEditorStateToDoc before Version and Statement are written, so the two tabs stay faithful to each other and the editor never rewrites text the user typed. Note this is editor fidelity only: the admin API's policy_engine.PolicyDocument carries just Version and Statement, and DocumentJSON is never populated, so such fields are still discarded by the server once a policy is saved. Making them survive a save would require a backend change, which is out of scope here. Regenerated policies_templ.go with the already-stamped templ v0.3.1001. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * admin: warn before a policy save discards unsupported top-level fields The editor round-trips unmanaged top-level keys (e.g. Id) between the Editor and JSON tabs, but the admin API's policy_engine.PolicyDocument carries only Version and Statement, so the server drops them on save and the user saw no indication. Added confirmPolicyFieldDiscard(), called from createPolicy and updatePolicy after the active tab is committed (so the field list is accurate whichever tab is showing). It names the fields that will be lost and lets the user confirm or cancel. Not wired into validatePolicyDocument, which doesn't persist anything. Chose the warning over the alternative of persisting these fields through the backend: policy_engine.PolicyDocument is shared by the S3 bucket-policy engine and IAM evaluation, so extending it would change the stored document shape for every policy in the codebase - far beyond the scope of this editor. Regenerated policies_templ.go with the already-stamped templ v0.3.1001. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * admin: reject malformed Effect and Resource/NotResource conflicts in policy editor Two review findings on the IAM policy editor: - policyDocToEditorState defaulted any non-"Deny" Effect (missing, misspelled, wrong case) to "Allow". A statement meant to be "Deny" with a typo like "deny" would silently become a permissive "Allow" instead of being rejected. It now throws on anything but an exact "Allow" or "Deny", naming the offending statement and value. commitPolicyTextareaToEditor catches this the same way it already catches invalid JSON: alert the user and keep the JSON tab active instead of switching to the Editor tab with wrong data. - policyEditorStateToDoc could save a statement with both Resource (from the structured field) and NotResource (surviving in the "advanced fields" extras from before the user switched to using Resource) set at once - a contradictory combination neither the admin's ValidatePolicy handler nor policy_engine's evaluator rejected. When the structured Resource field is non-empty it now deletes any leftover NotResource from extras, consistent with the file's existing rule that structured fields take precedence over extras. Mirrored the existing Principal/NotPrincipal exclusivity check in weed/admin/handlers/policy_handlers.go's ValidatePolicy to reject the same combination server-side, since create/update perform no validation at all. Deliberately left policy_engine.validateStatement (used by the S3 bucket-policy PUT handler for every bucket policy in the product) unchanged - extending that shared validator is a larger, separate change outside this admin-editor fix's scope. Added a handler test for the new Resource+NotResource rejection. Regenerated policies_templ.go with the already-stamped templ v0.3.1001. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * admin: add NotResource support to the visual policy editor Since Resource and NotResource are mutually exclusive (enforced by a previous fix), NotResource could previously only be set through the raw JSON in a statement's "advanced fields" box. Promote it to a first-class mode of the structured editor: - The static "Resources" label is now a Resource/NotResource dropdown; the same list of values underneath is reused for either key depending on the selected mode, with a short form-text explaining the semantics. - NotResource is added to POLICY_STATEMENT_KNOWN_KEYS, since it's now a managed field like Resource rather than something that falls through to extras. - policyDocToEditorState derives resourceMode from which key is present on load, and throws (same handling as the existing malformed-Effect case: alert, keep the JSON tab active) if a hand-edited document has both Resource and NotResource on one statement, since that can't be represented by the dropdown. - policyEditorStateToDoc writes only the key matching the selected mode, replacing the previous one-directional "delete NotResource whenever Resource is set" fix with mode-driven logic that also deletes Resource when NotResource is selected. - displayPolicyDetails (the read-only View modal) now shows the actual NotResource values with a distinct label instead of a static "(NotResource used instead)" placeholder. No backend changes: the server-side "cannot specify both" check added previously in policy_handlers.go's ValidatePolicy already covers this. Regenerated policies_templ.go with the already-stamped templ v0.3.1001. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Eb2a51LciCsyY35sqDoGNe * admin: reject non-object policy documents; catch Principal/NotPrincipal conflicts server-side Two review findings: - policyDocToEditorState treated a top-level JSON value that wasn't an object (null, or a bare string/number/boolean) as an empty statement list instead of failing explicitly. If the user typed e.g. "hello" or 42 in the JSON tab and switched to the Editor tab, their input was silently discarded and replaced with an empty policy - the same class of "guess instead of reject" bug fixed for malformed Effect and Resource/NotResource conflicts previously. Added an explicit check that throws for null/scalar input, while leaving array and object document shapes accepted exactly as before. - weed/admin/handlers/policy_handlers.go's ValidatePolicy checked the Resource/NotResource conflict by non-empty length (len(...Strings()) > 0), which misses a statement where Resource is explicitly present but an empty list (e.g. "Resource": []) alongside a non-empty NotResource. Switched that check to field presence (!= nil), matching how policy_engine's own validateStatement already treats Principal/NotPrincipal exclusivity. Also added the equivalent Principal/NotPrincipal presence check to this handler, which had none before - the advanced-fields box in the visual editor lets a user set both today, and nothing server-side caught it. The existing non-empty "Resource or NotResource is required" check is left as a length check, since an empty array shouldn't count as "provided". Added test cases for both conflict checks in policy_handlers_test.go. Regenerated policies_templ.go with the already-stamped templ v0.3.1001. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Eb2a51LciCsyY35sqDoGNe * admin: add Principal/NotPrincipal support to the visual policy editor (v1, AWS-only) Adds a first, deliberately narrow structured editor for a statement's Principal/NotPrincipal, left out when NotResource support was added: - A Principal/NotPrincipal mode dropdown mirrors the existing Resource/NotResource one (same mutual-exclusivity handling: the two fields can't be set at once, and switching modes reuses the same value list). - A simple repeatable text-value list feeds a single {"AWS": [...]} object on save - always the AWS type, never the "bare" (untyped) SeaweedFS-extension shape. Per policy_engine's allowedPrincipalKeys, Service/Federated/CanonicalUser also parse successfully, but nothing in the S3 bucket-policy evaluation path ever sets a real caller's principal to a service name, an OIDC provider ARN, or a canonical user ID, so only AWS is functionally meaningful today - out of scope for this v1. - On load, only the exact {"AWS": ...} single-key shape is unwrapped into the structured field and removed from "extras". Anything else (bare string/array, a different single type key, or several type keys at once) is left untouched in "extras" exactly as before, with a visible warning under the dropdown so the user knows a Principal/NotPrincipal exists but isn't shown there. Saving with the structured field left empty never touches whatever's already in extras, so a preserved complex form isn't silently dropped just because the user didn't touch this field. - The read-only View modal now displays Principal/NotPrincipal for any shape (via a small generic summarizer), not just the AWS-simple one. - Generalized the action/resource field-to-state-key mapping (used by commitPolicyEditorForm and the add/remove-item click handler) into a shared lookup table instead of stacking another ternary, now that a third field (principal) exists. Regenerated policies_templ.go with the already-stamped templ v0.3.1001. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Eb2a51LciCsyY35sqDoGNe * admin: support the bare "*" wildcard Principal in the visual editor "Principal": "*" (and NotPrincipal: "*") is the standard AWS shorthand for "everyone" and is common in real bucket policies, but the v1 Principal/NotPrincipal editor only recognized the {"AWS": ...} object form, leaving a bare "*" statement's principal hidden in Advanced fields. parseSimpleAwsPrincipal now also accepts the bare string "*" as a simple, structurally-editable value. On save, a principal value list containing exactly ["*"] is written back as the bare "*" string (matching the common convention) rather than wrapped as {"AWS": "*"}; anything else still wraps under AWS as before. Updated the field's form-text hint accordingly. Regenerated policies_templ.go with the already-stamped templ v0.3.1001. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Eb2a51LciCsyY35sqDoGNe * admin: add Principal field autocomplete backed by users + IAM roles Adds a datalist-backed autocomplete for the policy editor's Principal/ NotPrincipal text fields, sourced from a new API listing existing identities: - weed/admin/dash/principal_suggestions.go: AdminServer.GetPrincipalSuggestions combines S3 user ARNs (via the existing GetObjectStoreUsers + iam.UserArn) with IAM role ARNs (via integration.NewFilerRoleStore / ListRoles, reusing the exact same construction already used in iam_manager.go - no new dependency risk introduced). Role ARNs are reconstructed from the role name using SeaweedFS's default arn:aws:iam::role/<name> convention rather than fetching each role's stored definition, since this only backs a suggestion list. Role listing failures are logged and swallowed rather than failing the whole request - an incomplete suggestion list is fine, blocking policy editing over it is not. Service accounts are deliberately not listed separately: a service account's ARN is identical to its parent user's, already covered by the user list. - weed/admin/handlers/policy_handlers.go: GetPrincipalSuggestions handler exposing this as {"principals": [...]}. - Route registered at the API root (GET /api/principals) rather than under policyApi's "/object-store/policies" prefix, since that subrouter's existing "/{name}" GET route would shadow any single-segment GET route registered after it (the same class of gotcha previously seen with "/validate"). - weed/admin/view/app/policies.templ: a shared, lazily-fetched-once policyPrincipalSuggestions datalist (flat list - unlike the progressive per-folder Resource ARN autocomplete, users/roles aren't hierarchical), wired into policyListRowHtml for field:"principal" and populated on input/focus, with "*" always offered first. Added tests for the new ARN-construction helper and route registration. Regenerated policies_templ.go with the already-stamped templ v0.3.1001. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Eb2a51LciCsyY35sqDoGNe * admin: fix fieldset/legend styling in the structured policy editor Bootstrap's form reset stretches <legend> to the fieldset's full width (float: left; width: 100%), which loses the native "notch in the border" look and makes each section's label bar as wide as the card. Add two scoped classes: .policy-stmt-fieldset (border, rounded corners, spacing between sections) and .policy-stmt-legend (undoes the float/width so the legend hugs its content, with a little padding). Applied to the three per-statement sections (Actions, Resource/NotResource, Principal/NotPrincipal), replacing the ad hoc "border rounded" utility classes that were doubling up with the fieldset's own border. Also gave the "Advanced fields" <details> a small top margin to match the new spacing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Eb2a51LciCsyY35sqDoGNe * admin: suggest bucket/* alongside the bucket itself in Resource autocomplete At the bucket-name stage of the Resource field's progressive autocomplete, only "arn:aws:s3:::bucket" was offered. Add "arn:aws:s3:::bucket/*" right alongside it, since granting access to everything in a bucket is the more common case and previously required typing a "/" first to reach the folder-level "*" suggestion. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Eb2a51LciCsyY35sqDoGNe * admin: keep an unparseable policy in the JSON tab instead of wiping it editPolicy() built the structured state inside the fetch .then, so a policy the editor cannot model threw into the sibling .catch, which alerted and called hide(). Showing the alert at that moment left the modal on screen with an empty editor and the document only in the JSON tab, and Save Changes then serialized the empty state over the policy. Reachable two ways, since neither create path rejects these: the admin API never validates on create, so "Effect":"allow" is stored as typed, and policy_engine.validateStatement lets Resource and NotResource sit in the same statement. Hand the document to the JSON tab instead, which is what that tab is for, and mark the state so nothing serializes the placeholder over it. * admin: validate a policy document before saving it Validation was wired only to the Validate button, so nothing stopped a document the server's own validator rejects from being stored. With the structured editor supplying the boilerplate and required dropped from the textarea, opening the modal, typing a name and clicking Create Policy was enough to save a statement-less policy. Share validatePolicyJSON with the two save paths and abort on failure. * admin: bound the folder autocomplete listing maxListFoldersEntries caps the folders collected, but nothing capped the entries paged through to find them, so a bucket holding only flat object keys - no subfolders to count - was walked to the end, 200 entries per round trip, behind one keystroke. Measured against an in-process filer: 6 entries 0.5ms, 3k entries 7.7ms, 30k entries 53ms, all of it linear in the directory rather than in the answer. Cap the scan as well, and let GetFileBrowser take a prefix so the segment the user is still typing is filtered by the filer instead of by paging. The same 30k directory now answers in 0.6ms once a prefix is typed. * admin: clean the path before scoping list-folders to /buckets util.CleanWindowsPath only rewrites backslashes, so "/buckets/../etc" walked straight past the prefix check the endpoint relies on for its scope. Nothing leaked - filer paths are literal keys, so the traversal resolved to nothing - but the check reads as a boundary and wasn't one, and the test asserting it didn't cover the one input that would try. validateAndCleanFilePath in the same file already does this. * admin: stringify policy values before escaping them escapeHtml calls text.replace directly, and the Sid, the per-item action and resource inputs, and the View modal's Resource/NotResource all pass values straight out of JSON.parse. A policy carrying "Sid": 5 or "Action": [1] threw "text.replace is not a function" and took the render with it. escapedJoin already coerced; use it everywhere and coerce the editor state at the point it's built. * admin: only show the NotResource hint in NotResource mode The hint rendered unconditionally, so it sat under a selector reading "Resource" telling the user the statement applies to everything except what they'd listed. Redraw the card when the selector changes so it follows the mode. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chris.lu@gmail.com>	2026-08-22 12:33:08 -07:00
4 people	df93d01c06	admin: add bucket lifecycle rule editing (#10860) ... * admin: add bucket lifecycle rule editing * address greptile's comments * more small fixes * coderabbit's comments * more comment fixes * more fixes * more * maybe last * last ? * 14850 * 14851 * filer: stamp the content MD5 on every SaveInsideFiler write An entry's ETag falls back to Attributes.Md5, so conditional writers key IF_ETAG_MATCH off it. SaveInsideFiler carried the looked-up attributes forward without refreshing the hash, leaving it describing whatever the previous writer stored: a later conditional write matched the stale hash and overwrote content that had already changed. * s3api: give the bucket lifecycle constants and the write route key one definition each The extended-attribute keys, the XML size cap and the object-write ring key prefix were each spelled out in two places, so the admin dashboard's copies could drift from the gateway's. Move them to the packages both sides already import and alias them where the short local name reads better. * admin: patch the bucket entry's lifecycle keys instead of rewriting the entry The save read the bucket entry, edited its extended map and wrote the whole entry back, guarded by IF_UNMODIFIED_SINCE. Nothing that writes a bucket entry advances its mtime - not the S3 gateway's patchBucketEntry, not SetBucketOwner, not SetBucketQuota - so the guard never fired and the stale snapshot reverted whatever else had changed since the lookup. Send the PATCH_EXTENDED mutation the S3 gateway already uses for these keys: the filer re-reads and merges under the bucket path lock, so only the two lifecycle keys move. That removes the reason for the mtime snapshot, the verification retry loop and the compensating restore of the cleared day-TTL rules, which the migration now logs instead. * s3api: run the delete-lifecycle day-TTL migration through the shared helper DeleteBucketLifecycleHandler kept its own copy of the read-strip-write sequence the put handler now shares, including a missing return that let a ToText failure persist a truncated filer.conf and write a second response. It also wrote the whole file back unconditionally, reverting any concurrent edit; the shared helper writes conditionally. * admin: answer 404 when a lifecycle request names a bucket that does not exist Every SetBucketLifecycle failure came back as 500, including the lookup miss for an unknown bucket, so a client or monitor read a caller error as a server fault and retried it. * s3api: emit lifecycle XML a client would recognize Two changes to what MarshalCanonical writes, both visible through GetBucketLifecycleConfiguration, which replays the stored bytes verbatim: stamp the S3 namespace on the root, and put a size range under <And>. A <Filter> carries one predicate, so two size bounds side by side is a shape AWS does not document. Parsing still accepts either. * admin: fix the lifecycle editor's handling of stored status, deletes and empty saves Four things the editor got wrong: A stored <Status> the S3 API never validated, say 'enabled', left both radio buttons unchecked, so reading the form threw on a null querySelector result and Save did nothing. Collapse anything but an exact 'Enabled' to 'Disabled', which is what the engine already does with it. Deleting a rule re-rendered an open edit form from the snapshot taken when editing began, discarding what had been typed; every other transition folds the form in first. The Transition warning only matched a bare <Transition>, missing the form with attributes, self-closed or namespace-prefixed. Saving an emptied rule list clears the configuration through a path with no prompt, next to a Delete-all-rules button that asks. Also collapses the three divergent copies of formatBytes on this page to one. * filer: stop the day-TTL migration from deleting an operator's path rule The migration removed every rule under the bucket's path that carried a day TTL in the bucket's collection. The add path it is retiring used AddLocationConf, which merged its TTL onto whatever already sat at the prefix, so a rule can hold operator settings the lifecycle path never wrote - a disk type, WORM retention, a read-only flag, a placement pin. Deleting the whole rule to retire its TTL took those with it, leaving objects under that prefix on defaults nobody asked for. Delete only rules shaped like ones the add path created from scratch; anything else keeps its settings and loses just the TTL. --------- Co-authored-by: Chris Lu <chris.lu@gmail.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	2026-08-21 23:42:26 -07:00
Chris LuandGitHub	8c7d714d5e	Lance catalog, and a Rust plugin worker to maintain it (#10841) ... * iceberg: skip tables the maintenance worker does not own A Lance dataset registered through the Lance namespace's Iceberg REST adapter arrives as an Iceberg table with a placeholder schema and table_type=lance, and keeps its fragments under data/ - the same subdirectory the orphan cleaner walks. Every fragment is unreferenced by the Iceberg metadata, so a maintenance pass deletes the dataset. Views share the entry shape and were only skipped because parsing their metadata happened to fail first. Gate the scan and the execution path on the entry actually being an Iceberg table. Maintenance is off by default, so this was latent rather than live. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * s3tables: let a table declare a format the catalog does not interpret CreateTable accepted ICEBERG and nothing else. A Lance table has no metadata file for the catalog to maintain - the entry records a name and the dataset root, and the client owns everything under it - so accept LANCE, and carry the declared format on the entry instead of hardcoding it back on the way out. ListTables now reports format and metadataLocation, so listing a catalog that holds both kinds takes one pass rather than a GetTable per row. AWS omits both fields; adding them is additive. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * s3tables: move the in-memory filer into its own package The Lance namespace tests need the same harness, and copying it would leave two of them to keep in step. Extracted as it was, plus the two fidelity gaps that only surface once a paginating caller uses it: ListEntries ignored startFromFileName and limit, so a caller that paginates re-read the first page until it hit its own cap and reported the same entry over and over, and GetFilerConfiguration was missing, which CreateTableBucket needs to resolve the buckets directory. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance: serve the Lance Namespace REST spec A second catalog surface beside the Iceberg one, over the same table buckets: the namespace and table metadata operations, the $-delimited identifier codec, the spec's numeric error model, the directory-catalog marker files, and storage_options vending through the STS path the Iceberg catalog already uses. Listens on -port.lance, 9101 by default, and inherits ARNs, policies and tags from the storage layer, so a Lance table needs no second permission model. Identifiers map bucket / namespace / table onto the three levels Lance clients already use, which is why there is no warehouse selector to invent. The data plane needs Lance format support that does not exist in Go and answers with the spec's Unsupported code rather than a bare 404. Two things it deliberately will not do: create a table bucket as a side effect of creating a namespace inside one, since a bucket carries its own policy and lifecycle, and resolve an Iceberg table's location for a Lance client, which would hand it a table another engine owns. The design note this follows is in design-lance-catalog.md, including the .lance directory suffix it proposed and this does not implement. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * mini: give the Lance port the same treatment as the Iceberg one The flag was registered but nothing else knew about it, so mini would start the server without reserving its port, waiting for it, or saying where it is. Adds it to the startup service list, the conflict resolver, the gRPC allocator's reserved set, the readiness wait, the stop reporting and the banner. The admin server still takes only the Iceberg port, because there is no Lance page for it to link to. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance: stop deregister and repoint from deleting the dataset Deregistering preserves data by definition, and this did the opposite: the catalog entry is the dataset directory, so DeleteTable took the files with it. Registering over an existing name had the same shape, destroying the dataset the name used to hold. Found by driving the running server rather than the in-memory filer, where both looked like success because the table did stop being listed. Deregistering is now a state on the entry - the marker file hides it, and declaring or registering the name again brings it back. Repointing a name at another dataset is an UpdateTable against the version token, so neither dataset loses files. Drop is left alone; it is the operation that does remove data. The storage endpoint now falls back to the advertised -ip where the Iceberg derivation gives up. An Iceberg client brings its own s3.endpoint and advertising the wrong one hijacks it, but storage_options is the only place a Lance client learns where the store is, and without it object_store quietly talks to real AWS. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * s3tables: refuse to create a table over one of another format Creating a table that already exists is idempotent, and that path returned the existing table without looking at its format. A Lance declare over an Iceberg table answered 200 and handed back a directory Iceberg owns, so the client would write its dataset on top. The view check immediately above it already guards the same class of collision. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * s3tables: let a table bucket hold a format other than Iceberg The S3 door validated every object written into a table bucket against Iceberg's file layout, so a Lance client could not write its dataset at all: it got 403 on data/*.lance, on _versions/, and on the _transactions/ directory it turned out to write as well. Table buckets were only neutral containers by intention; in practice they were Iceberg-shaped and enforced as such. The allowed set is now the union of what the supported formats write, because the validator runs where the table's format is not in hand. Underscore-prefixed directories are treated as belonging to the format, since enumerating them means guessing at the next one - _transactions is exactly the one this missed - and their contents are checked only for traversal. Iceberg writes none of them, so it loses nothing. Marker files at the table root are admitted too, which the namespace/table/dir/file shape had rejected as too shallow. Describe also honours the request-body spellings of with_table_uri, load_detailed_metadata and check_declared. The spec puts them in the query string, but real clients send both. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * design: record what the implementation found The table bucket being an Iceberg-shaped container, enforced at the S3 door, was the premise this design never questioned and the one that had to change before anything worked end to end. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * iceberg: prove the data loss the foreign-format guard prevents The guard landed with a unit test for the predicate and nothing showing what it saves. These seed what the Lance namespace's Iceberg REST adapter actually leaves behind - an Iceberg table with a placeholder schema and table_type=lance whose directory holds a Lance dataset - and assert both halves: orphan collection does flag the dataset's fragments, because the Iceberg metadata beside them references nothing, and the scan never reaches the table. An ordinary Iceberg table in the same shape is still scanned, so the guard is not just skipping everything. Confirmed against a running gateway first: our Iceberg catalog accepts the adapter's registration, and a real Lance client then writes a dataset into that table's location. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * s3tablestest: make the in-memory filer safe to race against Two gaps that only matter once a test drives concurrent writers, which is what an exclusive create has to be tested with: the entry map had no lock, and CreateEntry ignored O_EXCL entirely, so both writers of the same name would have won and the test would have passed while proving nothing. The BeforeUpdate hook runs before the lock is taken. Its whole purpose is to land a competing write in a handler's read-to-write window, and that write needs the lock the hook would otherwise be holding. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance: make the namespace an external manifest store Lance commits a version by writing _versions/{v}.manifest with put-if-not-exists. The S3 layer in front of this same filer evaluates If-None-Match by looking the entry up and then writing without a precondition, so two writers can both pass the check and one commit is lost. The filer itself has the primitive: CreateEntry with o_excl. Adds the four version operations a Lance client actually calls - create, list, describe and batch-delete - recording one entry per version under _lance_versions/, and advertises managed_versioning so the client routes its commits here. Reserving a version is the exclusive create, so exactly one of several racing writers wins and the rest rebase. Off by default, behind -lance.managedVersioning. Turning it on moves where a table's version history lives, and a reader that does not come through this namespace no longer sees all of it; that is the operator's call, not a default. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * design: record what managed versioning does and does not reach The first commit through a namespace-backed store works and is recorded the way the protocol specifies. Later commits do not, because lance 4.0.0 refuses put_if_exists on that path in its own code, so the feature is capped upstream rather than here. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: integration tests for the Lance namespace Everything this surface got wrong so far - a deregister that deleted the dataset, an S3 door that refused every Lance file, a version reservation that could not actually be exclusive - passed against an in-memory filer first. So these run against a live gateway, and where the claim is about data they check storage rather than visibility. Five Go tests on the shared harness: namespace and table lifecycle including that deregister keeps the bytes and drop removes them, that a Lance client cannot resolve or declare over an Iceberg table, that a Lance dataset's files get past the table-bucket layout guard while junk still does not, and that eight writers racing for one version produce exactly one winner. One Docker-gated test drives the real Lance client, which is the only way to check that the location and storage_options the namespace vends are between them enough to write and read a dataset. It overrides the endpoint with the container's view of the same gateway, because the shared harness binds a wildcard address and so vends none. The harness gains a Lance port and turns managed versioning on; the flag touches nothing outside that surface. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * s3tables: a directory with no namespace metadata is a missing namespace Three callers resolved a namespace by reading its metadata attribute and each tested only for a missing entry, so a directory that carried no metadata came back as an internal error saying "attribute not found". Creating a table under a namespace that does not exist answered 500. Collapses the three copies into one helper that reports both conditions as absent, which is what they are: a directory without namespace metadata is not a namespace. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * iceberg: stop reporting storage-layer refusals as server faults writeManagerError recognised a missing table bucket and sent everything else to 500, so a missing namespace, a duplicate name and a commit conflict all reached the client as InternalServerError with nothing to act on. Creating a table in a namespace that does not exist is the case that turned up: 500 where the spec wants 404 NoSuchNamespaceException. Maps the storage error types onto the exception names this package already uses, and keeps the existing bucket message, which explains how to select a table bucket. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * iceberg: skip a foreign-format table by name, not by failing to parse it A table the namespace created as LANCE carries no Iceberg metadata, so the worker skipped it only because the parse failed, and logged that as damaged metadata. The catalog records the format on the entry and this never read it. Reading it turns an accident into a decision, and separates a mixed catalog from a corrupt one in the logs. The property check beside it still covers the other shape: a real Iceberg table wearing table_type=lance, which is what the Lance namespace's Iceberg REST adapter writes. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * design: answer whether a Lance table needs maintenance It does, and index optimization has no Iceberg equivalent: rows written after an index was built are not covered by it, so a vector search quietly misses them. None of the three jobs can run in the Go worker, and there is no useful subset, because deciding what an old version still references means parsing Lance manifests. Version cleanup at least has an answer that needs nothing from us - Lance can enable it on the dataset itself. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * design: the Lance maintenance worker is a plugin worker, in Rust Framing it as a sidecar was wrong. plugin.proto already defines a language-agnostic gRPC contract for external maintenance workers, and "weed worker -admin=..." is the Go reference implementation of it from outside the admin process. seaweed-volume already compiles protos out of weed/pb with tonic_build, so a Lance worker is that build plus plugin.proto and the lance crate. Scheduling, retries, dedupe, progress and the admin settings page all come from the protocol: a worker that answers RequestConfigSchema with a descriptor gets its configuration form rendered without a line of Go. The data plane is the part that genuinely does need a process answering HTTP, and this had the two conflated. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * seaweed-worker: Rust plugin worker workspace, with Lance as the first one plugin.proto is language-agnostic and the Rust toolchain was already in the tree, so a Lance maintenance worker needs no new integration surface: core is the contract and nothing else, and a worker crate beside it supplies handlers and a binary. A second worker is a new member here rather than a fork of the protocol, which is why this is seaweed-worker and not seaweed-lance-worker. Verified against a running admin: it connects, is accepted, and admin prefetches descriptors for lance_compact, lance_optimize_indices and lance_cleanup_versions, so their settings pages render from the Rust side without a line of Go. The stream stays up across heartbeats. The job bodies are stubs that report failure. Doing the work means adding the lance crate and opening the dataset, and claiming success before that would be worse than saying so. Two things running it caught that reading the proto did not: the admin address has to be converted to the gRPC port the way pb.ServerToGrpcAddress does, or the dial fails as an h2 frame error; and the generated field names differ from the Go ones in several places, so JobCompleted carries success rather than a state enum. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance worker: implement compaction Detection lists tables from the namespace, opens each one, and proposes a job for any with more fragments than the policy allows; opening a dataset reads its manifest and not its data, so a sweep stays cheap. Execution re-resolves the table rather than trusting what detection saw - it may have been repointed, and the vended credentials expire - then compacts and reports the fragment counts either side. Verified against a live gateway: a twelve-fragment dataset became one fragment with all twelve rows intact. The test drives the handler directly and skips unless WEED_LANCE_NAMESPACE names a namespace, the way the Go integration tests skip without Docker. Running it turned up a gap the design had not: a gateway without STS vends no credentials at all, so the worker could not open anything and detection quietly proposed nothing. --access-key/--secret-key are the fallback, and whatever the namespace vends still wins over them. Two API assumptions did not survive contact either. Datasets open through DatasetBuilder::with_storage_options, not ReadParams, and lance 10's ObjectStoreParams has no storage_options field at all. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance worker: implement index optimization and version cleanup Index optimization is the job with no Iceberg equivalent: rows appended after an index was built are invisible to a search of it until this runs. Detection reads num_unindexed_rows from each index's statistics and proposes a table once more rows sit outside its indices than the budget allows; a table with no indices is skipped, which is different from one whose indices have fallen behind. Cleanup applies a retention window, refusing rather than silently dropping a tagged version, and leaving unverified files alone because they may belong to a commit still in flight. Both verified against a live gateway: 512 uncovered rows became 0, and a fourteen-version table lost its old ones. Each test now seeds what it needs, including building an IVF_PQ index and appending rows outside it. The first version of these depended on state a script had left, so the second run found the work already done and asserted nothing - a test that passes by doing nothing is worse than no test. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance: answer an empty catalog with an empty list, not null ListAllTables built its result from a nil slice, so a namespace holding no tables answered {"tables":null} on a field the spec marks required. A generated client may decode that differently from an empty list. Found running the namespace on a dev box, where the catalog was empty. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * admin: give Lance maintenance its own scheduler lane Lane assignment is a hardcoded map, so the three lance_* job types fell through to the default lane. That lane serialises its work under the cluster admin lock because volume management shares global state, which would queue a table's compaction behind volume balancing for no reason - Iceberg has its own lock-free lane for exactly this. Adds the lane, maps the three job types to it, and puts it in the sidebar beside Iceberg and Lifecycle. The lane routes were already generic, so only the nav was hand-written. The lane-coverage test spelled out the three known lanes, so a fourth failed it. It now checks against AllLanes(), which is the property it was reaching for and does not need editing next time. Found by connecting the Rust worker to a real admin: it registered fine and its job types were known, but they were filed under "default" and had no page. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance worker: log what detection saw "Detection proposed nothing" and "the worker could not read the table" look identical from the admin side, and the second is what a missing credential produces. One line per table separates them. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance worker: fix a leaked heartbeat and a silent reconnect loop spawn_heartbeat returned a handle to an empty task rather than the ticker it had just spawned, so aborting it aborted nothing and every reconnect left another heartbeat running against a dead channel. A stream that admin closes cleanly is not an error, but reconnecting in silence hides why. Two workers sharing an id evict each other forever and the log shows nothing but a login every five seconds - which is exactly how this presented on a dev box, and it took a look at the admin's own log to see it. The message now names the id to check. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance: a namespace cannot be created without its parent Storage keeps a namespace's parts flattened, so creating "a.b" with no "a" was accepted and left an intermediate that only existed inside a name. Listing derives child names by slicing those parts, so it reported "a", while describe and exists on "a" both answered 404 - a client walking the tree got a 404 on something the listing had just handed it. The spec asks for NamespaceNotFound when the parent is missing, which is also what keeps listing and describe telling the same story. Namespaces created through the S3 Tables API still bypass this, so listing keeps deriving intermediates rather than hiding whatever is already there. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * admin: say why a non-Iceberg table shows no schema The table pages read Iceberg metadata for schema and snapshots, and a Lance table has none, so both panels rendered "No schema available" - which reads as an empty table rather than a table this page cannot describe. The dataset behind the one that prompted this holds 1024 rows. The format is already on the entry and shown two rows above, so the empty states now use it: the catalog records where a LANCE table lives, not what is in it. Reading the schema for real needs Lance format code, which is the same wall as the data plane. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * seaweed-worker: run rustfmt over the workspace Committed the crates unformatted, so `cargo fmt --all --check` failed on files nothing had touched since. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * plugin: let a worker report what it saw about an object Admin cannot read a Lance table: it knows where the dataset lives and nothing else, so the details page had a location and two empty panels. The worker already opens every dataset during detection to decide whether it needs compacting, so it knows the schema, the row count and the fragment count at that moment. It just had no way to say so. Add a WorkerObservations body to the worker stream. Admin caches the last observation per object and serves it back, timestamped, for display; nothing schedules from it. The Lance compaction sweep reports what it opened, and the S3 Tables details page fills its schema panel from the cache when it has no metadata of its own, badged with when the worker looked and which worker it was. Nothing about this is Lance-specific past the reporting side, which is the point: any format admin cannot parse can describe itself the same way. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * design: record the observation channel Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * plugin: ask a worker for sample rows of a table admin cannot read Browse Data reads an Iceberg table's Parquet files directly, so it shows real rows. For a Lance table it showed "Table has no Iceberg metadata" and an empty grid, because there is no Go Lance reader and never will be one worth maintaining. The worker has the reader. Add RequestObjectPreview / ObjectPreviewResponse to the stream, mirroring the config-schema round trip that already exists, and give the Rust worker a PreviewProvider that scans the dataset and formats the rows with Arrow's own formatter, so a vector column reads as a vector. Admin picks the worker from the observation store: whichever one last described this table is the one that can read it. Unlike an observation the rows are not cached. They are the table's data rather than a description of it, and a copy sitting in admin would be both stale and nobody's business. The page fetches on load, bounded at 200 rows and a 15 second round trip, and drops the snapshot and data-file panels that only mean something for Iceberg. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * design: record the preview channel Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: disable the lance listener when two gateways share a host * test: keep AllocatePorts away from the lance default port * s3tables: let a table bucket declare the format it holds A bucket is a catalog, and a catalog serves one protocol. Format was recorded per table, so nothing could answer "where do I point a client at this bucket" without opening a table first, and an empty bucket had no answer at all. CreateTableBucket takes an optional format, stored with the rest of the bucket metadata and returned by Get and List. Empty means ICEBERG, which is what AWS S3 Tables serves and therefore what an SDK that has never heard of the field means. CreateTable refuses a table of another format, and CreateView refuses outright in a bucket that is not Iceberg, since a view is Iceberg metadata. Buckets that already exist carry no declaration and keep accepting anything, so nothing is migrated and nothing that worked stops working. The Lance namespace declares LANCE for the buckets it creates, which is what stops one of them being described to a client as an Iceberg catalog. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * admin: take the Lance port the way it takes the Iceberg one The UI cannot name the endpoint that serves a Lance bucket without it, and every format-aware page below needs to. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * admin: show which format a table bucket holds The bucket list printed an Iceberg endpoint for every bucket, including ones holding Lance datasets, where that endpoint serves nothing. It was the most visible place the UI assumed one format. The list gains a Format column and its endpoint column follows the bucket's declaration. The banner names both endpoints rather than asserting everything is Iceberg, and says so only for the servers that are actually running. Create Bucket picks a format with two cards rather than a dropdown, since what matters is not the name but which clients can read the result, and the endpoint under them updates as you choose so the operator leaves the modal knowing where to point one. A bucket from before the declaration existed shows "unset" in an outline badge, explained on hover. It is a fact about the bucket's age, not a fault, so nothing nags about it. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * admin: carry the bucket's format into the pages inside it Namespaces and tables are reached through a bucket, so both now say which catalog they belong to rather than making you go back up to find out. The tables list gains a Format column and a Rows column filled from what a worker last observed, since for a format admin cannot read that is the only row count there is; a table nothing has looked at shows a dash, not a zero. Create Table stops offering a choice the bucket has already made: in a declared bucket the format is fixed and says why, and only an undeclared one still offers both. Before this the select had exactly one option, hardcoded, which made a Lance table impossible to create from the UI at all. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * admin: let the table page speak the table's own format Partitions and Snapshot History are Iceberg's shape. Rendering them empty for a Lance table reads as a fault; a Lance table has neither, and says so by not showing them. In their place is a Versions panel, which is what that format calls its history, carrying the worker's timestamp so it is clear the numbers are a cached look rather than something read live. The breadcrumb carries the format badge, so the page names what it is looking at before you read a panel and wonder why it is empty. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * admin: show how to connect to either catalog, and group the two format workers The client examples on the buckets page were Iceberg's alone, so the one thing an operator wants after creating a Lance bucket - what to type to reach it - was not written down anywhere in the UI. Both formats now get a pair of snippets, and only for a server that is running. In the Workers menu, Iceberg moves below Lifecycle so it sits next to Lance: the two table-format workers together, the two cluster-wide ones above them. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * shell: create a table bucket of either format s3tables.bucket -create takes -format, so a Lance bucket can be made without going through the UI. The integration harness passes it too: its Lance tests were creating Iceberg buckets and getting away with it only because nothing checked. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * design: record that a bucket declares its format Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance: drop managed versioning; the store already orders commits The namespace offered itself as an external manifest store, so that a commit could reserve a version through a real put-if-not-exists. That was designed around a gateway that no longer exists: If-None-Match: * is reduced to a filer WriteCondition and evaluated at the object's owner under its per-path lock, or under the object write lock on the fallback path. Sixteen writers racing one fresh key get a single 200 and fifteen 412s, every time. Lance needs nothing else. commit_handler_from_url hands every s3:// dataset a ConditionalPutCommitHandler, which puts with PutMode::Create, which object_store sends as If-None-Match: *. So the feature solved a problem this store does not have, while moving a table's version history out of the dataset and into the catalog - and lance could not use it past the first commit anyway, since its own namespace-backed store answers "put_if_not_exists is not supported" to the second. The version operations answer Unsupported with the rest, managed_versioning is false, and the flag is gone. In place of the reserve-once test there is one that races eight writers at the manifest key through S3, which is the path a commit actually takes. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance worker: honour the version floor, the slot limits, and a shutdown Five findings from review, all of them things the worker claimed to do and did not. The version floor was checked when a cleanup job was proposed and ignored when it ran, so a table whose versions had aged past the retention window in between could be taken below the count the operator asked to keep. Execution now computes the floor itself and passes it as before_version; CleanupPolicy ANDs its clauses, so a version has to be both too old and below the floor to go. Both settings are clamped to the range the form offers, since Duration::hours panics on a large enough value and a negative min-versions wraps to a huge usize. Admin's shutdown was answered by returning from the stream, which the reconnect loop read as a healthy close and logged straight back in: the worker could not be stopped. serve_once now says which of the two happened. The advertised concurrency limits bounded nothing - every request spawned a task - and the heartbeat reported zero slots in use whatever was running. Both now go through semaphores sized from the limits, with the permits held for the life of the request and reported in the heartbeat. A namespace call had no timeout, so a gateway that accepted the connection and went quiet held a detection slot forever. And one table whose stats could not be read failed the whole sweep, losing the proposals for every table already scanned; it is now skipped and warned about, like a table that cannot be opened. The tests drove one shared catalog concurrently, which is why one of them asserted "no proposals at all" and passed by luck. They now take a lock and judge only their own tables. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * admin: fix the review findings on the format-aware pages The endpoint hint in Create Bucket built its HTML by concatenating the bucket name the operator is typing, so a name like <img onerror=...> ran in the admin origin as they typed it. It is built from DOM nodes now. A preview reply looked its channel up under the lock and then sent outside it, which Shutdown can close in between: a Gosched in that gap panics with "send on closed channel" every time. The send now happens under the lock. Observations were looked up by path alone, so a table dropped and remade in another format at the same path was described by the observation left behind. Lookups now have to agree on the format. Also: the Lance namespace caps a request body rather than reading whatever arrives; the details action no longer says "Iceberg" over a Lance table; mini stops advertising a catalog port when it is not running S3; a format whose server this cluster does not run cannot be picked in the modal or accepted by the API, since a bucket nothing can reach is not worth creating; and the unused catalogPortFor helper is gone. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance worker: let the control stream use mTLS The channel was hardcoded to http://, so off loopback the stream carried preview rows and execution commands in the clear - and a cluster with grpc TLS turned on would refuse the worker outright. --tls-ca, --tls-cert and --tls-key take the same certificates the Go worker reads from the [grpc.worker] section of security.toml, and must be given together: a CA on its own would quietly mean one-way TLS, which a mutual setup rejects anyway. Without them the stream stays plaintext, which is what the Go worker also does when nothing is configured. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance: answer null properties rather than an empty map The catalog does not keep a table's properties. Declare echoed the request's back and describe answered {}, both of which claim they were stored and are empty. Null says the catalog does not keep them, which is what the spec distinguishes and what is true here. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance worker: test the slot accounting The heartbeat reporting and the waiting are the two things the semaphores are for, and neither is observable from outside without catching a sweep mid-flight. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: fix the mixed-format catalog test, and name the binary it drives The integration suite passed locally and failed in CI on TestLanceRefusesIcebergTables. Both were right: CI builds the binary first, my tree had one from the day before, so locally the test drove a gateway with no format enforcement at all. The test itself no longer holds as written. It made a bucket, put an Iceberg table in it, and checked the Lance surface hid it - but a bucket that declares LANCE now refuses the Iceberg table outright. The invariant still matters from the other side, so it starts from an Iceberg bucket instead: Lance must not describe or list a table whose format it does not serve, and must refuse to declare one beside it. The harness now prints which weed binary it is about to run and when that was built. `make test` rebuilds first; a plain `go test` will happily drive a weeks-old binary and report a pass for code it never ran, which is exactly what happened here. Also make the row-limit conversion in the preview request explicitly bounded: CodeQL flagged the int-to-int32 conversion, and clamping by reassignment beforehand is not a form it recognises. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * lance: prove concurrent commits are kept, and preselect the only format on offer Two more from review. The commit test asserted that exactly one writer wins the conditional PUT, which is the mechanism, not the claim. The claim is that nothing is lost: the losers see the conflict, rebase and commit again. So there is now a test that has eight writers append to one dataset at once and counts the rows afterwards - all eight batches survive. That is also the sequence managed versioning could not finish, since its store refuses the second commit outright. And when Iceberg's endpoint is not running, the format picker offered two options with neither selected, so Create Bucket submitted no format at all, fell back to ICEBERG, and was refused by the guard added last round. Lance is preselected when it is the only format this cluster serves. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * Clamp the remaining worker settings, and bootstrap buckets in a served format Compaction and index optimization read their thresholds and cast straight to usize and u64, so a negative arrives as an enormous number and turns the threshold into "never": compaction and reindexing both go quiet with nothing to say. The cleanup job was fixed last round; these are the same bug. Clamped to the values that stay meaningful rather than to what the form offers - zero uncovered rows is a real setting, meaning reindex as soon as anything is not covered, so the floor there is zero and not the form's thousand. mini pre-creates the buckets named by -tableBucket, and did so without a format, which now means Iceberg. Started with the Iceberg endpoint off and the Lance one on, that left buckets nothing could reach and which refused every Lance table. It takes the format from the endpoint that is actually running, and creates nothing when neither is. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * s3: allow-unordered is a listing parameter, not an unimplemented subresource The guard that stops a bucket GET with an unknown subresource from being answered with a listing does not know about allow-unordered, so it answers 501 NotImplemented - to a parameter the listing handlers already read and already validate against delimiter. This is why test_bucket_list_unordered and test_bucket_listv2_unordered fail in the Ceph s3-tests suite. They fail on master too; this is not a Lance change and can be taken on its own. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm	2026-08-19 22:59:56 -07:00
Chris LuandGitHub	97a155d14d	admin: show capacity per storage tier and stop counting remote-tiered bytes as local disk usage (#10766) ... * admin: show capacity per storage tier and stop counting remote-tiered bytes as local disk usage A remote-tiered volume reports its cloud object's size, so summing volume sizes inflated the dashboard's used-vs-capacity numbers (the local .dat is gone after volume.tier.move). Split the accounting: DiskUsage now only counts bytes on local disks, with the cloud bytes surfaced separately per server and per remote storage name. The dashboard gains a Storage Tiers table breaking volumes and EC shards down by tier (each local disk type plus each remote storage), using the per-disk-type statfs numbers already in the VolumeList response. The volumes page badges remote-tiered volumes with their storage name, and the EC shards page fills in real per-shard sizes instead of hardcoding 0. * admin: review fixes for the tier capacity display - A disk that predates disk_total_bytes now contributes its logical bytes to the tier's DiskUsed, so a tier mixing old and new volume servers doesn't underreport usage; the usage bar always reflects the displayed Disk Used value (the DataSize fallback in UsagePercent is gone, and the percent math is overflow-safe). - getTopologyViaGRPC defaults a zero VolumeSizeLimitMb to 30000 MB like GetClusterVolumeServers, keeping slot-based capacities consistent. - The dashboard volume-servers column reads Usage / Capacity to match its cell content, and the hdd disk-type default is shared between the volumes-page badge and countUniqueDiskTypes.	2026-08-15 12:35:16 -07:00
Chris LuandGitHub	52d74df4d1	clients: stream the volume listings that ask for everything (#10679) ... * master: stream volume listings A listing of 800k volumes is 36MB on the wire but 305MB as messages, and the master built all of it, then held it while grpc encoded it. Two of those at once is most of a small master's heap, and the maintenance scanner asks every 30 minutes. The topology goes out first, listing nothing, then its volumes in batches, so the master holds a batch rather than a cluster: 341MB of live heap for one listing becomes 4.4MB. It allocates much the same either way -- what changes is how much of it has to be live at once, which is what sets the heap ceiling. Batches are built under their disk's lock and sent outside it, so a slow reader stalls the stream rather than the topology. They therefore do not share one instant, which a single listing did not either: it takes each disk's lock in turn, so a volume moving during either can be seen twice or not at all. The client helper hides which kind of master answered: one too old for the stream is asked the old way and its reply cut into the same batches. Either way the topology handed over lists no volumes, so a caller cannot come to depend on finding them there. * admin: stream the listing the maintenance scan reads It asks for every volume in the cluster every 30 minutes. Reassembling it client-side keeps the scan identical -- ActiveTopology splits disks by the disk ids on the volumes, so it needs them in the topology -- while the master no longer builds the whole reply to send it. * topology: report a disk id that does not depend on map order A topology disk that fronts several physical disks took its reported id from whichever volume the map yielded first, so two listings of an unchanged disk could disagree. Take the smallest instead. * topology: test that a streamed listing rebuilds to the whole one The callers that stream now rebuild the listing from a topology sent without volumes plus the batches after it, so that has to come out the same as being sent it whole, at every batch size and under a filter. * clients: stream the volume listings that ask for everything The dashboard's list and export pages, the collection and ec shard pages, the topology view, the worker metrics and two shell commands each asked the master to build all 800k volumes into one reply. They read the same listing as before, rebuilt on their side, so the master no longer holds it. The three that already ask for one volume or one collection stay as they are: their replies are small, and streaming one costs a round trip to say so.	2026-08-10 09:51:08 -07:00
Chris LuandGitHub	a2ff9cca27	master: let VolumeList ask for the volumes it wants (#10674) ... * master: let VolumeList ask for the volumes it wants The request carried nothing, so every caller was answered with the whole cluster. A dashboard opening one volume's page, or a capacity probe adding up one bucket, was served all 800k of them and threw away the rest -- and the master built every one of those messages first. The topology, its disks and their counters are still reported in full: a caller reading free space or replica placement needs the cluster whichever volumes it asked about. Only what is listed under a disk is selected, ec shards included. An empty collection and a zero volume id take everything, the way volume.list already reads its own -collectionPattern and -volumeId, so a caller that forgets to narrow is answered too much rather than answered wrongly. That leaves the default collection unnameable, since it is the one the empty string names, so it gets a field of its own. An older client sends none of it and is answered exactly as before. * admin: ask the master for the volume the page is showing A volume's detail page was pulling every volume in the cluster to find one and its replicas, and discarding the rest. * admin: ask the master for the ec volume the page is showing Same as the volume detail page: one volume's shards were found by pulling every ec shard in the cluster. * s3: ask the master for the bucket's own collection The SOSAPI capacity probe summed one collection's volumes out of a listing of every volume in the cluster. Cluster capacity still comes out the same: it is read from the disk counters, which a filtered listing reports in full. * topology: read the disk usage counters atomically They are written with atomic.AddInt64 from heartbeats but were read plainly by the two listings and by FreeSpace, and the map they sit in was iterated without the lock its neighbour takes. Under -race a listing concurrent with a heartbeat trips on both.	2026-08-09 21:59:42 -07:00
Chris LuandGitHub	f09e8345c6	storage: stop keeping the remote storage key on the master (#10672) ... A master decides nothing from it. Every caller that read it was asking whether a volume is remote, which the backend name answers, and the value itself is reported on demand by the server holding the volume, through the volume info in ReadVolumeFileStatus. It is also the one string here that cannot be shared: unique per volume, so unlike the collection and backend names it carries its own characters for every volume a master tracks. VolumeInfo goes from 136 bytes to 120. 800k volumes registered from a heartbeat that has been over the wire go from 214 to 163 B/volume when tiered. The volume server's own status page keeps showing the key, now read from the volume it holds rather than relayed through a master, which is also where the other volume server implementation reads it. The heartbeat digest drops it on the same grounds: a change to something the master does not hold cannot make its copy stale. Both implementations and their shared vectors move together, and the field-coverage test now names what is deliberately not retained rather than being loosened.	2026-08-09 12:43:31 -07:00
Chris LuandGitHub	0cf62a921a	admin: dashboard counts chunks, not files (#10598) ... * admin: count each chunk once in the dashboard total The dashboard summed file_count from every node's volume list, so a chunk was counted once per replica and deleted chunks were never subtracted. Reuse the collection aggregation, which dedupes replicas and EC shard holders and nets out tombstones. * admin: the dashboard card counts chunks, so name it that Volumes store chunks, and a file is split into one or more of them, so the 'Total Files' card always read far higher than the number of files in the filer. Rename it to 'Total Chunks' and say so in the tooltip. * admin: collections pages count chunks once and say so The collections list and detail pages summed file_count straight off the topology, so replicas multiplied the count, tombstones stayed in it, and the detail page ignored EC volumes entirely. Take the numbers from the shared collection aggregation and label them chunks. * admin: dedupe replica chunk counts per volume instead of dividing Dividing each replica's live count by the copy count truncated a chunk per odd-sized volume, and reported half the count while a volume's second replica had not checked in yet. Replicas mirror each other's needles and deletes, so keep the fullest report per volume id. * admin: fix the collections CSV export column mapping The exporter read chunks from the EC-volume cell and shifted size and disk types with it. Read every column the table actually has.	2026-08-06 11:22:06 -07:00
Chris LuandGitHub	b452a5e41b	s3: honor a bucket owner recorded as an identity (#10567) ... * s3: resolve a bucket owner recorded as an identity The admin UI and weed shell record a bucket's owner as an identity name in s3-identity-id and never write the account id the S3 API stores alongside it, so such a bucket looked unowned: its ACL owner fell back to the default admin account, and under the default BucketOwnerEnforced ownership every object uploaded to it was stamped with that account instead of the bucket owner. Resolve the identity to its account when no account id is recorded, in the one place both the bucket metadata and the bucket config derive the owner from. * s3: drop the recorded account when the bucket owner is reassigned Changing the owner of a bucket created through the S3 API left its old account id behind, and that outranks the identity when the owner is resolved, so the new owner never took effect for object ownership or the bucket ACL.	2026-08-04 16:00:04 -07:00
Chris LuandGitHub	4b0d09683a	iceberg: read manifest lists that omit the Avro format version (#10475) ... * s3tables: read Iceberg manifest lists that omit the Avro format version The Iceberg spec pins the Avro header metadata of manifest files but says nothing about manifest lists, so writers disagree. Java and PyIceberg record "format-version"; DuckDB writes no header metadata at all. iceberg-go reads a missing entry as v1, so every v2 manifest listed in a DuckDB-written list is rejected with manifest file's 'format-version' metadata indicates version 2, but entry from manifest list indicates version 1 and, because v1 has no "content" field, delete manifests silently decode as data manifests. ReadManifestList derives the version from the record schema the writer embedded - v2 added "content" and the sequence numbers, v3 added "first_row_id" - and splices it into the header before handing the bytes to iceberg-go. Lists that already carry the entry, and input that is not a parseable Avro container, go through untouched. * iceberg: parse DuckDB-written manifest lists in maintenance and data preview Every manifest list read - the four maintenance operations and the admin table data preview - went straight to iceberg-go, so tables written by DuckDB failed detection and all of compact, remove_orphans, rewrite_manifests and expire_snapshots before they touched anything. Route them through s3tables.ReadManifestList, which recovers the format version the writer left out of the Avro header. This also restores the manifest content type on those tables: with the list read as v1 every delete manifest looked like a data manifest, which hid deletes from the compaction guard and made the preview report a table with position deletes as having none.	2026-07-28 16:42:17 -07:00
5 people	6c4eb95a3a	fix(admin): implement ApplyPluginConfigFromToml to propagate settings (#10388) ... * fix(admin): implement ApplyPluginConfigFromToml to propagate settings to plugin config store * Update weed/admin/dash/config_toml.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * admin: overlay admin.toml onto plugin configs through bootstrap defaults Creating a config from scratch at startup skipped the descriptor-defaults bootstrap, so a job type with only worker keys in admin.toml persisted Enabled=false and RetryLimit=0 and silently stopped running. Overlay existing configs at startup, and apply the same overlay in enrichConfigDefaults when the plugin bootstraps a fresh config from descriptor defaults. Also place collection_filter in the admin values where workers read it, map preferred_tags as a string list, and stamp UpdatedAt. * admin: trim the admin.toml help text and call-site comment * admin: clamp toml retry values to the int32 range * admin: fail startup when admin.toml cannot reach the plugin config The legacy overlay already aborts startup when declared settings cannot persist; continuing here would let workers bootstrap with stale values. * admin: fix the retry clamp test on 32-bit A 32-bit int cannot hold the oversized toml value, so viper returns 0 before the clamp runs. --------- Co-authored-by: baracudaz <baracudaz@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Chris Lu <chris.lu@gmail.com>	2026-07-21 14:00:37 -07:00
Chris LuandGitHub	40ee4a08c5	admin: show bucket lifecycle rules in the Admin UI (#10313) ... * admin: surface lifecycle rule counts in bucket listing * admin: add bucket lifecycle JSON endpoint * admin: show lifecycle rules on the buckets page * admin: make the lifecycle count badge keyboard-accessible * admin: match buckets empty-state colspan to the column count * admin: drop stale lifecycle modal responses * make	2026-07-11 11:31:30 -07:00
Chris LuandGitHub	e7be6bb2f8	filer: allow clearing a bucket read-only flag stuck after quota removal (#10310) ... * filer: allow clearing a bucket read-only flag stuck after quota removal Once s3.bucket.quota.enforce marks a bucket read-only, removing or disabling the quota orphans the flag: enforcement skips buckets with a non-positive quota, and fs.configure merges booleans with OR so -readOnly=false could never turn it off. The only way out was deleting the whole path rule. - s3.bucket.quota -op=remove/disable and the admin server quota update now lift the read-only flag on the bucket's path rule - fs.configure now honors explicitly passed false boolean flags (-readOnly, -fsync, -worm) instead of OR-merging them away * filer: ClearBucketReadOnly reports unchanged when the save fails	2026-07-10 22:09:39 -07:00
Chris LuandGitHub	9a7cfaf371	admin: stop holding the shell admin lock across whole scheduler batches (#10290) ... * admin: let waiting shell clients win the admin lock The admin lock manager re-acquired the cluster shell lock immediately after releasing it, while a waiting weed shell client only polls the master once a second, so an operator could lose the race indefinitely. Leave the lock free for slightly more than one poll interval before re-acquiring, and once overlapping reference-counted holds have kept the lock continuously held for over a minute, make new admin-side acquires wait for a full release before piggybacking. * admin/plugin: take the shell lock per detection and per job, not per batch The default-lane scheduler acquired the cluster shell lock once and held it across the whole pass: every due job type's detection plus all of its dispatched jobs, each drained to completion. A manual weed shell operation sharing that lock could stall behind the batch for up to the extended execution window. Hold the lock only while it protects something: around each detection scan and around each dispatched job. Manual shell operations now wait for at most one in-flight job. The admin UI detect+execute path stops wrapping dispatch in an outer hold, since a nested acquire would deadlock against the lock manager's fairness window; detection and per-job dispatch take the lock themselves. * admin/plugin: stop extending the dispatch window to the largest job estimate When any proposal's estimated runtime exceeded the remaining JobTypeMaxRuntime, the whole dispatch context was replaced with a fresh one capped at eight hours, so a single balance backlog could hold the default lane (and with it erasure_coding and vacuum detection) for that long. Keep the dispatch window at JobTypeMaxRuntime and instead detach each started attempt onto its own estimated-runtime deadline. Large jobs still get their full time once started; jobs not yet started when the window closes are canceled and re-proposed by a later detection, so sibling job types get a turn every window. * worker/balance: re-check each planned move against the master before executing A balance plan is computed at detection time, but the admin lock is released between detection and execution, so a manual shell operation can rearrange the volume in the gap. The task's own guards catch a vanished source, but a target that gained a replica in the meantime would be silently overwritten by VolumeCopy and the source delete would then reduce the volume to a single copy. Before executing each move, ask the master for the volume's current locations (uncached) and skip the move if the volume has left the source or the target already holds a replica. Skipped moves fail with a stale-move error and the next detection replans them. Without master addresses in the cluster context the check is skipped, preserving the old behavior with older admins. * admin: block re-acquire while the final lock release is in flight Release dropped the manager mutex before calling ReleaseLock, so a concurrent Acquire could see hold count zero and call RequestLock while the locker still considered itself locked. That request no-ops, leaving a hold with no live master lease. Track the in-flight release and make Acquire wait for it. Also normalize a nil release function from lock manager implementations, and make the yield and fairness windows per-instance fields so tests stop mutating globals.	2026-07-09 20:43:55 -07:00
Chris LuandGitHub	60e7b30009	admin: browse Iceberg table data (#10227) ... * admin: move volume-server read JWT helper into dash The Iceberg data preview page needs the same per-fileId read token the file browser uses when streaming chunks from volume servers. Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur * admin: add Iceberg table data preview page The admin UI browses the Iceberg catalog down to table details but not the data itself. Add a Browse Data page per table that walks the selected snapshot's manifests and shows sample rows from its Parquet data files, plus the data file list with per-file preview, a snapshot switcher, and a row limit selector. Rows are read through a ranged ReaderAt over stream-content so only the Parquet footer and needed pages are fetched, with the volume read JWT applied when configured. Iceberg locations resolve into /buckets with traversal guards, and the file parameter must match a manifest-listed data file. Snapshots with delete files get a warning that raw rows are shown. Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur * admin: integration test for Iceberg catalog and data preview pages Starts a weed mini cluster with the admin UI, creates a table bucket, namespace, and tables via the S3 Tables manager, uploads real Parquet files via S3, writes manifests and snapshots with iceberg-go, and asserts on the rendered pages: catalog browsing, table details, current and historical snapshot previews, per-file preview, row limits, unknown snapshot and file errors, and a metadata-less table. Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur * admin: write Iceberg preview chunk reads straight into the caller slice ReadAt wrapped the caller's buffer in a bytes.Buffer, which would silently allocate a fresh backing array and drop bytes if it ever grew. Copy directly into the destination slice and reject negative offsets so the ReaderAt contract holds. Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur * admin: link to snapshot history when the preview switcher truncates The snapshot switcher caps at 25 entries; add a trailing item pointing at the table details page so older snapshots stay reachable. Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur * test: hoist mini cluster context assignment out of the goroutine Set MiniClusterCtx before launching the cluster goroutine and clear it in stop(), so the assignment is not buried in the command loop. Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur	2026-07-03 14:02:44 -07:00
Chris LuandGitHub	292c7493fa	s3: enforce bucket quota on logical size and surface read-only state in Admin UI (#10224) ... * s3: enforce bucket quota on logical size, not un-vacuumed physical size A bucket full of deleted/overwritten objects awaiting vacuum went read-only while its live data stayed under quota, because enforcement used the raw single-copy volume size with garbage included. Subtract DeletedByteCount via a LogicalSize() helper in the auto-enforce loop, the s3.bucket.quota.enforce command, and the bucket_size_bytes metric (labeled logical but counting garbage too). Deleting objects now relieves quota immediately and enforcement matches the UI usage figure. * admin: surface bucket read-only state in the S3 buckets UI Read the read-only flag quota enforcement writes to filer.conf and show it as a badge in the bucket list and a Status row in the details modal, so an operator can see why writes are being rejected.	2026-07-03 12:39:45 -07:00
3 people	9d75048594	admin: respect filerGroup for cluster discovery (#10170) ... * Respect filerGroup in admin discovery Admin discovery previously queried master cluster nodes with an empty filer group, so filers registered under a non-default group could not appear in the admin UI. Add an admin filerGroup flag and carry it through cluster-node discovery requests while preserving the empty default behavior. Constraint: SeaweedFS master ListClusterNodes filters by exact filer_group. Rejected: Discover all groups implicitly | no existing admin or shell behavior exposes cross-group discovery. Confidence: high Scope-risk: narrow Directive: Keep admin cluster discovery scoped to the configured filerGroup unless an explicit all-groups API is added. Tested: docker run --rm -v "$PWD:/src" -w /src golang:1.25 go test ./weed/admin/dash -run TestListClusterNodesRequest -count=1 Tested: docker run --rm -v "$PWD:/src" -w /src golang:1.25 go test ./weed/command -run '^$' -count=1 Not-tested: full repository test suite * mini: pass filer group to admin cluster discovery miniAdminOptions.filerGroup was never initialized, so startAdminServer dereferenced a nil *string. Share the filer.filerGroup flag pointer so the co-located admin queries the same group the filer registers under. --------- Co-authored-by: Chris Lu <chris.lu@gmail.com>	2026-07-01 10:56:58 -07:00
Chris LuandGitHub	36e51e5542	admin: fix 'send on closed channel' panic in worker gRPC server (#10175) ... * admin: never close the worker outgoing channel while senders are live conn.outgoing has multiple concurrent senders (heartbeat, task assignment, log request, registration handlers). Closing it on connection teardown raced a sender and paniced with "send on closed channel" — reliably reproduced when a laptop goes idle: heartbeats stall past the 2-minute stale cutoff, the cleanup routine closes the channel, and the resumed worker's heartbeat is received and handled at the same moment. The connection context is already the sole teardown signal, so stop closing the channel entirely. handleOutgoingMessages exits on conn.ctx.Done(), and the buffered channel is GC'd once the connection drops. Route sends through a sendToWorker helper that also selects on conn.ctx.Done() so they bail on teardown instead of blocking for the full timeout. * admin: bail on ctx.Done() while waiting for a worker log response	2026-06-30 21:22:28 -07:00
Chris LuandGitHub	77bf2a3ab0	volume.balance: gate on real physical disk usage (fixes #10160) (#10162) ... * shell: add volume.balance -byDiskUsage to balance by actual data The default balancer ranks servers by slot density, dividing used volumes by MaxVolumeCount. When MaxVolumeCount is configured higher than the disk can hold, a physically near-full server looks nearly empty and gets picked as the move target, so balancing drains less-full servers onto an already-full one. -byDiskUsage ranks servers by the actual data they hold (sum of volume sizes) instead, so the fullest-by-data server is treated as full and balancing drains it. It assumes comparable disk sizes per disk type and still respects each server's free volume slots. Default behavior is unchanged. * plumb physical disk usage into topology, gate volume.balance on it Volume servers now report each disk's filesystem total/free bytes in the heartbeat, and the master stores them in DiskInfo. volume.balance uses them to skip any move target whose disk is already near full (-maxDiskUsagePercent, default 90), so an over-configured maxVolumeCount can no longer make a physically full server look empty and get drained onto. The gate judges each server against its own disk, so heterogeneous disk sizes are fine; servers that do not report bytes fall back to slot-only behavior. Rust seaweed-volume mirrors the heartbeat reporting. * admin: report real physical disk capacity when volume servers provide it The dashboard estimated server capacity as maxVolumeCount * volumeSizeLimit, which overstates it when maxVolumeCount is set higher than the disk holds. Prefer the filesystem capacity now reported per disk, falling back to the estimate for servers that do not report it. * worker: gate automatic balance on physical disk fullness too The maintenance balance worker selects the least slot-utilized server as the move destination, so an over-configured maxVolumeCount makes a physically full server look empty and get drained onto — the same defect as the shell command. Now that DiskInfo carries real disk bytes, skip any destination whose disk is at/above 90% used (per server, against its own disk); a full server can still be a source. When every candidate destination is full, create no tasks. Servers that do not report disk bytes are not gated. * balance: share the physical-disk-fullness gate between shell and worker The shell volume.balance command and the maintenance balance worker each grew their own copy of the disk-fullness gate (targetDiskTooFull / destinationDiskTooFull) and a maxDiskUsagePercent=90 constant. Pull both into weed/topology/balancer (DiskTooFullAfter + DefaultMaxDiskUsagePercent) so the policy has one home and the two balancers can't drift. * balance: harden the physical-disk gate Guard against a nil DiskInfo in the byte/slot lookups. Let a zero disk-capacity report clear previously stored bytes (0 means "not reported" for bytes, unlike maxVolumeCount), so a server that stops reporting falls back to slot-only instead of trusting stale capacity. In the worker, charge each planned move's bytes to its destination within a detection cycle so the gate sees a target fill up rather than only its heartbeat-time free space. Note the per-location capacity summing assumes one location per filesystem (the used ratio the gate relies on stays correct regardless; absolute capacity can over-report).	2026-06-30 19:31:12 -07:00
Chris Lu	57ffef8543	fix(admin): skip task state files with no task data on load ... An empty or truncated tasks/*.pb file unmarshals into a TaskStateFile with a nil Task, and protobufToMaintenanceTask dereferenced it immediately, panicking the whole admin process on startup. Guard the nil case so the loader logs a warning and skips the bad file.	2026-06-26 17:36:42 -07:00
Minsoo KimandGitHub	638f6ff433	admin: surface user inline policies in object store user details (#10013) ... GetObjectStoreUserDetails only returned identity.PolicyNames (attached managed policies) and omitted per-user inline policies. Inline policies are stored separately from the identity record and are authoritative at S3 enforcement time (they take precedence over the legacy Actions list), so an operator could not see what actually governed a user's access via the admin API/UI. Include inline policy names (via credentialManager.ListUserInlinePolicies) in the returned PolicyNames. Adds a unit test using the memory credential store.	2026-06-18 22:20:56 -07:00
Chris LuandGitHub	7df43ad9b5	admin: add connected Mount Clients page and dashboard section (#9968) ... * admin: add connected mount clients page and dashboard section The filer is the authority on who is subscribed to its metadata stream (FUSE/VFS mounts, S3, peer filers, ...), but its in-memory listener registry only tracked clientId->epoch and was not exposed. - Enrich the filer subscriber registry with name/type/address/path/ connected-time, populated in addClient and cleared in deleteClient so it reflects currently-connected clients only. - Add a ListMetadataSubscribers filer gRPC (optional client-type filter). - Admin server fans out to every filer, filters to mount types ("mount" Go weed mount, "sw-vfs" Rust VFS), and renders a new Cluster > Mount Clients page plus a Mount Clients dashboard section. Read-only; no behavior change to the subscribe hot path. * admin: address review — parallelize filer fan-out, guard nil map, robust CSV - GetMountClients now queries filers concurrently, each under a 5s timeout, so a slow/unreachable filer can't stall the admin dashboard. - Defensively initialize fs.subscribers before first write. - Mount Clients CSV export uses a Blob with quote-escaping instead of a data: URI, so special characters in paths export correctly.	2026-06-14 21:44:10 -07:00
Chris LuandGitHub	d47cc45b1f	admin: fold dashboard sparklines into the existing cards (de-dup) (#9964) ... admin: fold dashboard sparklines into the existing cards The trend sparklines added in #9957 lived in a separate "Cluster Trends" row that duplicated the existing summary cards (Volumes, Files, Disk Used, EC Shards). Remove that row and instead render each sparkline inside the matching summary card, so every headline number shows its recent trend without duplication. The two maintenance metrics that have no existing card — Active Tasks and Workers — now fill the previously-empty columns of the EC row (also with sparklines). DashboardTrends changes from a Cards slice to named per-card sparkline SVGs (+ current values for the two maintenance cards). Drops the now-unused trendBytes helper (disk size keeps using the existing formatBytes).	2026-06-14 14:17:43 -07:00
Chris LuandGitHub	b56d155b31	admin: native at-a-glance trend sparklines on the dashboard (#9957) ... * admin: native at-a-glance trend sparklines on the dashboard Add a "Cluster Trends" row to the admin Dashboard with inline-SVG sparklines for volumes, EC shards, disk used, files, active maintenance tasks, and workers. The data comes entirely from what the admin already holds — the cached cluster topology and the in-process maintenance queue — sampled into a small bounded ring buffer on the existing maintenance-metrics ticker (~15 min of history). No Prometheus/Grafana dependency, no JS chart library, no extra goroutine: the sparklines are self-contained SVG rendered server-side via templ. This gives basic trend visibility out of the box for clusters that don't run Prometheus, and a quick glance next to the cluster controls; Grafana remains the place for deep/historical dashboards. * admin: cap trendBytes unit index to avoid out-of-bounds panic A value >= 1 ZiB would push exp past the end of the units string and panic on units[exp]; cap exp at the last unit (EiB).	2026-06-14 13:55:26 -07:00
Chris LuandGitHub	37962e2445	admin: configure maintenance tasks via admin.toml (#9926) ... * admin: configure maintenance tasks via admin.toml Maintenance task settings could only be edited in the admin UI and live under <dataDir>/conf, so they silently reverted to defaults whenever the data directory was recreated. An optional admin.toml now declares vacuum, balance, and erasure coding settings; keys set there are written through to the persisted task configs at every startup, overriding UI edits, so the configuration stays declarative. Generate an example with "weed scaffold -config=admin". * vacuum: round min volume age up to whole hours MinVolumeAgeSeconds was truncated by integer division when converted to the hour-granular protobuf field, so a sub-hour setting silently became 0 and disabled the age guard. * admin: split and normalize preferred_tags from admin.toml A comma-separated string, as set via environment variable, came through viper as a single slice element. Split on commas and reuse util.NormalizeTagList, matching the plugin config path. * scaffold: clarify admin.toml wording	2026-06-11 11:04:52 -07:00
Chris LuandGitHub	3fadbef3eb	feat(admin): export full cluster volume list as JSON (#9876) ... Adds an "Export All (JSON)" button on the Cluster Volumes page that pulls the whole cluster's volume list from the master in one call, a superset of volume.list. Beyond the table columns it carries garbage and fullness ratios, modified time, compact revision, remote tiering keys, per-disk capacity counts, EC shard sizes with file/delete counts, and a cluster-wide duplicate-volume-id scan. Honors the active collection filter. The existing per-page CSV export stays as "Export Page".	2026-06-08 15:01:02 -07:00
Chris LuandGitHub	4c050ad76b	Don't mangle filer paths with the OS separator on Windows (#9878) ... fix: don't mangle filer paths with the OS separator on Windows filepath.Dir/Join use the platform separator, so on Windows they rewrite a forward-slash filer path like /buckets/x into \buckets\x. The mangled value then goes into a filer RPC and operates on the wrong key, so the op silently targets nothing. The admin file browser hit this in New Folder (the entry landed under \buckets\my-bucket and never showed up under /buckets/my-bucket), and the same way in delete, view and properties. MQ topic retention and consumer-offset listing, and the SFTP home dir plus create-permission parent lookup, had the same bug. Switch all of these to the path package, which always uses "/".	2026-06-08 13:56:02 -07:00
Chris LuandGitHub	b2127c86f4	admin: show S3 servers under Cluster (#9847) ... * s3: register data center with master on startup * admin: show S3 servers under Cluster * admin: add S3 servers to the dashboard	2026-06-07 00:32:20 -07:00
Chris LuandGitHub	ca81c0c525	fix(ec): pass per-volume data-shard count to the parity-shard split (#9781) ... * fix(ec): pass per-volume data-shard count to the parity-shard split ShardsInfo.DeleteParityShards/MinusParityShards looped ids 10..13, assuming the fixed 10+4 layout. For a non-default ratio this splits data vs parity wrong — a wide ratio (12+4, 16+6) drops real data ids >= 10, which breaks ec.decode. They now take a dataShards argument (<= 0 falls back to DataShardsCount) and clear ids dataShards..MaxShardCount. ec.decode threads the data-shard count from collectEcNodeShardsInfo to both split call sites, and admin LogicalSize passes DataShardsCount. Also: EC cleanup now sets an explicit per-disk storage impact (-len(ShardIds)) instead of falling back to the TotalShardsCount constant, so freed-capacity accounting matches the shards actually removed. OSS is always 10+4, so behavior is unchanged here; this keeps the split ratio-correct and the API aligned with the enterprise per-volume override. Adds parity-split ratio tests. * ec: clear parity shards in one locked pass Address review: DeleteParityShards looped si.Delete, taking the lock once per id. shards is sorted by Id and shardBits is a bitmap, so mask off the high bits and truncate the sorted slice at the first parity id (binary search) under a single lock. Preserves the dataShards<=0 -> DataShardsCount default.	2026-06-01 19:25:15 -07:00
Chris LuandGitHub	25beb7ec48	admin: expose Prometheus metrics (#9652) ... * admin: add -metricsPort flag to expose Prometheus metrics The admin command had no metrics endpoint, so passing -metricsPort (as the operator does for spec.admin.metricsPort) crashed the process with "flag provided but not defined". Wire up -metricsPort/-metricsIp and start the shared Prometheus metrics server, matching filer, master, and volume. * admin: emit maintenance task and worker fleet metrics Add Prometheus metrics for the admin server's distinctive work: the maintenance task queue and the worker fleet that executes it. Task lifecycle: maintenance_tasks_by_status / _by_type gauges (snapshot of the queue), maintenance_tasks_completed_total{type,outcome} counter and maintenance_task_duration_seconds{type} histogram (recorded when a task reaches a terminal state), and last/next scan timestamp gauges. Worker fleet: workers_connected and worker_slots{used,max} gauges, plus worker_events_total{event} counting register/unregister/stale removals. Gauges are snapshotted by a background goroutine on the admin server; counters and the histogram are recorded at their event sites. * admin: read worker slot totals under lock, clear next-scan gauge when idle GetWorkers returns live worker pointers; summing CurrentLoad/MaxConcurrent outside the queue lock races with task assignment and completion. Add GetWorkerSlotTotals to aggregate under the lock. Also reset maintenance_next_scan_timestamp_seconds to 0 when the scanner is not running, so it can't retain a stale value after a stop.	2026-05-24 14:09:02 -07:00
Chris LuandGitHub	87fdea5330	fix(admin): carry filer addresses as ServerAddress in plugin cluster context (#9600) ... The plugin cluster context forwarded filers as gRPC-only addresses (host:grpcPort). The admin-script worker stored that in ShellOptions.FilerAddress, whose shell commands re-derive the gRPC port via ToGrpcAddress() and re-add the +10000 offset, dialing a non-existent host:28888. Carry filers in pb.ServerAddress form (host:httpPort.grpcPort) and let each consumer convert when it dials: the admin shell uses it verbatim, while the s3_lifecycle and iceberg workers collapse it to a gRPC address. Rename the proto field filer_grpc_addresses -> filer_addresses so the name matches the content.	2026-05-21 02:10:27 -07:00
Chris LuandGitHub	7c5296dfb1	fix(admin): switch file browser upload/download to filer gRPC + volume HTTP (#9538) ... * fix(admin): switch file browser upload/download to filer gRPC + volume HTTP The admin file browser proxied uploads and downloads through the filer's HTTP listener, so the whole feature 404'd against filers started with -disableHttp=true even though S3 still worked on its own port. Re-route through the filer gRPC service: LookupDirectoryEntry + StreamContent for reads (chunks flow straight from the volume servers), AssignVolume + volume HTTP POST + CreateEntry for writes. Volume read tokens come from jwt.signing.read.key when configured; the old jwt.filer_signing tokens no longer apply since the filer HTTP surface is bypassed. * admin file browser: propagate request context + track response writes Pass r.Context() into uploadFileToFiler so a client disconnect cancels the in-flight chunked upload instead of letting it run to completion against the volume servers. For DownloadFile, replace the Content-Type probe with a small response-writer wrapper that records whether headers or bytes have actually been sent, so the error path can't silently convert a pre-stream failure into a partial response if future code moves the header-setting around.	2026-05-18 20:33:16 -07:00
Chris LuandGitHub	6b94701213	mini: quieter startup with a docker-compose-style progress board (#9524) ... * mini: quieter startup with a docker-compose-style progress board Replaces noisy startup/shutdown logs with a single in-place progress table on a TTY (or one line per state change off-TTY). Each component renders as `pending -> starting -> ready` during startup and `stopping -> stopped` during shutdown, with elapsed time on transition. Also folds in a few cleanups uncovered while making this readable: - route the admin.go startup prints through glog so quietMiniLogs() filters them under mini but standalone weed admin still shows them - generate a dev SSE-S3 KEK + passphrase on first run via WEED_S3_SSE_KEK and WEED_S3_SSE_KEK_PASSPHRASE env vars (viper.Set has a nested-key conflict between s3.sse.kek and s3.sse.kek.passphrase); persisted under the data folder so restarts reuse the same key - demote worker/master gRPC Recv 'context canceled' to V(1); those are the normal shutdown signal, not Errors/Warnings - drop the 'Optimized Settings' block and the 'credentials loaded from environment variables' message from the welcome banner - only show the credentials setup hints when no S3 identities exist (new s3api.HasAnyIdentity accessor backed by an atomic.Bool) - use S3_BUCKET in the credentials hint so it pairs with AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY - reorder running-services list to master / volume / filer / webdav / s3 / iceberg / admin * mini: refuse in-memory-only SSE-S3 dev keys; surface admin serve errors loadOrCreateMiniHexSecret returns "" when os.WriteFile fails, so SSE-S3 won't encrypt data under a KEK that the next restart can't reproduce (which would orphan whatever was written this run). The caller already treats "" as "skip setting WEED_S3_SSE_* env vars", so SSE-S3 and IAM just stay disabled for this run. startAdminServer's serve goroutine used to only log ListenAndServe failures, so a bind error left the caller blocked on ctx.Done() with no listener. Forward the error through a buffered channel and select on it alongside ctx.Done(). * ci(s3-proxy-signature): match weed mini's new progress-board ready line The readiness probe grepped for "S3 (gateway|service).*(started|ready)", which matched weed mini's old "S3 service is ready at ..." line. Mini now emits " S3 ready (Xs)" from its progress board, so the old pattern misses and the test timed out at the 30-second wait. Widen the alternation to also accept "S3\s+ready". The curl HEAD fallback already covers any remaining cases.	2026-05-17 19:13:09 -07:00
Chris LuandGitHub	b4289abb0a	admin: convert filer address to gRPC form before dispatch (#9523) ... The master returns each registered filer in pb.ServerAddress dual-port form (host:httpPort.grpcPort, e.g. 10.0.0.1:8888.18888). The admin's plugin context builder forwarded that string verbatim as filer_grpc_address, so workers calling grpc.DialContext on it failed every job in ~3ms with "dial tcp: lookup tcp/8888.18888: unknown port". Run each entry through pb.ServerAddress.ToGrpcAddress before populating ClusterContext.FilerGrpcAddresses. The lifecycle integration test now pins filer.port.grpc to a value that breaks the FILER_PORT+10000 assumption, and a new dispatch test drives the admin's /api/plugin/job-types/s3_lifecycle/run path end-to-end and asserts the dispatched job both reaches the filer and deletes the backdated object.	2026-05-17 11:33:54 -07:00
Chris LuandGitHub	62821964dd	filer/iam-grpc: make admin Bearer auth opt-in (fixes #9509) (#9514) ... PR #9442 made the filer refuse to register the IAM gRPC service unless jwt.filer_signing.key was set in security.toml, which broke the admin UI Users/Groups/Policies pages for every deployment that ships without a security.toml — weed mini, plain Helm, vanilla weed filer. The Users tab returns Unimplemented and the page is unusable. Issues #9504, #9505 and #9509 all trace to this gap. The rest of the filer's gRPC surface is unauthenticated by default; treat IAM the same way. The service now always registers, and the auth gate is a no-op when no signing key is configured. When the key is set, every RPC still requires an admin-signed Bearer token, matching the post-#9442 behaviour. Operators who expose the filer gRPC port beyond a trusted network should set the key on both filer and admin. The admin client (IamGrpcStore.withIamClient) already skips attaching the authorization metadata when its key is empty, so no changes there.	2026-05-15 13:15:20 -07:00
Chris LuandGitHub	c47eab1a5d	admin: attach admin-signed Bearer token on filer IAM gRPC calls (#9498) ... * admin: attach admin-signed Bearer token on filer IAM gRPC calls PR #9442 added Bearer-JWT enforcement on the filer's IAM gRPC service but didn't update its only production client, IamGrpcStore. The admin UI Users/Groups pages went through that client and started failing in 4.24 with either Unimplemented (filer refuses to register the service when jwt.filer_signing.key is empty) or Unauthenticated (the client sent no token). Issues #9495 and #9496 both trace to this gap. Plumb jwt.filer_signing.key into IamGrpcStore via a new SetAdminSigning hook called from the admin server, and append a freshly minted Bearer token to outgoing metadata on every call. The mint helper security.GenJwtForFilerAdmin existed since #9442 but had no production caller; this wires it up. Add an integration test alongside grpc_store.go that runs a real IamGrpcServer over a real grpc.Server listener and exercises the store end-to-end: matching key succeeds, wrong key returns Unauthenticated, no key returns Unauthenticated. Without the client-side token attach the success path fails, so the regression cannot land again. * address review: include adminSigningExpiresAfterSec in mu comment	2026-05-14 10:51:04 -07:00
Chris LuandGitHub	5d43f84df7	refactor(plugin): rename detection_interval_seconds → detection_interval_minutes (#9366) ... Minutes is the natural granularity for detection cadence — every production handler already set the seconds field to a 60-multiple (17*60, 30*60, 3600, 24*60*60). Switching to minutes drops the *60 arithmetic and matches the unit conventions used elsewhere in the plugin worker forms. - Proto: AdminRuntimeDefaults + AdminRuntimeConfig.detection_interval_* field renamed. - Helpers: durationFromMinutes / minutesFromDuration alongside the existing seconds variants in plugin_scheduler.go. - Handlers: vacuum, ec_balance, balance, erasure_coding, iceberg, admin_script, s3_lifecycle now declare DetectionIntervalMinutes. - Admin: scheduler_status + types + UI templ + plugin_api.go pass through the new field; UI label and table cells switch to "min".	2026-05-08 10:33:02 -07:00
Chris LuandGitHub	7f254e158e	feat(worker/s3_lifecycle): plugin handler with admin UI config (#9362) ... * feat(s3/lifecycle): scheduler — N pipelines over an even shard split Scheduler.Run spawns Workers Pipeline goroutines plus one engine-refresh ticker. Each worker owns a contiguous AssignShards(idx, total) slice of [0, ShardCount) and runs Pipeline.Run with EventBudget bounding each iteration; brief RetryBackoff between iterations avoids hot-loop on errors. The refresh ticker rebuilds the engine snapshot from the filer's bucket configs every RefreshInterval. LoadCompileInputs / IsBucketVersioned / AllActivePriorStates are exported from a configload.go sibling so the shell command can move to this shared implementation in a follow-up. * refactor(shell): reuse scheduler.LoadCompileInputs in run-shard Drop the local copies of loadLifecycleCompileInputs / isBucketVersioned / allActivePriorStates / lifecycleParseError that the new scheduler package now exports. Same behavior, one source of truth. * feat(worker/s3_lifecycle): plugin handler with admin UI config Registers a JobHandler for s3_lifecycle via pluginworker.RegisterHandler. Admin pulls the descriptor over the worker plugin gRPC and renders the AdminConfigForm + WorkerConfigForm in the existing UI: Admin form (cluster shape): - workers (1..16, default 1) - s3_grpc_endpoints (comma list) Worker form (operational tuning): - dispatch_tick_ms (default 5000) - checkpoint_tick_ms (default 30000) - refresh_interval_ms (default 300000) - event_budget (default 0 = unbounded) Detect emits a single proposal whenever S3 endpoints + filer addresses are configured. MaxExecutionConcurrency=1 so admin only ever runs one lifecycle daemon per worker; a fresh proposal next cycle restarts it if the prior Execute exits. Execute dials the configured S3 endpoint + filer, builds a scheduler.Scheduler with the parsed config, and runs it until ctx cancellation. Reuses the existing scheduler / dispatcher / reader / engine packages — the handler is the thin glue that parses descriptor values and wires the long-running daemon. * proto(plugin): add s3_grpc_addresses to ClusterContext So workers can dial s3 servers discovered by the master rather than a hand-typed list in the admin form. * feat(admin): populate ClusterContext.s3_grpc_addresses from master ListClusterNodes(S3Type) returns the live S3 servers; the plugin scheduler now hands these to job handlers alongside filer/volume addresses. * feat(worker/s3_lifecycle): discover s3 endpoints from cluster context Drop the s3_grpc_endpoints admin form field and read the master-supplied ClusterContext.S3GrpcAddresses instead. Operators no longer maintain a hand-typed list, and a stale entry self-heals when the master's view updates. * feat(worker/s3_lifecycle): time-based runtime cap, friendlier cadence units - dispatch_tick_minutes (was *_ms): minutes is the natural granularity for a daily batch; default 1 minute. - checkpoint_tick_seconds: seconds for the durable cursor write; default 30 seconds. - refresh_interval_minutes: minutes for the engine snapshot rebuild. - max_runtime_minutes replaces event_budget. Each daily run is bounded by wall clock — typical run wraps in well under an hour because the cursor persists and the meta-log streams fast. Default 60 minutes. - AdminRuntimeDefaults.DetectionIntervalSeconds = 86400 so the admin schedules one job per day.	2026-05-08 10:30:02 -07:00
4 people	db34e8b6fd	feat(admin): prefer stored S3 Content-Type metadata over key-extension MIME inference (#9286) ... * feat(admin): prefer stored filer mime in file browser and properties * feat(mime): enhance MIME type registration and improve fallback logic Co-authored-by: Copilot <copilot@github.com> --------- Co-authored-by: baracudaz <baracudaz@users.noreply.github.com> Co-authored-by: Copilot <copilot@github.com>	2026-04-29 10:21:03 -07:00
Parviz MiriyevandGitHub	f407bdaa36	fix(admin): use TLS-aware HTTP client for /dir/status fetch (#9227) ... fetchPublicUrlMap() in weed/admin/dash/cluster_topology.go uses a dedicated &http.Client{} that doesn't honor security.toml client TLS configuration, and hardcodes "http://" in the URL. When master is configured HTTPS-only ([https.master] set), every cluster topology cache refresh logs: NOTICE: http: TLS handshake error from <admin-ip>:<port>: client sent an HTTP request to an HTTPS server The function falls through to glog.V(1).Infof and returns nil, so the admin UI loses PublicUrl enrichment for data nodes. Cosmetic but noisy. Switch to util_http.GetGlobalHttpClient() whose Do() calls NormalizeHttpScheme(), which automatically rewrites http:// to https:// when [https.client] is enabled and presents the configured client cert. Preserve the 5-second timeout via context.WithTimeout(). Same pattern as weed/admin/handlers/file_browser_handlers.go, weed/server/master_server.go, weed/shell/command_volume_fsck.go.	2026-04-26 12:25:53 -07:00
Chris LuandGitHub	5eead9409a	fix(admin): S3 Tables CSRF token + non-empty 409 status (#9221) ... * fix(admin): attach CSRF token to S3 Tables write requests Several POST/PUT/DELETE calls in s3tables.js were sent without an X-CSRF-Token header while the corresponding handlers in weed/admin/dash/s3tables_management.go enforce CSRF via requireSessionCSRFToken, so authenticated users hit "invalid CSRF token" on actions like creating a table bucket (#9220), updating policies, and managing tags. Add an s3tWriteHeaders helper that pulls the token from the existing csrf-token meta tag and use it on every write to /api/s3tables/buckets, /bucket-policy, /tables, /table-policy, and /tags. The Iceberg-page write paths already attached the token and are unchanged. Fixes #9220 * fix(admin): map BucketNotEmpty/NamespaceNotEmpty to 409 for S3 Tables DELETE on a non-empty table bucket or namespace returned HTTP 500 because s3TablesErrorStatus didn't list ErrCodeBucketNotEmpty or ErrCodeNamespaceNotEmpty in its conflict case, even though the backend handler emits them with 409 Conflict (matching AWS S3 Tables). Add both codes to the existing conflict mapping. * refactor(admin): route Iceberg S3 Tables writes through s3tWriteHeaders Iceberg namespace/table create and Iceberg table delete were still hand-rolling CSRF headers. Replace those blocks with the existing s3tWriteHeaders() helper so every S3 Tables write uses the same code path. Drop the now-unused csrfTokenInput.value population in initIcebergNamespaces and initIcebergTables (the templ hidden inputs have no server-rendered value, and nothing reads the input now that the JS reads the token from the meta tag via getCSRFToken()).	2026-04-24 22:48:41 -07:00
3 people	c6302fcb54	feat(iam): allow caller-supplied AccessKeyId and SecretAccessKey in CreateAccessKey (#9172) ... * feat(iam): support caller-supplied AccessKeyId and SecretAccessKey in CreateAccessKey Both IAM implementations (standalone and embedded) now check for caller-supplied AccessKeyId and SecretAccessKey form parameters before generating random credentials. If provided, the caller-supplied values are used. If empty, random keys are generated as before. This enables programmatic identity provisioning where the caller needs to control the S3 credentials. Backward-compatible: no behavior change for callers that omit these parameters. * refactor(iam): extract shared caller-supplied credential validation Move the AccessKeyId/SecretAccessKey format checks and the in-memory collision scan into weed/iam so the standalone IAM API, the embedded IAM in s3api, and the admin dashboard all enforce the same rules. - ValidateCallerSuppliedAccessKeyId: 4-128 alphanumeric (rejects SigV4-breaking characters like '/' and '='). - ValidateCallerSuppliedSecretAccessKey: 8-128 chars. - FindAccessKeyOwner: scans identities and service accounts and returns the owning entity type + name for debug logging, without exposing the owner in caller-facing error messages. The admin dashboard previously only length-checked caller-supplied keys; it now enforces the same alphanumeric rule, which matches what SigV4 actually accepts anyway. * fix(iam): reject partial caller-supplied AccessKeyId/SecretAccessKey Previously, if a caller supplied only one of AccessKeyId or SecretAccessKey, CreateAccessKey logged a warning and auto-generated the missing half. That silently returns a credential the caller did not fully choose, which is surprising and easy to miss in a response they expected to echo back their input. Return ErrCodeInvalidInputException instead: either both are supplied or neither is. Updates the mixed-supply tests in weed/iamapi and weed/s3api to assert the rejection. * chore(iam): centralize and broaden sensitive form redaction DoActions and ExecuteAction both had an inline loop that redacted SecretAccessKey from their debug-level request log. Replace the two copies with iam.RedactSensitiveFormValues, backed by an explicit sensitive-keys set. The set now also covers Password, OldPassword, NewPassword, PrivateKey, and SessionToken. None of those parameters are used by today's IAM actions, but naming them here makes the log-safety guarantee survive future additions such as LoginProfile / STS. * test(iam): cover the upper length bound for CreateAccessKey TestCreateAccessKeyBoundary / TestEmbeddedIamCreateAccessKeyBoundary only exercised the 3/4-char lower edge. Add cases for 128 (accepted) and 129 (rejected) for AccessKeyId, plus 7 / 128 / 129-char cases for SecretAccessKey, so both ends of the validator are locked in at the handler level (the pure validators in weed/iam already cover this). * fix(s3api/iam): verify user existence before RNG and collision scan In the embedded IAM CreateAccessKey, the user lookup ran last: a request for a non-existent user still walked the whole identity / service-account list for collisions and, if no caller-supplied keys were present, generated fresh random credentials with crypto/rand before the NoSuchEntity error finally surfaced. Reorder: validate inputs, then find the target identity, then do the collision scan, then generate keys. A missing user now fails fast and consumes no entropy, and the handler returns NoSuchEntity instead of a misleading EntityAlreadyExists when both the user is missing and the supplied AccessKeyId happens to collide with another identity's key. Add TestEmbeddedIamCreateAccessKeyRejectsMissingUser to lock in the "no mutation on unknown user" guarantee. The standalone iamapi CreateAccessKey intentionally keeps its pre-existing "create-or-attach" semantics where a missing user is implicitly provisioned — that is a behavior change beyond the scope of this PR. * test(iam): tighten collision leak assertion and cover 8-char secret - Rename the collision-owner identity in TestCreateAccessKeyRejectsCollision (both iamapi and the embedded s3api test) from "existing" / "ExistingUser" to "ownerAlpha". The old assert.NotContains check was effectively a no-op because the error message never contained those substrings; a distinctive name shared with no part of the expected error body makes the leak guard actually meaningful if the wording ever drifts. The embedded test also adds a NotContains assertion that was previously missing entirely. - Add an explicit 8-char SecretAccessKey pass case to both boundary tests so the lower edge of the validator is locked in at the handler level alongside the 7 / 128 / 129-char cases. * fix(iamapi): enforce both-or-none before the collision lookup In the standalone IAM CreateAccessKey, FindAccessKeyOwner ran before the partial-credential check. If a caller supplied only AccessKeyId and it happened to collide with an existing key, the response was EntityAlreadyExists instead of the more fundamental InvalidInput for omitting SecretAccessKey — wrong error class, and leaked the fact that the probed key is already in use. Swap the order: validate both-or-none first, then do the collision scan. Matches the embedded IAM path and AWS behavior. Add a case to TestCreateAccessKeyRejectsPartialSupply that combines partial supply with a collision to lock in the ordering. * fix(admin): reject partial caller-supplied AccessKey/SecretKey The admin dashboard path silently generated the missing half when a caller supplied only one of AccessKey or SecretKey, while the IAM API and embedded IAM paths now reject this. Align the three: if exactly one is provided, return ErrInvalidInput. Also simplifies the generator block — either both are provided or neither is, so there is no mixed path to handle. * test(s3api/iam): guard dereferences in caller-supplied-keys test TestEmbeddedIamCreateAccessKeyWithCallerSuppliedKeys dereferenced *AccessKeyId/*SecretAccessKey/*UserName and indexed Identities[0].Credentials[0] without first verifying shape, so any future regression that returns a partial response or skips the config mutation would panic mid-assertion instead of failing with a clear message. Add require.NotNil on the response pointers and require.Len on the identities/credentials slices before the asserts. * test(iamapi): exercise the service-account branch of the collision check FindAccessKeyOwner scans both Identities[*].Credentials and ServiceAccounts[*].Credential, but TestCreateAccessKeyRejectsCollision only covered the identity branch. Split the test into two subtests — one per branch — so a future refactor that drops the service-account scan (or mutates the existing credential) trips a failure. Also asserts the existing service-account credential is not mutated and no credential is attached to the target identity on rejection. * test(iam): isolate 129-char secret subcase from prior credential In both TestCreateAccessKeyBoundary (iamapi) and TestEmbeddedIamCreateAccessKeyBoundary (s3api), the 129-char SecretAccessKey subcase reused the "validkey" AccessKeyId that the preceding 8-char subcase had just persisted into the config. The test still asserted the right outcome because the handler validates secret length before running the collision scan — but if the two checks ever swap, the subcase would pass (or fail) for the wrong reason. Reset the in-memory credentials before the 129-char subcase, matching the pattern already used by the 3/128/129-char AccessKeyId and 7-char secret subcases. No behavior change; purely test isolation. --------- Co-authored-by: Chris Lu <chris.lu@gmail.com>	2026-04-22 12:35:55 -07:00