Scan floating IPs in the background, page by page, so thousands of addresses work

The "Scan Floating IP" button failed with a client timeout: the project now
holds ~6.4k floating IPs and the scan listed them all in one unpaginated,
timeout-less Neutron request on the HTTP request context.

openstack: ListFreeFloatingIPs reads marker-based pages (fields= keeps them
small) with per-page retry/backoff on transport errors, 5xx and 429, and every
request now has a timeout (also ends hangs inside the orchestrator tick).

orchestrator: the scan is a single-flight background job on the process
context with progress (clearing/listing/enqueuing/done/error), dry_run, full
discovery before anything is enqueued, then SubmitIPs in chunks of 500 in
ascending IP order; a failed read leaves the queue untouched. The auto-cycle
gets a "scanning" phase that polls the job, so the control loop and
autoCycleMu are never held across OpenStack/DB work; it recovers after a
restart and waits for (instead of adopting) a scan started by someone else.

db: migration 0009 (indexes), paged ListIPsPage/ListRegistryPage, GROUP BY
counters, EXISTS completion check, set-based ClearAllIPs.

API: POST /admin/ips/scan -> 202 (dry_run, wait), GET /admin/ips/scan, paging
and filters on /admin/ips and /admin/registry (bare arrays without limit),
results_by_overall in /admin/status.

dashboard: scan progress panel and dry-run button, paginated /ips and
/registry with server-side filters, Overview on counters and capped lists
with progress/ETA, "select all N by filter", hx-params fix for per-row
buttons, real counts in confirmations.

Also: docs (API, USAGE, DASHBOARD, README), plan and review under
docs/changes/, bin/ rebuilt with new SHA256SUMS.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
ayurishchevandClaude Sonnet 5.5 committed 2026-10-01 19:31:11 +03:00
1 parent debf2afed2
commit aff8fe38b5
61 files changed
+5833 -536

No files matched your search

+40 -30
View File
@@ -2,14 +2,19 @@ package dashboard
import (
"net/http"
"net/url"
"strings"
)
type registryPageData struct {
PageData
Items []registryItem
Query string
StatusFilter string
Items []registryItem
Query string
StatusFilter string
Page, PerPage int
Total int
Pager pagerData
PerPageOptions []int
}
type registryDetailData struct {
@@ -22,43 +27,48 @@ type registryDetailData struct {
// record that survives an address being deleted from /ips and later
// re-added. See internal/db/migrations/0007_ip_registry.sql. Optional
// ?q=&status= query params narrow the list by address substring and by
// LastResult — see filterRegistryItems.
// LastResult, and ?page=&per_page= select a page — all applied server-side
// (control-api's ListRegistryPage), so only the visible rows are transferred.
func (s *Server) handleRegistryPage(w http.ResponseWriter, r *http.Request) {
items, err := s.CA.ListRegistry(r.Context())
q := strings.TrimSpace(r.URL.Query().Get("q"))
status := r.URL.Query().Get("status")
if !containsStr(ipResults, status) {
status = ""
}
perPage := parsePerPage(r.URL.Query().Get("per_page"))
page := parsePage(r.URL.Query().Get("page"))
query := registryQuery{Q: q, LastResult: status, Limit: perPage, Offset: (page - 1) * perPage}
res, err := s.CA.ListRegistryPage(r.Context(), query)
if err == nil {
if clamped := clampPage(page, res.Total, perPage); clamped != page {
page = clamped
query.Offset = (page - 1) * perPage
res, err = s.CA.ListRegistryPage(r.Context(), query)
}
}
params := url.Values{}
if q != "" {
params.Set("q", q)
}
if status != "" {
params.Set("status", status)
}
data := registryPageData{
Items: filterRegistryItems(items, q, status),
Query: q,
StatusFilter: status,
Items: res.Items,
Query: q,
StatusFilter: status,
Page: page,
PerPage: perPage,
Total: res.Total,
Pager: newPager("/registry", "registry-table-wrap", params, page, perPage, res.Total),
PerPageOptions: perPageOptions,
}
data.ActiveNav = "registry"
data.Banner = bannerFor(err)
s.renderPage(w, r, "registry_page", data)
}
// filterRegistryItems narrows items to those whose address contains q
// (case-insensitive substring) and, if status is set, whose LastResult
// matches it exactly — the registry list's search-by-IP and
// filter-by-status, mirroring filterQueueItems in handlers_overview.go.
func filterRegistryItems(items []registryItem, q, status string) []registryItem {
if q == "" && status == "" {
return items
}
q = strings.ToLower(q)
out := make([]registryItem, 0, len(items))
for _, it := range items {
if q != "" && !strings.Contains(strings.ToLower(it.IPAddress), q) {
continue
}
if status != "" && it.LastResult != status {
continue
}
out = append(out, it)
}
return out
}
// handleRegistryDetail shows one address's full retained check history
// across every cycle it has ever run, not just the current attempt — see
// ip_detail_content in ip_detail.html for the attempt-scoped equivalent.