Scan floating IPs in the background, page by page, so thousands of addresses work

The "Scan Floating IP" button failed with a client timeout: the project now
holds ~6.4k floating IPs and the scan listed them all in one unpaginated,
timeout-less Neutron request on the HTTP request context.

openstack: ListFreeFloatingIPs reads marker-based pages (fields= keeps them
small) with per-page retry/backoff on transport errors, 5xx and 429, and every
request now has a timeout (also ends hangs inside the orchestrator tick).

orchestrator: the scan is a single-flight background job on the process
context with progress (clearing/listing/enqueuing/done/error), dry_run, full
discovery before anything is enqueued, then SubmitIPs in chunks of 500 in
ascending IP order; a failed read leaves the queue untouched. The auto-cycle
gets a "scanning" phase that polls the job, so the control loop and
autoCycleMu are never held across OpenStack/DB work; it recovers after a
restart and waits for (instead of adopting) a scan started by someone else.

db: migration 0009 (indexes), paged ListIPsPage/ListRegistryPage, GROUP BY
counters, EXISTS completion check, set-based ClearAllIPs.

API: POST /admin/ips/scan -> 202 (dry_run, wait), GET /admin/ips/scan, paging
and filters on /admin/ips and /admin/registry (bare arrays without limit),
results_by_overall in /admin/status.

dashboard: scan progress panel and dry-run button, paginated /ips and
/registry with server-side filters, Overview on counters and capped lists
with progress/ETA, "select all N by filter", hx-params fix for per-row
buttons, real counts in confirmations.

Also: docs (API, USAGE, DASHBOARD, README), plan and review under
docs/changes/, bin/ rebuilt with new SHA256SUMS.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
ayurishchevandClaude Sonnet 5.5 committed 2026-10-01 19:31:11 +03:00
1 parent debf2afed2
commit aff8fe38b5
61 files changed
+5833 -536

No files matched your search

+30 -3
View File
@@ -2,6 +2,8 @@ package httpapi
import (
"net/http"
"strconv"
"strings"
"cloudipvalidator/internal/db"
)
@@ -9,9 +11,30 @@ import (
// handleAdminRegistry lists every address ever submitted to the check
// queue, each with a summary of its accumulated check history — the
// durable record that survives an address being deleted from ip_queue and
// later re-added. See migrations/0007_ip_registry.sql.
// later re-added. See migrations/0007_ip_registry.sql. Without `limit` it is
// the bare array of every row; with `limit` (1..1000) it returns the envelope
// {items,total,limit,offset} (filters: offset, q = substring of the address,
// last_result = pass|partial|fail|cancelled).
func (s *Server) handleAdminRegistry(w http.ResponseWriter, r *http.Request) {
items, err := s.DB.ListRegistry(r.Context())
q := r.URL.Query()
limit, offset, paged, err := parsePaging(q)
if err != nil {
writeError(w, http.StatusBadRequest, err.Error())
return
}
filter := db.RegistryFilter{Query: strings.TrimSpace(q.Get("q")), LastResult: q.Get("last_result")}
if filter.LastResult != "" && !db.IsValidResult(filter.LastResult) {
writeError(w, http.StatusBadRequest, "invalid last_result "+strconv.Quote(filter.LastResult)+" (valid: pass, partial, fail, cancelled)")
return
}
var items []db.RegistrySummary
var total int
if len(q) == 0 {
items, err = s.DB.ListRegistry(r.Context())
} else {
items, total, err = s.DB.ListRegistryPage(r.Context(), filter, limit, offset)
}
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error())
return
@@ -20,7 +43,11 @@ func (s *Server) handleAdminRegistry(w http.ResponseWriter, r *http.Request) {
for i, it := range items {
out[i] = registrySummaryToDTO(it)
}
writeJSON(w, http.StatusOK, out)
if !paged {
writeJSON(w, http.StatusOK, out)
return
}
writeJSON(w, http.StatusOK, registryPageResponse{Items: out, Total: total, Limit: limit, Offset: offset})
}
// handleAdminRegistryHistory returns one address's registry record plus its