Retry a failed self-check on another validator; add the self-check failure ceiling
A validator that failed the self-check of an address no longer gets that address
again in the current round (ClaimNextQueued skips it); the validator itself stays
in service and takes all other addresses. The verdict fail is set when the number
of failed self-checks of an address reaches settings.self_check_max_attempts
(1..50, default 5, independent of the number of validators); max_retries and
retry_count are no longer used for self-check. If every working validator has
already failed the address, a new round starts and the exclusions lapse.
Migration 0012: ip_self_check_failures (permanent history per registry address),
ip_queue.sc_failures and sc_round_start_cycle (cycle_id is used instead of
attempt_number, which restarts when a queue row is recreated), the setting.
db.FailSelfCheck does it in one transaction; re-submission starts a new series.
API: self_check_max_attempts in GET/PUT /admin/config/orchestrator,
self_check_failed_on in /admin/ips/{ip} and /admin/registry/{ip}. Dashboard: the
field on /settings and the line "Self-check не прошёл на: ..." on the address
pages. Docs, plan and summary in docs/changes/.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b7669c9e41
commit
e95b5eb7d5
34 files changed
+1092
-82
No files matched your search
@@ -0,0 +1,38 @@
|
||||
-- Self-check failures per address (see
|
||||
-- docs/changes/2026-10-04_08-01_self-check-exclude-validator-plan.md).
|
||||
--
|
||||
-- A failed self-check no longer sends the address back to the validator that
|
||||
-- failed it: ClaimNextQueued skips an address for every validator that failed
|
||||
-- it in the current round. ip_self_check_failures is the permanent history of
|
||||
-- those failures; it is keyed by registry_id (like checks and events), so it
|
||||
-- outlives the ip_queue row and is not touched by re-checks. No foreign keys
|
||||
-- on purpose: the manual cleanup in docs/ADMIN_CLEANUP.md deletes freely.
|
||||
--
|
||||
-- A round is the part of a series of failures in which validators that failed
|
||||
-- stay excluded. ip_queue.sc_round_start_cycle is the first cycle_id of the
|
||||
-- current round: a failure excludes its validator only if its cycle_id is not
|
||||
-- below it. cycle_id (not attempt_number) is used because it never repeats for
|
||||
-- an address, even when the ip_queue row is deleted and created again.
|
||||
--
|
||||
-- ip_queue.sc_failures counts self-check failures of the current series; a
|
||||
-- manual re-check or re-submission starts a new series.
|
||||
--
|
||||
-- settings.self_check_max_attempts is the ceiling of self-check failures per
|
||||
-- address, after which the address gets the verdict fail (1..50, default 5).
|
||||
|
||||
CREATE TABLE ip_self_check_failures (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
registry_id INTEGER NOT NULL,
|
||||
run_id INTEGER,
|
||||
cycle_id INTEGER NOT NULL,
|
||||
attempt_number INTEGER NOT NULL,
|
||||
validator_id TEXT NOT NULL,
|
||||
failed_at TIMESTAMP NOT NULL,
|
||||
detail TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
CREATE INDEX idx_sc_failures_registry ON ip_self_check_failures(registry_id, validator_id);
|
||||
|
||||
ALTER TABLE ip_queue ADD COLUMN sc_failures INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE ip_queue ADD COLUMN sc_round_start_cycle INTEGER NOT NULL DEFAULT 0;
|
||||
|
||||
ALTER TABLE settings ADD COLUMN self_check_max_attempts INTEGER NOT NULL DEFAULT 5;
|
||||
Reference in new issue
Block a user