ayurishchevandClaude Sonnet 5.5
0532baff09
A mass check on 2026-10-02 stalled 7 of 20 validators and sent 42 addresses to fail without a single check. A validator busy with slow checks went silent, was marked unreachable, and its next heartbeat put it back to idle while it still held the address; it was handed a second one, whose association never ran (the in-flight guard was keyed by validator), and both waited for their leases to expire. - Heartbeat/re-register return an unreachable validator to assigned when it still holds an address, else idle. - A validator is released only from the address it currently holds (ReleaseFIP, RequeueOrFail, MarkFIPOccupied, FreeValidator); an unreachable validator stays unreachable until its next heartbeat, so a dead validator is no longer handed a new address every lease period. - ClaimNextQueued refuses a validator that still has an address; a ReconcileValidators pass on every tick repairs rows that disagree with the queue. - Association guard is keyed by address, not validator. - The agent sends heartbeats from their own goroutine. - Clear queue / delete: detach only floating IPs of unfinished rows (done, failed and occupied rows kept their fip_id and made a clear issue >1000 sequential cloud calls: 256 s), at most 8 in parallel; the operation no longer dies with the client connection (10 minute limit). Includes the incident analysis and the plan under analysis/ and docs/changes/, and rebuilt bin/control-api and bin/validator-agent. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
43 lines
594 B
Plaintext
43 lines
594 B
Plaintext
37.139.32.70
|
|
37.139.32.71
|
|
37.139.32.72
|
|
37.139.32.151
|
|
37.139.33.219
|
|
37.139.33.227
|
|
37.139.33.230
|
|
37.139.34.75
|
|
37.139.34.94
|
|
37.139.34.128
|
|
37.139.40.122
|
|
37.139.41.103
|
|
37.139.41.129
|
|
37.139.41.157
|
|
37.139.41.202
|
|
37.139.42.126
|
|
37.139.43.9
|
|
79.137.174.150
|
|
79.137.174.172
|
|
79.137.174.205
|
|
79.137.174.210
|
|
79.137.175.14
|
|
79.137.175.51
|
|
79.137.175.71
|
|
79.137.175.162
|
|
83.166.232.116
|
|
83.166.232.117
|
|
83.166.232.159
|
|
83.166.233.37
|
|
83.166.233.75
|
|
83.166.233.78
|
|
83.166.234.136
|
|
83.166.235.75
|
|
83.166.235.132
|
|
83.166.235.247
|
|
83.166.237.59
|
|
83.166.237.228
|
|
83.166.238.19
|
|
83.166.248.57
|
|
83.166.248.60
|
|
83.166.248.92
|
|
83.166.248.188
|