Commit Graph
81 Commits
Author SHA1 Message Date
vasyansk d5192e1cf7 Add optional pprof endpoint for debugging 2026-07-04 21:31:53 +07:00
vasyansk dca1363ec9 Add IMAP connection timeout handling
Introduce idleConn wrapper to prevent wedged connections
Add test for silent server timeout behavior
Implement proper TLS handshake and greeting handling
Set explicit dial and read timeouts
2026-07-04 19:37:36 +07:00
vasyansk 8656f46674 Create 2026-07-03-scheduled-task-runs.md 2026-07-03 13:59:12 +07:00
vasyanskandClaude Opus 4.8 e104a77de1 Merge feat/scheduled-task-runs: recurring task execution
A 30s in-process scheduler auto-runs tasks on a recurring interval
(1/3/6/12/24h). Interval is measured from the last run's completion; a task
already running is never re-triggered. Enabling a schedule requires all
accounts tested OK. A breaker disables the schedule and flags the task broken
(red badge in list + detail) when a scheduled run errors or panics. Next-run
time (browser-local) and a run-log modal (per-run status/totals) are shown.
Migration 0004 adds tasks.schedule_interval_seconds/schedule_anchor/broken and
runs.trigger; task delete cascades runs as before.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:40:39 +07:00
vasyansk 64bcb1d60b fix: anchor floor on re-enable; breaker on scheduled panic; propagate next-run err; reject negative interval; close phantom runs on startup 2026-07-03 13:31:58 +07:00
vasyansk e8acab6920 feat(web): schedule control, next-run, broken badge, run-log modal 2026-07-03 13:19:14 +07:00
vasyanskandClaude Opus 4.8 d2c69c6a5e feat(api): schedule + runs endpoints; next_run_at on task detail
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:13:10 +07:00
vasyansk be777dc908 feat(orchestrator): thread run trigger; breaker on scheduled runs with errors 2026-07-03 13:08:21 +07:00
vasyansk b9421d388c feat(scheduler): 30s polling scheduler with pure NextRun/dueTaskIDs 2026-07-03 13:07:05 +07:00
vasyansk 8f93dcd97b feat(store): task schedule columns, run trigger, scheduling queries 2026-07-03 13:01:20 +07:00
vasyanskandClaude Opus 4.8 e8f29064fb docs: spec for scheduled (recurring) task runs
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 12:40:26 +07:00
vasyanskandClaude Opus 4.8 1810497c2a Merge feat/per-account-folder-mapping: per-account folder mapping + exclusion
Move folder config from task-level to per-account (accounts.folder_mapping +
excluded_folders, migrating existing task mapping). Add per-account folders
button that re-probes an account and opens the mapping modal with sync
checkboxes to exclude folders. Orchestrator builds each account's plan via
planFolders honoring per-account mapping/exclusions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 12:10:57 +07:00
vasyansk b59ce67497 fix: validate account on mapping PUT; gate folders btn during run; prune stale mapping keys; drop dead export 2026-07-03 12:08:22 +07:00
vasyansk 964e4cfc33 feat(web): per-account folders button + edit-mapping modal 2026-07-03 11:56:47 +07:00
vasyansk 1ba7b54e05 feat(web): folder-sync checkboxes + per-account mapping API client 2026-07-03 11:55:40 +07:00
vasyansk bdb3b72f38 feat(api): probe existing account + persist per-account folder mapping 2026-07-03 11:50:32 +07:00
vasyansk bc72f8172f feat(orchestrator): build copy plan from per-account mapping + exclusions 2026-07-03 11:46:31 +07:00
vasyansk d6d17ee544 feat(store): per-account folder_mapping + excluded_folders columns 2026-07-03 11:39:21 +07:00
vasyanskandClaude Opus 4.8 e7870c6aa4 docs: spec + plan for per-account folder mapping & exclusion
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 11:35:17 +07:00
vasyansk 79fd200e57 many fixes 2026-07-03 11:18:40 +07:00
vasyansk d909618ced Create docker-compose-traefik.yml 2026-07-02 15:31:27 +07:00
vasyanskandClaude Opus 4.8 4b9c806719 Merge fix/error-reporting-and-modal-reset
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 15:14:11 +07:00
vasyanskandClaude Opus 4.8 477077d9e3 fix: consistent error reporting + reset mapping modal per account
Error consistency: folder-level failures were counted only toward the task's
done_with_errors status, not the account's error_count, so a task showed
DONE_WITH_ERRORS while its only account showed 0 errors / DONE. Now folder
errors increment the account counter and the account status becomes
done_with_errors when errs>0.

Visibility: persist accounts.last_error (migration 0002) so the failing folder
/ login error survives a page reload (shown red under the source login);
cleared at the start of each run.

Modal reset: the folder-mapping modal kept its selections across opens, so
adding a second account showed the first account's mapping. It now mounts
fresh per add (conditional render + key), reflecting the newly-probed folders.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 15:14:11 +07:00
vasyanskandClaude Opus 4.8 cdb93926bf Merge feat/folder-mapping-ui: probe + folder mapping modal on add
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 14:39:32 +07:00
vasyanskandClaude Opus 4.8 0bb584fe10 feat: folder mapping UI on account add (probe + map modal)
ADD now probes both connections, lists folders on each side, and opens a
mapping modal to route source->destination folders (e.g. Спам -> Spam) so we
append into the existing folder instead of creating a duplicate.

- store: SetTaskFolderMapping (+ round-trip test)
- httpapi: POST /tasks/{id}/probe (test both, return folder lists),
  PUT /tasks/{id}/folder-mapping
- web: FolderMappingModal (reuses Modal, size=lg), submitAccount probes then
  opens the modal; confirm creates the account and saves the task mapping

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 14:39:31 +07:00
vasyanskandClaude Opus 4.8 331497aff4 Merge feat/streaming-scan: streaming metadata scan with live feedback
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:35:08 +07:00
vasyanskandClaude Opus 4.8 a2234077df feat: stream the metadata scan instead of Collect
CopyFolder now streams envelope metadata via Next() in a first pass (dedup +
queue new messages), then streams bodies for new ones in a second pass —
no more blocking Collect of the whole folder with zero feedback, and memory
stays flat (only new-message meta is held).

- imapx: two-pass streaming CopyFolder + CopyDeps.OnScan(scanned,total)
- orchestrator: throttled 'scan' events during the metadata pass
- web: per-account 'scanning folder: X/N' line under the progress bar;
  scan events kept out of the log to avoid flooding

Verified on greenmail: idempotency and internal-date preservation still hold.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:35:08 +07:00
vasyanskandClaude Opus 4.8 39285a2ee7 Merge feat/plan-phase-progress: upfront folder count + account-wide progress
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:21:57 +07:00
vasyanskandClaude Opus 4.8 4959173f39 feat: planning pass counts all folders up front for account-wide progress
Before copying, EXAMINE every folder to sum the account's total message count
and emit a 'plan' event; progress events now carry account_total so the UI
shows a real overall bar, percent and ETA (not just per-folder).

- imapx.FolderMessageCount: read-only count of a folder
- orchestrator: plan pass + grandTotal, plan event, account_total in progress
- web: live progress keyed on account total; PLAN log line; overall bar/ETA

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:21:57 +07:00
vasyanskandClaude Opus 4.8 ca7c494a06 Merge feat/live-progress: live counters + progress bar/speed/ETA
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:14:55 +07:00
vasyanskandClaude Opus 4.8 e1911ef13b feat: real-time progress with per-folder bar, speed and ETA
- orchestrator: progress events now carry account-level cumulative copied/
  skipped plus current folder done/total, throttled to ~3/sec per account
- web: RUN CONTROL counters and account copied/skipped read live WS values
  (DB only advances per folder, so the summary lagged); new Progress column
  shows a bar, percent, avg messages/sec and folder ETA while running

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:14:55 +07:00
vasyanskandClaude Opus 4.8 6395099b53 Merge fix/stale-running-recovery: clear phantom running state
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:57:39 +07:00
vasyanskandClaude Opus 4.8 fa72f1b323 fix: recover from phantom 'running' state after crash/restart
The run-cancel registry is in-memory; a container restart mid-run leaves
accounts/tasks persisted as 'running' with no goroutine, wedging cancel
(not-in-map -> 409) and blocking remove/re-run.

- startup: ResetRunningOnStartup clears stale 'running' -> 'idle' on boot
- cancel handler: when no live goroutine, ClearStuckAccount + ReconcileTaskStatus
  reset the stuck account (and its task) instead of returning 409

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:57:39 +07:00
vasyanskandClaude Opus 4.8 6a10697548 Merge feat/cancel-and-folder-progress: cancel + folder counts
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:47:07 +07:00
vasyanskandClaude Opus 4.8 f024f329fc feat: per-account cancel + folder message-count progress in log
- orchestrator: per-account cancellable context registry + CancelAccount;
  on cancel, close IMAP connections to unblock in-flight FETCH; account ends
  in 'cancelled' status with a cancelled event
- imapx: CopyDeps.OnFolder callback fires after EXAMINE with the folder's
  message count (before the long fetch) for visibility
- httpapi: POST /tasks/{id}/accounts/{accountId}/cancel
- web: per-row cancel button while running, folder event shows N messages,
  cancelled/done_with_errors status badges

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:47:07 +07:00
vasyanskandClaude Opus 4.8 1ed150382e Merge feat/clear-log: clear event log button
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:32:04 +07:00
vasyanskandClaude Opus 4.8 694cedc529 feat(web): clear-log button on the event log panel
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:32:04 +07:00
vasyanskandClaude Opus 4.8 98dbdbd35b Merge fix/trim-account-credentials: trim manual account creds
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 10:38:00 +07:00
vasyanskandClaude Opus 4.8 79bd55aad8 fix(httpapi): trim whitespace on manually-added account login/password
Pasted app passwords (e.g. mail.ru) often carry a trailing space/newline that
the IMAP server rejects; the CSV path already trims, so make manual add match.
Source and destination passwords remain fully independent (src_pass_enc /
dst_pass_enc), verified end-to-end — this only strips surrounding whitespace.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 10:38:00 +07:00
vasyanskandClaude Opus 4.8 b8d1505329 Merge feat/confirm-modal: reusable modal + keyboard-driven confirm dialogs
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:36:08 +07:00
vasyanskandClaude Opus 4.8 29ccbc22e9 feat(web): reusable Modal + ConfirmProvider, replace native confirm()
Modal component: portal, ESC to close, Tab focus-trap, focus-on-open
(prefers [data-modal-autofocus]), focus restore, overlay click, scroll lock.
ConfirmProvider exposes useConfirm(): async confirm({...}) as a drop-in for
window.confirm; Enter confirms, ESC cancels. Task/account deletes now use it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:36:08 +07:00
vasyanskandClaude Opus 4.8 fcbe438f32 Merge feat/crud-and-rich-log: CRUD deletes/edit + detailed event log
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:23:14 +07:00
vasyanskandClaude Opus 4.8 b88f88c1a3 feat: delete task/account, edit endpoint, richer event log
- store: DeleteAccount, DeleteTask, UpdateEndpoint (+ cascade/update tests)
- httpapi: DELETE /tasks/{id}, DELETE /tasks/{id}/accounts/{accountId},
  PUT /endpoints/{id}; delete guarded with 409 while task running
- orchestrator: enrich WS events with login/host/port/error (test + run + errors)
- web: delete buttons (task, account) with confirm, endpoint edit form,
  human-readable event log (source/dest, host:port, error text)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:23:14 +07:00
vasyanskandClaude Opus 4.8 9019511d6f Merge feat/csv-example-download: downloadable CSV example link
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:09:08 +07:00
vasyanskandClaude Opus 4.8 647ea0dbbd feat(web): replace CSV columns caption with downloadable example.csv link
Generates a 3-row sample CSV client-side (Blob) so users see the exact
src_login,src_pass,dst_login,dst_pass layout instead of a plain caption.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:09:08 +07:00
vasyansk 53276c41f0 Update docker-compose.yml 2026-07-02 09:04:45 +07:00
vasyanskandClaude Opus 4.8 2a2fd7ad64 Merge fix/empty-list-null: empty lists return [] not null
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:04:33 +07:00
vasyanskandClaude Opus 4.8 7eff28053a fix(api): return empty arrays not null for empty lists (frontend null.length crash)
Root cause: store List* methods used 'var out []T' which stays nil on empty
result sets and serializes to JSON null; the SPA then crashed on .length/.map
(e.g. endpoints.length on the Tasks page right after deploy). Return []T{}
at the source; coerce null->[] on the frontend load sites as defense-in-depth.
Regression test asserts List* are non-nil when empty.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:04:33 +07:00
vasyanskandClaude Opus 4.8 d87a8c57ad Merge feat/imap-copier-mvp: imap-copier MVP
IMAP-to-IMAP mail migration tool: Go backend (streaming, non-destructive,
idempotent copy), React realtime UI, Postgres, Docker+Caddy. Full-stack E2E
proved idempotency; final review Critical (double-run race) fixed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-01 19:39:28 +07:00
vasyanskandClaude Opus 4.8 8af7b11791 fix(orchestrator): recover from panics in run goroutines to avoid process crash and stuck 'running' task
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-01 19:36:40 +07:00