Commit Graph

59 Commits

Author SHA1 Message Date
vasyansk 477077d9e3 fix: consistent error reporting + reset mapping modal per account
Error consistency: folder-level failures were counted only toward the task's
done_with_errors status, not the account's error_count, so a task showed
DONE_WITH_ERRORS while its only account showed 0 errors / DONE. Now folder
errors increment the account counter and the account status becomes
done_with_errors when errs>0.

Visibility: persist accounts.last_error (migration 0002) so the failing folder
/ login error survives a page reload (shown red under the source login);
cleared at the start of each run.

Modal reset: the folder-mapping modal kept its selections across opens, so
adding a second account showed the first account's mapping. It now mounts
fresh per add (conditional render + key), reflecting the newly-probed folders.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 15:14:11 +07:00
vasyansk cdb93926bf Merge feat/folder-mapping-ui: probe + folder mapping modal on add
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 14:39:32 +07:00
vasyansk 0bb584fe10 feat: folder mapping UI on account add (probe + map modal)
ADD now probes both connections, lists folders on each side, and opens a
mapping modal to route source->destination folders (e.g. Спам -> Spam) so we
append into the existing folder instead of creating a duplicate.

- store: SetTaskFolderMapping (+ round-trip test)
- httpapi: POST /tasks/{id}/probe (test both, return folder lists),
  PUT /tasks/{id}/folder-mapping
- web: FolderMappingModal (reuses Modal, size=lg), submitAccount probes then
  opens the modal; confirm creates the account and saves the task mapping

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 14:39:31 +07:00
vasyansk 331497aff4 Merge feat/streaming-scan: streaming metadata scan with live feedback
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:35:08 +07:00
vasyansk a2234077df feat: stream the metadata scan instead of Collect
CopyFolder now streams envelope metadata via Next() in a first pass (dedup +
queue new messages), then streams bodies for new ones in a second pass —
no more blocking Collect of the whole folder with zero feedback, and memory
stays flat (only new-message meta is held).

- imapx: two-pass streaming CopyFolder + CopyDeps.OnScan(scanned,total)
- orchestrator: throttled 'scan' events during the metadata pass
- web: per-account 'scanning folder: X/N' line under the progress bar;
  scan events kept out of the log to avoid flooding

Verified on greenmail: idempotency and internal-date preservation still hold.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:35:08 +07:00
vasyansk 39285a2ee7 Merge feat/plan-phase-progress: upfront folder count + account-wide progress
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:21:57 +07:00
vasyansk 4959173f39 feat: planning pass counts all folders up front for account-wide progress
Before copying, EXAMINE every folder to sum the account's total message count
and emit a 'plan' event; progress events now carry account_total so the UI
shows a real overall bar, percent and ETA (not just per-folder).

- imapx.FolderMessageCount: read-only count of a folder
- orchestrator: plan pass + grandTotal, plan event, account_total in progress
- web: live progress keyed on account total; PLAN log line; overall bar/ETA

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:21:57 +07:00
vasyansk ca7c494a06 Merge feat/live-progress: live counters + progress bar/speed/ETA
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:14:55 +07:00
vasyansk e1911ef13b feat: real-time progress with per-folder bar, speed and ETA
- orchestrator: progress events now carry account-level cumulative copied/
  skipped plus current folder done/total, throttled to ~3/sec per account
- web: RUN CONTROL counters and account copied/skipped read live WS values
  (DB only advances per folder, so the summary lagged); new Progress column
  shows a bar, percent, avg messages/sec and folder ETA while running

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 13:14:55 +07:00
vasyansk 6395099b53 Merge fix/stale-running-recovery: clear phantom running state
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:57:39 +07:00
vasyansk fa72f1b323 fix: recover from phantom 'running' state after crash/restart
The run-cancel registry is in-memory; a container restart mid-run leaves
accounts/tasks persisted as 'running' with no goroutine, wedging cancel
(not-in-map -> 409) and blocking remove/re-run.

- startup: ResetRunningOnStartup clears stale 'running' -> 'idle' on boot
- cancel handler: when no live goroutine, ClearStuckAccount + ReconcileTaskStatus
  reset the stuck account (and its task) instead of returning 409

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:57:39 +07:00
vasyansk 6a10697548 Merge feat/cancel-and-folder-progress: cancel + folder counts
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:47:07 +07:00
vasyansk f024f329fc feat: per-account cancel + folder message-count progress in log
- orchestrator: per-account cancellable context registry + CancelAccount;
  on cancel, close IMAP connections to unblock in-flight FETCH; account ends
  in 'cancelled' status with a cancelled event
- imapx: CopyDeps.OnFolder callback fires after EXAMINE with the folder's
  message count (before the long fetch) for visibility
- httpapi: POST /tasks/{id}/accounts/{accountId}/cancel
- web: per-row cancel button while running, folder event shows N messages,
  cancelled/done_with_errors status badges

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:47:07 +07:00
vasyansk 1ed150382e Merge feat/clear-log: clear event log button
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:32:04 +07:00
vasyansk 694cedc529 feat(web): clear-log button on the event log panel
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 12:32:04 +07:00
vasyansk 98dbdbd35b Merge fix/trim-account-credentials: trim manual account creds
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 10:38:00 +07:00
vasyansk 79bd55aad8 fix(httpapi): trim whitespace on manually-added account login/password
Pasted app passwords (e.g. mail.ru) often carry a trailing space/newline that
the IMAP server rejects; the CSV path already trims, so make manual add match.
Source and destination passwords remain fully independent (src_pass_enc /
dst_pass_enc), verified end-to-end — this only strips surrounding whitespace.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 10:38:00 +07:00
vasyansk b8d1505329 Merge feat/confirm-modal: reusable modal + keyboard-driven confirm dialogs
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:36:08 +07:00
vasyansk 29ccbc22e9 feat(web): reusable Modal + ConfirmProvider, replace native confirm()
Modal component: portal, ESC to close, Tab focus-trap, focus-on-open
(prefers [data-modal-autofocus]), focus restore, overlay click, scroll lock.
ConfirmProvider exposes useConfirm(): async confirm({...}) as a drop-in for
window.confirm; Enter confirms, ESC cancels. Task/account deletes now use it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:36:08 +07:00
vasyansk fcbe438f32 Merge feat/crud-and-rich-log: CRUD deletes/edit + detailed event log
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:23:14 +07:00
vasyansk b88f88c1a3 feat: delete task/account, edit endpoint, richer event log
- store: DeleteAccount, DeleteTask, UpdateEndpoint (+ cascade/update tests)
- httpapi: DELETE /tasks/{id}, DELETE /tasks/{id}/accounts/{accountId},
  PUT /endpoints/{id}; delete guarded with 409 while task running
- orchestrator: enrich WS events with login/host/port/error (test + run + errors)
- web: delete buttons (task, account) with confirm, endpoint edit form,
  human-readable event log (source/dest, host:port, error text)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:23:14 +07:00
vasyansk 9019511d6f Merge feat/csv-example-download: downloadable CSV example link
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:09:08 +07:00
vasyansk 647ea0dbbd feat(web): replace CSV columns caption with downloadable example.csv link
Generates a 3-row sample CSV client-side (Blob) so users see the exact
src_login,src_pass,dst_login,dst_pass layout instead of a plain caption.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:09:08 +07:00
vasyansk 53276c41f0 Update docker-compose.yml 2026-07-02 09:04:45 +07:00
vasyansk 2a2fd7ad64 Merge fix/empty-list-null: empty lists return [] not null
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:04:33 +07:00
vasyansk 7eff28053a fix(api): return empty arrays not null for empty lists (frontend null.length crash)
Root cause: store List* methods used 'var out []T' which stays nil on empty
result sets and serializes to JSON null; the SPA then crashed on .length/.map
(e.g. endpoints.length on the Tasks page right after deploy). Return []T{}
at the source; coerce null->[] on the frontend load sites as defense-in-depth.
Regression test asserts List* are non-nil when empty.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-02 09:04:33 +07:00
vasyansk d87a8c57ad Merge feat/imap-copier-mvp: imap-copier MVP
IMAP-to-IMAP mail migration tool: Go backend (streaming, non-destructive,
idempotent copy), React realtime UI, Postgres, Docker+Caddy. Full-stack E2E
proved idempotency; final review Critical (double-run race) fixed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-01 19:39:28 +07:00
vasyansk 8af7b11791 fix(orchestrator): recover from panics in run goroutines to avoid process crash and stuck 'running' task
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-01 19:36:40 +07:00
vasyansk 2429c786e4 fix(orchestrator): prevent concurrent double-run duplicating messages; reflect errors in status 2026-07-01 19:32:04 +07:00
vasyansk 1373aa0a77 feat(deploy): docker image, caddy, compose, e2e script
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-01 19:16:38 +07:00
vasyansk 38005c0618 fix(api): add snake_case json tags to Endpoint/Task/request bodies for frontend contract
Go's encoding/json does not bridge snake_case <-> PascalCase field names,
so store.Endpoint, store.Task and the anonymous request bodies in
accounts.go/auth.go were silently decoding empty/zero values from the
frontend's snake_case JSON contract (tls_mode, role_label,
src_endpoint_id, dst_endpoint_id, src_login/pass, dst_login/pass).
Adds explicit json tags; DB layer is unaffected since pgx binds by
positional params, not struct-tag reflection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-01 19:04:46 +07:00
vasyansk 1a451f9dbb feat(web): React SPA with realtime task detail over WebSocket
Vite + React 19 + TS console-style operator UI: hash-routed Login,
Endpoints, Tasks, and TaskDetail (realtime accounts table over /ws,
Run gated on all accounts testing ok on both sides).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-01 19:01:05 +07:00
vasyansk 4c57848c35 fix(httpapi): detect ws client disconnect via CloseRead to prevent subscriber leak
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-01 18:47:54 +07:00
vasyansk 9ec6acd414 feat(httpapi): websocket, router, embed static, main entrypoint 2026-07-01 18:37:48 +07:00
vasyansk 0bd4ba37e3 fix(httpapi): fail CSV import on encryption error instead of storing empty passwords 2026-07-01 18:33:09 +07:00
vasyansk bb83bbd989 feat(httpapi): REST resources for endpoints/tasks/accounts/csv/run 2026-07-01 18:27:11 +07:00
vasyansk 839febb83a fix(httpapi): bind session token to current AuthUser; add negative auth tests
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMHQTtnQtQqL8muAXHr9kd
2026-07-01 18:24:35 +07:00
vasyansk cae124931d feat(httpapi): env-based login and session auth middleware 2026-07-01 18:19:57 +07:00
vasyansk f9f01b981b fix(csvimport): report accurate physical line numbers via FieldPos; add blank-line + zero-row tests 2026-07-01 18:17:54 +07:00
vasyansk 7fe8896f4b feat(csvimport): validated CSV account parser 2026-07-01 18:12:57 +07:00
vasyansk 2def11a870 fix(orchestrator): reuse src connection to list folders instead of extra TestLogin 2026-07-01 18:10:52 +07:00
vasyansk 8c871d9d26 feat(orchestrator): worker pool run + account testing gate 2026-07-01 18:04:44 +07:00
vasyansk ec8835538b feat(wshub): per-task event hub with non-blocking publish 2026-07-01 17:59:43 +07:00
vasyansk a54ec1d148 fix(imapx): preserve source internal date on APPEND 2026-07-01 17:57:12 +07:00
vasyansk 06c0598b80 feat(imapx): streaming per-folder copy with dedup, idempotent 2026-07-01 17:39:43 +07:00
vasyansk 79f5c5b93a fix(imapx): honor context in Connect with bounded dial retry 2026-07-01 17:28:10 +07:00
vasyansk 0451b79c64 feat(imapx): connect, endpoint test, login test with folder listing 2026-07-01 17:20:03 +07:00
vasyansk 37cb8ba076 feat(imapx): message dedup key (Message-ID + header fallback) 2026-07-01 17:03:18 +07:00
vasyansk 67a2367baa feat(store): tasks, accounts, runs, dedup journal 2026-07-01 16:58:33 +07:00
vasyansk 0cf9de38c4 fix(store): close pool on failed Ping; add ListEndpoints test 2026-07-01 16:55:24 +07:00