Merge feat/per-account-folder-mapping: per-account folder mapping + exclusion

Move folder config from task-level to per-account (accounts.folder_mapping +
excluded_folders, migrating existing task mapping). Add per-account folders
button that re-probes an account and opens the mapping modal with sync
checkboxes to exclude folders. Orchestrator builds each account's plan via
planFolders honoring per-account mapping/exclusions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-03 12:10:57 +07:00
14 changed files with 1493 additions and 68 deletions
@@ -0,0 +1,873 @@
# Per-account Folder Mapping & Exclusion Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Give every account its own source→destination folder mapping and let the operator exclude folders from the copy, editable per account via a "folders" button.
**Architecture:** Move folder config from `tasks.folder_mapping` to two new per-account JSONB columns (`folder_mapping`, `excluded_folders`), migrating existing task data onto accounts. The orchestrator builds each account's copy plan from its own config via a pure, unit-tested `planFolders` helper. New HTTP endpoints re-probe an existing account (server-side password decrypt) and persist its mapping. The React `FolderMappingModal` gains per-folder sync checkboxes; a per-row "folders" button opens it in edit mode.
**Tech Stack:** Go 1.x (net/http, pgx v5, golang-migrate), Postgres 18, React 19 + Vite + TypeScript.
## Global Constraints
- Non-destructive: never drop `tasks.folder_mapping`; migration copies out of it.
- Default behavior must stay backward-compatible: empty mapping + empty exclusions = sync every folder name-matched.
- New source folders (not in an account's config) default to **synced**, never silently excluded.
- pgx v5 auto-(de)serializes JSONB into Go `map[string]string` / `[]string` — scan/pass the Go value directly (see `store/tasks.go` `FolderMapping`).
- Store tests require `TEST_DATABASE_URL`; they `t.Skip` without it. Migrations are applied by `psql` against the test DB before running (see Task 1 verify).
- Backend verification commands: `go build ./...`, `go vet ./...`, `go test ./...`.
- Frontend verification: `cd web && npm run build` (tsc + vite), `npx oxlint src/`.
- Commit after each task.
---
### Task 1: Per-account folder-config storage (migration + store)
**Files:**
- Create: `migrations/0003_account_folder_mapping.up.sql`
- Create: `migrations/0003_account_folder_mapping.down.sql`
- Modify: `internal/store/accounts.go` (Account struct ~L8-22; `ListAccountsByTask` SELECT/Scan ~L39-58; add `SetAccountFolderMapping`)
- Test: `internal/store/accounts_test.go`
**Interfaces:**
- Produces: `store.Account.FolderMapping map[string]string`, `store.Account.ExcludedFolders []string`; `func (s *Store) SetAccountFolderMapping(ctx context.Context, id int64, mapping map[string]string, excluded []string) error`.
- [ ] **Step 1: Write the migration up/down**
`migrations/0003_account_folder_mapping.up.sql`:
```sql
ALTER TABLE accounts ADD COLUMN folder_mapping JSONB NOT NULL DEFAULT '{}';
ALTER TABLE accounts ADD COLUMN excluded_folders JSONB NOT NULL DEFAULT '[]';
-- Carry the existing task-wide mapping onto its accounts so behavior is preserved.
UPDATE accounts a SET folder_mapping = t.folder_mapping
FROM tasks t WHERE a.task_id = t.id AND t.folder_mapping <> '{}'::jsonb;
```
`migrations/0003_account_folder_mapping.down.sql`:
```sql
ALTER TABLE accounts DROP COLUMN excluded_folders;
ALTER TABLE accounts DROP COLUMN folder_mapping;
```
- [ ] **Step 2: Write the failing store test**
Add to `internal/store/accounts_test.go`:
```go
func TestSetAccountFolderMapping(t *testing.T) {
s := testStore(t)
ctx := context.Background()
epSrc, _ := s.CreateEndpoint(ctx, Endpoint{RoleLabel: "src", Host: "a", Port: 993, TLSMode: "ssl"})
epDst, _ := s.CreateEndpoint(ctx, Endpoint{RoleLabel: "dst", Host: "b", Port: 993, TLSMode: "ssl"})
taskID, _ := s.CreateTask(ctx, Task{Name: "t", SrcEndpointID: epSrc, DstEndpointID: epDst})
accID, _ := s.CreateAccount(ctx, Account{TaskID: taskID, SrcLogin: "u", SrcPassEnc: "x", DstLogin: "u2", DstPassEnc: "y"})
// Fresh account defaults: empty map, empty exclusions (not nil after scan).
accs, _ := s.ListAccountsByTask(ctx, taskID)
if len(accs) != 1 || len(accs[0].FolderMapping) != 0 || len(accs[0].ExcludedFolders) != 0 {
t.Fatalf("defaults: map=%v excl=%v", accs[0].FolderMapping, accs[0].ExcludedFolders)
}
if err := s.SetAccountFolderMapping(ctx, accID,
map[string]string{"Спам": "Spam"}, []string{"Trash"}); err != nil {
t.Fatalf("set: %v", err)
}
accs, _ = s.ListAccountsByTask(ctx, taskID)
a := accs[0]
if a.FolderMapping["Спам"] != "Spam" {
t.Fatalf("mapping not persisted: %v", a.FolderMapping)
}
if len(a.ExcludedFolders) != 1 || a.ExcludedFolders[0] != "Trash" {
t.Fatalf("excluded not persisted: %v", a.ExcludedFolders)
}
}
```
- [ ] **Step 3: Run test to verify it fails**
Run: `go test ./internal/store/ -run TestSetAccountFolderMapping` (with a migrated `TEST_DATABASE_URL`).
Expected: FAIL to **compile**`a.FolderMapping`/`ExcludedFolders` and `SetAccountFolderMapping` undefined.
- [ ] **Step 4: Add struct fields**
In `internal/store/accounts.go`, extend the `Account` struct (after `LastError string`):
```go
LastError string
FolderMapping map[string]string
ExcludedFolders []string
```
- [ ] **Step 5: Select + scan the new columns**
In `ListAccountsByTask`, add both columns to the SELECT and the Scan. New SELECT:
```go
rows, err := s.Pool.Query(ctx,
`SELECT id, task_id, src_login, src_pass_enc, dst_login, dst_pass_enc,
test_src_status, test_dst_status, status, copied_count, skipped_count,
error_count, last_error, folder_mapping, excluded_folders
FROM accounts WHERE task_id=$1 ORDER BY id`, taskID)
```
New Scan (add the two trailing pointers):
```go
if err := rows.Scan(&a.ID, &a.TaskID, &a.SrcLogin, &a.SrcPassEnc, &a.DstLogin, &a.DstPassEnc,
&a.TestSrcStatus, &a.TestDstStatus, &a.Status, &a.Copied, &a.Skipped, &a.Errors, &a.LastError,
&a.FolderMapping, &a.ExcludedFolders); err != nil {
return nil, err
}
```
- [ ] **Step 6: Add the setter**
Append to `internal/store/accounts.go`:
```go
// SetAccountFolderMapping persists an account's per-folder rename map and the
// set of source folders to skip. nil is normalized to empty so JSONB stays
// '{}' / '[]' rather than null.
func (s *Store) SetAccountFolderMapping(ctx context.Context, id int64, mapping map[string]string, excluded []string) error {
if mapping == nil {
mapping = map[string]string{}
}
if excluded == nil {
excluded = []string{}
}
_, err := s.Pool.Exec(ctx,
`UPDATE accounts SET folder_mapping=$2, excluded_folders=$3 WHERE id=$1`,
id, mapping, excluded)
return err
}
```
- [ ] **Step 7: Run the test to verify it passes**
Run: `go test ./internal/store/ -run TestSetAccountFolderMapping -v`
Expected: PASS.
- [ ] **Step 8: Verify build + full store tests**
Run: `go build ./... && go vet ./... && go test ./internal/store/`
Expected: all pass.
- [ ] **Step 9: Commit**
```bash
git add migrations/0003_account_folder_mapping.up.sql migrations/0003_account_folder_mapping.down.sql \
internal/store/accounts.go internal/store/accounts_test.go
git commit -m "feat(store): per-account folder_mapping + excluded_folders columns"
```
---
### Task 2: Orchestrator plan honors per-account mapping + exclusions
**Files:**
- Modify: `internal/orchestrator/orchestrator.go` (lift `folderPlan` to package scope; add `planFolders`; replace the plan-building loop in `runAccount` ~L271-293)
- Test: Create `internal/orchestrator/planfolders_test.go`
**Interfaces:**
- Consumes: `store.Account.FolderMapping`, `store.Account.ExcludedFolders` (Task 1).
- Produces: package-level `type folderPlan struct { src, dst string; total int64 }`; `func planFolders(folders []string, mapping map[string]string, excluded []string) []folderPlan`.
- [ ] **Step 1: Write the failing unit test**
`internal/orchestrator/planfolders_test.go`:
```go
package orchestrator
import "testing"
func names(ps []folderPlan) []string {
out := make([]string, len(ps))
for i, p := range ps {
out[i] = p.src + "->" + p.dst
}
return out
}
func TestPlanFolders(t *testing.T) {
folders := []string{"INBOX", "Спам", "Trash", "Sent"}
// Empty config: every folder syncs to its own name, in order.
got := planFolders(folders, nil, nil)
if len(got) != 4 || got[0].src != "INBOX" || got[0].dst != "INBOX" {
t.Fatalf("empty config: %v", names(got))
}
// Rename + exclusion.
got = planFolders(folders,
map[string]string{"Спам": "Spam"},
[]string{"Trash"})
if len(got) != 3 {
t.Fatalf("want 3 plans (Trash excluded), got %v", names(got))
}
for _, p := range got {
if p.src == "Trash" {
t.Fatalf("excluded folder present: %v", names(got))
}
if p.src == "Спам" && p.dst != "Spam" {
t.Fatalf("rename not applied: %v", names(got))
}
if p.src == "INBOX" && p.dst != "INBOX" {
t.Fatalf("unmapped folder should keep its name: %v", names(got))
}
}
}
```
- [ ] **Step 2: Run test to verify it fails**
Run: `go test ./internal/orchestrator/ -run TestPlanFolders`
Expected: FAIL to compile — `folderPlan` is a local type inside `runAccount`, `planFolders` undefined.
- [ ] **Step 3: Lift `folderPlan` to package scope and add `planFolders`**
In `internal/orchestrator/orchestrator.go`, add near the top (package level, e.g. above `runAccount`):
```go
// folderPlan is one source folder scheduled for copy and its destination name.
type folderPlan struct {
src, dst string
total int64
}
// planFolders decides which of an account's source folders to copy and where.
// Excluded source folders are dropped; a folder absent from mapping keeps its
// own name. Order follows the input folder list.
func planFolders(folders []string, mapping map[string]string, excluded []string) []folderPlan {
skip := make(map[string]struct{}, len(excluded))
for _, e := range excluded {
skip[e] = struct{}{}
}
plan := make([]folderPlan, 0, len(folders))
for _, f := range folders {
if _, ok := skip[f]; ok {
continue
}
df := f
if m, ok := mapping[f]; ok && m != "" {
df = m
}
plan = append(plan, folderPlan{src: f, dst: df})
}
return plan
}
```
- [ ] **Step 4: Rewrite the plan-building loop in `runAccount`**
Replace the existing block (the local `type folderPlan struct {...}`, `plan := make(...)`, and the `for _, folder := range folders { ... df := folder; if m, ok := task.FolderMapping[folder]; ... }` loop, currently ~L271-293) with:
```go
// Planning pass: decide folders from the account's own config, then EXAMINE
// each to learn message counts for the overall progress bar.
plan := planFolders(folders, a.FolderMapping, a.ExcludedFolders)
var grandTotal int64
for i := range plan {
if actx.Err() != nil {
break
}
n, cerr := imapx.FolderMessageCount(src, plan[i].src)
if cerr != nil && actx.Err() == nil {
slog.Warn("count folder failed", "account", a.ID, "folder", plan[i].src, "err", cerr)
}
plan[i].total = n
grandTotal += n
}
```
Leave the subsequent `o.hub.Publish(... "plan" ...)` and the copy loop (`for _, fp := range plan`) unchanged — they already read `fp.src`, `fp.dst`, `fp.total`.
- [ ] **Step 5: Run the unit test to verify it passes**
Run: `go test ./internal/orchestrator/ -run TestPlanFolders -v`
Expected: PASS.
- [ ] **Step 6: Verify build + vet + full test suite**
Run: `go build ./... && go vet ./... && go test ./...`
Expected: all pass (store tests skip without `TEST_DATABASE_URL`; that's fine here).
- [ ] **Step 7: Commit**
```bash
git add internal/orchestrator/orchestrator.go internal/orchestrator/planfolders_test.go
git commit -m "feat(orchestrator): build copy plan from per-account mapping + exclusions"
```
---
### Task 3: HTTP — probe existing account + persist its mapping
**Files:**
- Modify: `internal/httpapi/accounts.go` (AccountView + accountDTO; add `handleProbeAccountFolders`, `handleSetAccountFolderMapping`)
- Modify: `internal/httpapi/router.go` (register two routes)
**Interfaces:**
- Consumes: `store.SetAccountFolderMapping`, `store.Account.FolderMapping/ExcludedFolders`, existing `crypto.Decrypt`, `imapx.TestLogin`, `s.store.ListAccountsByTask`.
- Produces: routes `POST /api/tasks/{id}/accounts/{accountId}/probe`, `PUT /api/tasks/{id}/accounts/{accountId}/folder-mapping`; `AccountView.FolderMapping`, `AccountView.ExcludedFolders`.
- [ ] **Step 1: Expose the config on AccountView**
In `internal/httpapi/accounts.go`, add to `AccountView` (after `LastError`):
```go
LastError string `json:"last_error,omitempty"`
FolderMapping map[string]string `json:"folder_mapping"`
ExcludedFolders []string `json:"excluded_folders"`
```
And in `accountDTO`, populate them:
```go
LastError: a.LastError,
FolderMapping: a.FolderMapping,
ExcludedFolders: a.ExcludedFolders,
```
- [ ] **Step 2: Add a helper to fetch one account by id**
The store exposes `ListAccountsByTask` only. Add a small lookup inside the handler (no new store method needed) — a private helper in `accounts.go`:
```go
// findAccount returns the account with accID under taskID, or ok=false.
func (s *Server) findAccount(r *http.Request, taskID, accID int64) (store.Account, bool) {
accs, err := s.store.ListAccountsByTask(r.Context(), taskID)
if err != nil {
return store.Account{}, false
}
for _, a := range accs {
if a.ID == accID {
return a, true
}
}
return store.Account{}, false
}
```
- [ ] **Step 3: Add the probe-by-account handler**
Append to `internal/httpapi/accounts.go`. It reuses the same probe shape as `handleProbeFolders` but decrypts stored passwords:
```go
// handleProbeAccountFolders re-probes an existing account using its stored
// (encrypted) credentials, so the operator can (re)map an account added via CSV.
func (s *Server) handleProbeAccountFolders(w http.ResponseWriter, r *http.Request) {
taskID, err := pathID(r, "id")
if err != nil {
http.Error(w, "bad id", http.StatusBadRequest)
return
}
accID, err := pathID(r, "accountId")
if err != nil {
http.Error(w, "bad account id", http.StatusBadRequest)
return
}
task, err := s.store.GetTask(r.Context(), taskID)
if err != nil {
http.Error(w, "not found", http.StatusNotFound)
return
}
a, ok := s.findAccount(r, taskID, accID)
if !ok {
http.Error(w, "account not found", http.StatusNotFound)
return
}
srcEP, err := s.store.GetEndpoint(r.Context(), task.SrcEndpointID)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
dstEP, err := s.store.GetEndpoint(r.Context(), task.DstEndpointID)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
srcPass, err := crypto.Decrypt(s.cfg.EncKey, a.SrcPassEnc)
if err != nil {
http.Error(w, "decrypt", http.StatusInternalServerError)
return
}
dstPass, err := crypto.Decrypt(s.cfg.EncKey, a.DstPassEnc)
if err != nil {
http.Error(w, "decrypt", http.StatusInternalServerError)
return
}
probe := func(ep store.Endpoint, login string, pass []byte) map[string]any {
folders, err := imapx.TestLogin(r.Context(),
imapx.Endpoint{Host: ep.Host, Port: ep.Port, TLSMode: ep.TLSMode},
strings.TrimSpace(login), strings.TrimSpace(string(pass)))
if err != nil {
return map[string]any{"ok": false, "error": err.Error(), "folders": []string{}}
}
return map[string]any{"ok": true, "folders": folders}
}
writeJSON(w, http.StatusOK, map[string]any{
"src": probe(srcEP, a.SrcLogin, srcPass),
"dst": probe(dstEP, a.DstLogin, dstPass),
})
}
```
- [ ] **Step 4: Add the per-account mapping setter handler**
Append to `internal/httpapi/accounts.go`:
```go
// handleSetAccountFolderMapping persists one account's rename map + excluded set.
func (s *Server) handleSetAccountFolderMapping(w http.ResponseWriter, r *http.Request) {
accID, err := pathID(r, "accountId")
if err != nil {
http.Error(w, "bad account id", http.StatusBadRequest)
return
}
var body struct {
Mapping map[string]string `json:"mapping"`
Excluded []string `json:"excluded"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
http.Error(w, "bad json", http.StatusBadRequest)
return
}
if err := s.store.SetAccountFolderMapping(r.Context(), accID, body.Mapping, body.Excluded); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusNoContent)
}
```
- [ ] **Step 5: Register the routes**
In `internal/httpapi/router.go`, after the existing `.../accounts/{accountId}/cancel` line:
```go
api.HandleFunc("POST /api/tasks/{id}/accounts/{accountId}/probe", s.handleProbeAccountFolders)
api.HandleFunc("PUT /api/tasks/{id}/accounts/{accountId}/folder-mapping", s.handleSetAccountFolderMapping)
```
- [ ] **Step 6: Verify build + vet**
Run: `go build ./... && go vet ./...`
Expected: pass (confirms `s.cfg.EncKey`, `crypto.Decrypt`, `imapx.TestLogin`, `writeJSON` all resolve).
- [ ] **Step 7: Commit**
```bash
git add internal/httpapi/accounts.go internal/httpapi/router.go
git commit -m "feat(api): probe existing account + persist per-account folder mapping"
```
---
### Task 4: Frontend API client + exclusion checkboxes in the mapping modal
**Files:**
- Modify: `web/src/api.ts` (Account fields; `probeAccountFolders`; `setAccountFolderMapping`)
- Modify: `web/src/components/FolderMappingModal.tsx` (checkbox column, `initialExcluded`, `onConfirm(mapping, excluded)`)
- Modify: `web/src/app.css` (checkbox + excluded-row styles)
**Interfaces:**
- Consumes: backend routes from Task 3.
- Produces: `probeAccountFolders(taskId, accId): Promise<ProbeResult>`; `setAccountFolderMapping(taskId, accId, mapping, excluded): Promise<unknown>`; `FolderMappingModal` prop `initialExcluded: string[]` and `onConfirm: (mapping: Record<string,string>, excluded: string[]) => void`.
- [ ] **Step 1: Extend the API client**
In `web/src/api.ts`, add to `Account` (after `last_error?`):
```ts
folder_mapping?: Record<string, string>
excluded_folders?: string[]
```
Add near the existing probe/mapping helpers:
```ts
export const probeAccountFolders = (taskId: number, accId: number) =>
api<ProbeResult>(`/api/tasks/${taskId}/accounts/${accId}/probe`, { method: 'POST' })
export const setAccountFolderMapping = (
taskId: number,
accId: number,
mapping: Record<string, string>,
excluded: string[],
) =>
api(`/api/tasks/${taskId}/accounts/${accId}/folder-mapping`, {
...jsonBody({ mapping, excluded }),
method: 'PUT',
})
```
- [ ] **Step 2: Add exclusion to FolderMappingModal — props + state**
In `web/src/components/FolderMappingModal.tsx`, change the `Props` type and signature:
```ts
type Props = {
open: boolean
srcFolders: string[]
dstFolders: string[]
initialMapping: Record<string, string>
initialExcluded: string[]
onConfirm: (mapping: Record<string, string>, excluded: string[]) => void
onCancel: () => void
}
export function FolderMappingModal({
open, srcFolders, dstFolders, initialMapping, initialExcluded, onConfirm, onCancel,
}: Props) {
const [choice, setChoice] = useState<Record<string, string>>({})
const [synced, setSynced] = useState<Record<string, boolean>>(() => {
const excl = new Set(initialExcluded)
return Object.fromEntries(srcFolders.map((f) => [f, !excl.has(f)]))
})
```
- [ ] **Step 3: Update `confirm()` to emit excluded**
Replace the `confirm()` body:
```ts
function confirm() {
const mapping: Record<string, string> = { ...initialMapping }
const excluded: string[] = []
for (const src of srcFolders) {
if (synced[src] === false) {
excluded.push(src)
delete mapping[src]
continue
}
const dst = valueFor(src)
if (dst === src) delete mapping[src]
else mapping[src] = dst
}
onConfirm(mapping, excluded)
}
```
- [ ] **Step 4: Add the checkbox + "sync all" to the JSX**
Add a select-all control above `.map-grid` and a checkbox per row. Replace the `.map-grid` block:
```tsx
<label className="map-all">
<input
type="checkbox"
checked={srcFolders.every((f) => synced[f] !== false)}
onChange={(e) => {
const on = e.target.checked
setSynced(Object.fromEntries(srcFolders.map((f) => [f, on])))
}}
/>
sync all folders
</label>
<div className="map-grid">
{srcFolders.map((src) => {
const on = synced[src] !== false
const val = valueFor(src)
const creates = !dstFolders.includes(val)
return (
<div className={`map-row${on ? '' : ' map-row-off'}`} key={src}>
<label className="map-check">
<input
type="checkbox"
checked={on}
aria-label={`Sync folder ${src}`}
onChange={(e) => setSynced((s) => ({ ...s, [src]: e.target.checked }))}
/>
</label>
<span className="map-src" title={src}>
{src}
</span>
<span className="map-arrow"></span>
<select
className="map-select"
aria-label={`Destination folder for ${src}`}
value={val}
disabled={!on}
onChange={(e) => setChoice((c) => ({ ...c, [src]: e.target.value }))}
>
{options(src).map((f) => (
<option key={f} value={f}>
{f}
{f === src && !dstFolders.includes(src) ? ' (create)' : ''}
</option>
))}
</select>
{on ? (
creates && <span className="map-new">new</span>
) : (
<span className="map-excluded">excluded</span>
)}
</div>
)
})}
</div>
```
Note: `.map-row` grid now has a leading checkbox column — update its `grid-template-columns` in Step 5.
- [ ] **Step 5: Add CSS for the checkbox column + excluded row**
In `web/src/app.css`, replace the `.map-row` rule and add new rules:
```css
.map-row {
display: grid;
grid-template-columns: auto 1fr auto 1fr auto;
align-items: center;
gap: 10px;
}
.map-row-off .map-src,
.map-row-off .map-arrow {
color: var(--fg-faint);
text-decoration: line-through;
}
.map-check {
display: flex;
align-items: center;
}
.map-check input,
.map-all input {
accent-color: var(--accent);
cursor: pointer;
}
.map-all {
display: flex;
align-items: center;
gap: 8px;
margin-bottom: 12px;
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--fg-dim);
cursor: pointer;
}
.map-excluded {
font-size: 10px;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--fg-faint);
}
```
- [ ] **Step 6: Verify build + lint**
Run: `cd web && npm run build && npx oxlint src/`
Expected: build succeeds; only the two pre-existing oxlint warnings (`ConfirmProvider` fast-refresh, `TaskDetail` exhaustive-deps). This step will still show a type error at the `FolderMappingModal` call site in `TaskDetail.tsx` (missing `initialExcluded`, new `onConfirm` arity) — that is fixed in Task 5. If building Task 4 alone, temporarily expect that call-site error; it resolves once Task 5 lands. (Preferred: run Task 5 before the final build.)
- [ ] **Step 7: Commit**
```bash
git add web/src/api.ts web/src/components/FolderMappingModal.tsx web/src/app.css
git commit -m "feat(web): folder-sync checkboxes + per-account mapping API client"
```
---
### Task 5: Frontend TaskDetail — "folders" button, edit mode, per-account save
**Files:**
- Modify: `web/src/pages/TaskDetail.tsx`
**Interfaces:**
- Consumes: `probeAccountFolders`, `setAccountFolderMapping` (Task 4); `FolderMappingModal` new props (Task 4); `Account.folder_mapping/excluded_folders` (Task 4).
- [ ] **Step 1: Import the new API functions**
In `web/src/pages/TaskDetail.tsx`, extend the import from `../api` to add `probeAccountFolders` and `setAccountFolderMapping` (keep the rest). Remove `setFolderMapping` from the import (single-add stops using it — see Step 4):
```ts
import { cancelAccount, createAccount, deleteAccount, getTask, importCSV, probeAccountFolders, probeFolders, runTask, setAccountFolderMapping, setFolderMapping, testAccounts, type TaskDetail as TaskDetailData } from '../api'
```
(Keep `setFolderMapping` in the import only if still referenced; after Step 4 it is not — drop it to avoid an unused-import lint error.)
- [ ] **Step 2: Add edit-modal state**
Next to `const [mapState, setMapState] = useState<...>(null)`, add:
```ts
const [editMap, setEditMap] = useState<{
accId: number
src: string[]
dst: string[]
mapping: Record<string, string>
excluded: string[]
} | null>(null)
```
- [ ] **Step 3: Add the open-editor + save handlers**
Add two functions inside the component (near `onDeleteAccount`):
```ts
async function onEditFolders(a: { id: number; src_login: string; folder_mapping?: Record<string, string>; excluded_folders?: string[] }) {
setBusy('probe')
setError(null)
try {
const res = await probeAccountFolders(id, a.id)
if (!res.src.ok || !res.dst.ok) {
const parts: string[] = []
if (!res.src.ok) parts.push(`source: ${res.src.error ?? 'login failed'}`)
if (!res.dst.ok) parts.push(`destination: ${res.dst.error ?? 'login failed'}`)
setError(`Folder probe failed — ${parts.join('; ')}`)
return
}
setEditMap({
accId: a.id,
src: res.src.folders ?? [],
dst: res.dst.folders ?? [],
mapping: a.folder_mapping ?? {},
excluded: a.excluded_folders ?? [],
})
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to probe folders')
} finally {
setBusy(null)
}
}
async function saveEditMapping(mapping: Record<string, string>, excluded: string[]) {
if (!editMap) return
setBusy('add')
setError(null)
try {
await setAccountFolderMapping(id, editMap.accId, mapping, excluded)
setEditMap(null)
reload()
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to save mapping')
} finally {
setBusy(null)
}
}
```
- [ ] **Step 4: Switch single-add to per-account save**
In `confirmMapping`, replace the task-level call. Change `onConfirm` signature to accept `excluded` and persist to the new account:
```ts
async function confirmMapping(mapping: Record<string, string>, excluded: string[]) {
if (!mapState) return
setBusy('add')
setError(null)
try {
const { id: accId } = await createAccount(id, mapState.creds)
await setAccountFolderMapping(id, accId, mapping, excluded)
setForm(emptyAccount)
setMapState(null)
reload()
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to add account')
} finally {
setBusy(null)
}
}
```
(`createAccount` already returns `{ id }`.) After this, `setFolderMapping` is unused — ensure it is removed from the Step 1 import.
- [ ] **Step 5: Add the "folders" button to the actions cell**
In the accounts table actions cell, put a `folders` button left of remove/cancel. Replace the last `<td className="num-cell">…</td>` action cell with:
```tsx
<td className="num-cell">
<div className="row-actions">
{a.status !== 'running' && (
<button
type="button"
className="link-btn"
onClick={() => onEditFolders(a)}
disabled={busy !== null}
>
folders
</button>
)}
{a.status === 'running' ? (
<button type="button" className="link-btn danger" onClick={() => onCancelAccount(a.id)}>
cancel
</button>
) : (
<button
type="button"
className="link-btn danger"
onClick={() => onDeleteAccount(a.id, a.src_login)}
disabled={busy !== null || data?.task.status === 'running'}
>
remove
</button>
)}
</div>
</td>
```
- [ ] **Step 6: Pass `initialExcluded` to the add-modal and render the edit-modal**
Update the existing add-modal usage to pass `initialExcluded={[]}` (new accounts sync all) — `onConfirm={confirmMapping}` now matches the `(mapping, excluded)` arity:
```tsx
{mapState && (
<FolderMappingModal
key={`${mapState.creds.src_login}|${mapState.creds.dst_login}`}
open
srcFolders={mapState.src}
dstFolders={mapState.dst}
initialMapping={task.folder_mapping ?? {}}
initialExcluded={[]}
onCancel={() => setMapState(null)}
onConfirm={confirmMapping}
/>
)}
{editMap && (
<FolderMappingModal
key={`edit-${editMap.accId}`}
open
srcFolders={editMap.src}
dstFolders={editMap.dst}
initialMapping={editMap.mapping}
initialExcluded={editMap.excluded}
onCancel={() => setEditMap(null)}
onConfirm={saveEditMapping}
/>
)}
```
- [ ] **Step 7: Add `.row-actions` CSS**
In `web/src/app.css`, add:
```css
.row-actions {
display: inline-flex;
gap: 12px;
align-items: center;
justify-content: flex-end;
}
```
- [ ] **Step 8: Verify build + lint**
Run: `cd web && npm run build && npx oxlint src/`
Expected: build succeeds; only the two pre-existing oxlint warnings. No type errors.
- [ ] **Step 9: Commit**
```bash
git add web/src/pages/TaskDetail.tsx web/src/app.css
git commit -m "feat(web): per-account folders button + edit-mapping modal"
```
---
### Task 6: End-to-end verification on prod
Per project rules, functional verification runs on prod after deploy (Web via playwright-cli). This task has no code; it gates "done".
- [ ] **Step 1: Build + apply migration + deploy**
Run: `make build && make up` (migration 0003 auto-applies at startup via golang-migrate). Wait for `curl -fsS http://localhost/healthz`.
- [ ] **Step 2: Migration sanity — existing accounts preserved**
Open an existing task that had a task-level mapping. Confirm accounts still run and copy the same folders (migrated mapping carried over).
- [ ] **Step 3: Per-account exclusion + rename E2E**
1. CSV-import (or add) an account. In the accounts table, click **folders**.
2. Modal opens with the account's freshly probed source folders, all checked.
3. Uncheck one folder (e.g. `Trash`) → its select greys out, shows "excluded".
4. Rename another (e.g. `Спам → Spam`). Save.
5. Reopen **folders** on the same account → the excluded folder is unchecked and the rename is preserved (state re-read from DB).
6. Run the task. Verify: the excluded folder is NOT copied (absent from progress/log), the renamed folder lands under its new name on the destination.
- [ ] **Step 4: Record result**
Save an E2E note under `./swarm-report/per-account-folder-mapping-<YYYY-MM-DD>.md` with pass/fail per step.
---
## Self-Review
- **Spec coverage:** per-account model (Task 1) ✓; migration of task mapping (Task 1 up.sql) ✓; exclusion list w/ safe default (Task 1 defaults + Task 2 planFolders) ✓; orchestrator honors config (Task 2) ✓; probe-by-account decrypt (Task 3) ✓; PUT per-account mapping (Task 3) ✓; AccountView exposes config (Task 3) ✓; api.ts + modal checkboxes (Task 4) ✓; folders button + edit mode + single-add switch (Task 5) ✓; tests: store round-trip + defaults (Task 1), planFolders unit (Task 2), E2E (Task 6) ✓.
- **Placeholder scan:** none — all steps carry real code/commands.
- **Type consistency:** `folderPlan{src,dst,total}` package-level (Task 2) matches copy-loop usage; `SetAccountFolderMapping(id, mapping, excluded)` identical across Tasks 1/3; `onConfirm(mapping, excluded)` arity consistent across modal (Task 4) and both call sites (Task 5); `probeAccountFolders`/`setAccountFolderMapping` signatures match between api.ts (Task 4) and TaskDetail (Task 5).
- **Note:** Task 4 built in isolation leaves a transient type error at the modal call site until Task 5; documented in Task 4 Step 6. Prefer running Tasks 4→5 back-to-back before a clean build.
@@ -0,0 +1,160 @@
# Per-account folder mapping & folder exclusion — design
**Date:** 2026-07-03
**Status:** awaiting user review
## Context
Two gaps in the current folder-mapping feature:
1. **No per-account mapping for bulk (CSV) accounts.** Folder mapping is collected
only in the single "Add account" flow. Accounts imported from CSV inherit the
task-wide mapping and cannot be mapped individually. The user wants a **folders**
button per account row (left of remove) that re-probes that account's live
folders and opens the mapping modal for it.
2. **No exclusion.** Mapping can only rename `src → dst`; it cannot skip folders
the user doesn't want to copy (Spam, Trash, per-client junk). The user wants
**checkboxes** to choose which source folders are synced.
### Current architecture (as-is)
- `tasks.folder_mapping` (JSONB `map[src]dst`) — **one mapping for the whole task**,
applied to every account by the orchestrator: `df := task.FolderMapping[folder]`
(`internal/orchestrator/orchestrator.go:283-286`).
- Single-add flow: `probeFolders` (form creds) → `FolderMappingModal``createAccount`
then `setFolderMapping` (task-level) (`TaskDetail.tsx:167-204`).
- `handleProbeFolders` takes **plaintext** creds from the request body
(`internal/httpapi/accounts.go:39-83`). CSV accounts store only encrypted
passwords, so re-probing an existing account needs a server-side decrypt path.
- `store.Account` has **no** mapping field.
## Decision: fully per-account model
Folder configuration moves from the task to the account. `tasks.folder_mapping`
is retired as the source of truth (column kept, no longer read) and its data is
migrated onto each account. This matches the intent ("each account its own
folders") and removes the current oddity where adding one account rewrites the
whole task's mapping.
Exclusion is stored as a **separate `excluded_folders` list** (not a sentinel in
the mapping). Rationale: a source folder that is *not configured* (new folder
appearing after mapping, or a CSV account never opened) must **default to synced**
— a copy tool must never silently drop mail. An exclusion list gives that safe
default; an inclusion list would silently skip new folders.
### Semantics
For an account, given its live source folders:
- `excluded_folders: string[]` — source folder names to **skip** entirely.
- `folder_mapping: map[src]dst` — rename for synced folders; a src absent from the
map syncs to its own name (existing behavior).
- Default (empty mapping, empty exclusions) = sync every folder name-matched =
current behavior. Backward compatible.
## Data model
**Migration `0003_account_folder_mapping`:**
```sql
-- up
ALTER TABLE accounts ADD COLUMN folder_mapping JSONB NOT NULL DEFAULT '{}';
ALTER TABLE accounts ADD COLUMN excluded_folders JSONB NOT NULL DEFAULT '[]';
-- carry the existing task-wide mapping onto its accounts
UPDATE accounts a SET folder_mapping = t.folder_mapping
FROM tasks t WHERE a.task_id = t.id AND t.folder_mapping <> '{}';
-- down
ALTER TABLE accounts DROP COLUMN excluded_folders;
ALTER TABLE accounts DROP COLUMN folder_mapping;
```
**`store.Account`** gains:
- `FolderMapping map[string]string`
- `ExcludedFolders []string`
`ListAccountsByTask` selects/scans both (JSONB). New helper:
- `SetAccountFolderMapping(ctx, id int64, mapping map[string]string, excluded []string) error`
`tasks.folder_mapping` column and `SetTaskFolderMapping` remain but are no longer
read by the orchestrator (kept for the migration source and to avoid a destructive
change; single-add stops writing it).
## Orchestrator
Extract the per-folder plan decision into a small **pure, testable** helper so the
exclusion/mapping logic can be unit-tested without IMAP:
```go
// planFolders returns the (src,dst) pairs to copy for an account, dropping
// excluded source folders and applying renames.
func planFolders(folders []string, mapping map[string]string, excluded []string) []folderPlan
```
`runAccount` replaces the inline `task.FolderMapping[folder]` loop with
`planFolders(folders, a.FolderMapping, a.ExcludedFolders)`, then counts messages
per surviving folder as today. Excluded folders never enter the plan (not counted,
not scanned, not copied).
## HTTP API
- **`POST /api/tasks/{id}/accounts/{accId}/probe`** — new. Loads the account,
decrypts its stored src/dst passwords, reuses the existing `imapx.TestLogin`
probe, returns `{src:{ok,folders,error}, dst:{...}}`. No creds in the request.
- **`PUT /api/tasks/{id}/accounts/{accId}/folder-mapping`** — new. Body
`{mapping: map[string]string, excluded: string[]}``SetAccountFolderMapping`.
- **`AccountView` / `accountDTO`** expose `folder_mapping` and `excluded_folders`
so the modal can seed saved state when reopened.
- Existing task-level `POST /probe` stays (single-add probes before the account
exists). Task-level `PUT /folder-mapping` handler stays but is unused by the UI.
## Frontend
**`api.ts`**
- `Account` interface gains `folder_mapping?: Record<string,string>` and
`excluded_folders?: string[]`.
- `probeAccountFolders(taskId, accId)``ProbeResult`.
- `setAccountFolderMapping(taskId, accId, mapping, excluded)`.
**`FolderMappingModal`** — add exclusion:
- New props `initialExcluded: string[]`, and `onConfirm(mapping, excluded)`.
- Each row gets a leading **checkbox** (checked = sync). Unchecking greys/disables
that row's dst `<select>` and the "new/create" marker, and adds the src to the
excluded set.
- A header "sync all" checkbox toggles every row.
- `confirm()` emits `mapping` (only for synced rows) and `excluded` (unchecked rows).
**`TaskDetail`**
- Actions cell: add a `folders` link-btn **left of** remove/cancel, shown for every
account that is not currently running. Click → `probeAccountFolders(accId)`
open the modal in **edit-account** mode seeded with the account's saved
`folder_mapping` + `excluded_folders` and the freshly probed folders.
- New state `editMap: { accId; src: string[]; dst: string[]; mapping; excluded } | null`,
mounted like the existing add-modal with a `key` per account for clean remount.
- On confirm in edit mode → `setAccountFolderMapping(accId, mapping, excluded)` → reload.
- Single-add `confirmMapping` switches from task-level `setFolderMapping` to
`setAccountFolderMapping(newAccId, mapping, excluded)` after `createAccount`.
- Probe button shows a busy state (`busy: 'probe'`) while re-probing an account.
## Error handling
- Probe-by-account failures (login fails, IMAP down) surface via the existing
`error` banner (now `role="alert"`), same as single-add probe failures.
- Saving mapping while a run is in progress is blocked: the `folders` button is
hidden/disabled for `running` accounts (mapping is read at run start).
## Testing
- **store**: `SetAccountFolderMapping` round-trips mapping+excluded via
`ListAccountsByTask`; migration 0003 copies `tasks.folder_mapping` onto accounts.
- **orchestrator**: unit-test the extracted `planFolders` — excluded folders
dropped, renames applied, absent-src keeps its name, empty config = all folders.
- **httpapi**: probe-by-account decrypts and returns folders; PUT persists.
- **frontend / e2e**: `/run` the app — CSV-import an account, click **folders**,
uncheck a folder + rename another, save, run, verify the excluded folder is not
copied and the rename lands. Reduced to a manual E2E checklist per project rules.
## Out of scope (YAGNI)
- Task-wide "apply to all accounts" bulk mapping editor.
- Regex/glob folder matching.
- Removing the `tasks.folder_mapping` column (kept to avoid a destructive migration).
+118 -11
View File
@@ -12,16 +12,18 @@ import (
) )
type AccountView struct { type AccountView struct {
ID int64 `json:"id"` ID int64 `json:"id"`
SrcLogin string `json:"src_login"` SrcLogin string `json:"src_login"`
DstLogin string `json:"dst_login"` DstLogin string `json:"dst_login"`
TestSrcStatus string `json:"test_src_status"` TestSrcStatus string `json:"test_src_status"`
TestDstStatus string `json:"test_dst_status"` TestDstStatus string `json:"test_dst_status"`
Status string `json:"status"` Status string `json:"status"`
Copied int64 `json:"copied"` Copied int64 `json:"copied"`
Skipped int64 `json:"skipped"` Skipped int64 `json:"skipped"`
Errors int64 `json:"errors"` Errors int64 `json:"errors"`
LastError string `json:"last_error,omitempty"` LastError string `json:"last_error,omitempty"`
FolderMapping map[string]string `json:"folder_mapping"`
ExcludedFolders []string `json:"excluded_folders"`
} }
func accountDTO(a store.Account) AccountView { func accountDTO(a store.Account) AccountView {
@@ -29,7 +31,9 @@ func accountDTO(a store.Account) AccountView {
ID: a.ID, SrcLogin: a.SrcLogin, DstLogin: a.DstLogin, ID: a.ID, SrcLogin: a.SrcLogin, DstLogin: a.DstLogin,
TestSrcStatus: a.TestSrcStatus, TestDstStatus: a.TestDstStatus, TestSrcStatus: a.TestSrcStatus, TestDstStatus: a.TestDstStatus,
Status: a.Status, Copied: a.Copied, Skipped: a.Skipped, Errors: a.Errors, Status: a.Status, Copied: a.Copied, Skipped: a.Skipped, Errors: a.Errors,
LastError: a.LastError, LastError: a.LastError,
FolderMapping: a.FolderMapping,
ExcludedFolders: a.ExcludedFolders,
} }
} }
@@ -149,3 +153,106 @@ func (s *Server) handleCreateAccount(w http.ResponseWriter, r *http.Request) {
} }
writeJSON(w, http.StatusCreated, map[string]int64{"id": id}) writeJSON(w, http.StatusCreated, map[string]int64{"id": id})
} }
// findAccount returns the account with accID under taskID, or ok=false.
func (s *Server) findAccount(r *http.Request, taskID, accID int64) (store.Account, bool) {
accs, err := s.store.ListAccountsByTask(r.Context(), taskID)
if err != nil {
return store.Account{}, false
}
for _, a := range accs {
if a.ID == accID {
return a, true
}
}
return store.Account{}, false
}
// handleProbeAccountFolders re-probes an existing account using its stored
// (encrypted) credentials, so the operator can (re)map an account added via CSV.
func (s *Server) handleProbeAccountFolders(w http.ResponseWriter, r *http.Request) {
taskID, err := pathID(r, "id")
if err != nil {
http.Error(w, "bad id", http.StatusBadRequest)
return
}
accID, err := pathID(r, "accountId")
if err != nil {
http.Error(w, "bad account id", http.StatusBadRequest)
return
}
task, err := s.store.GetTask(r.Context(), taskID)
if err != nil {
http.Error(w, "not found", http.StatusNotFound)
return
}
a, ok := s.findAccount(r, taskID, accID)
if !ok {
http.Error(w, "account not found", http.StatusNotFound)
return
}
srcEP, err := s.store.GetEndpoint(r.Context(), task.SrcEndpointID)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
dstEP, err := s.store.GetEndpoint(r.Context(), task.DstEndpointID)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
srcPass, err := crypto.Decrypt(s.cfg.EncKey, a.SrcPassEnc)
if err != nil {
http.Error(w, "decrypt", http.StatusInternalServerError)
return
}
dstPass, err := crypto.Decrypt(s.cfg.EncKey, a.DstPassEnc)
if err != nil {
http.Error(w, "decrypt", http.StatusInternalServerError)
return
}
probe := func(ep store.Endpoint, login string, pass []byte) map[string]any {
folders, err := imapx.TestLogin(r.Context(),
imapx.Endpoint{Host: ep.Host, Port: ep.Port, TLSMode: ep.TLSMode},
strings.TrimSpace(login), strings.TrimSpace(string(pass)))
if err != nil {
return map[string]any{"ok": false, "error": err.Error(), "folders": []string{}}
}
return map[string]any{"ok": true, "folders": folders}
}
writeJSON(w, http.StatusOK, map[string]any{
"src": probe(srcEP, a.SrcLogin, srcPass),
"dst": probe(dstEP, a.DstLogin, dstPass),
})
}
// handleSetAccountFolderMapping persists one account's rename map + excluded set.
func (s *Server) handleSetAccountFolderMapping(w http.ResponseWriter, r *http.Request) {
taskID, err := pathID(r, "id")
if err != nil {
http.Error(w, "bad id", http.StatusBadRequest)
return
}
accID, err := pathID(r, "accountId")
if err != nil {
http.Error(w, "bad account id", http.StatusBadRequest)
return
}
if _, ok := s.findAccount(r, taskID, accID); !ok {
http.Error(w, "account not found", http.StatusNotFound)
return
}
var body struct {
Mapping map[string]string `json:"mapping"`
Excluded []string `json:"excluded"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
http.Error(w, "bad json", http.StatusBadRequest)
return
}
if err := s.store.SetAccountFolderMapping(r.Context(), accID, body.Mapping, body.Excluded); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusNoContent)
}
+2
View File
@@ -27,6 +27,8 @@ func (s *Server) Router() http.Handler {
api.HandleFunc("POST /api/tasks/{id}/test", s.handleTestAccounts) api.HandleFunc("POST /api/tasks/{id}/test", s.handleTestAccounts)
api.HandleFunc("POST /api/tasks/{id}/run", s.handleRun) api.HandleFunc("POST /api/tasks/{id}/run", s.handleRun)
api.HandleFunc("POST /api/tasks/{id}/accounts/{accountId}/cancel", s.handleCancelAccount) api.HandleFunc("POST /api/tasks/{id}/accounts/{accountId}/cancel", s.handleCancelAccount)
api.HandleFunc("POST /api/tasks/{id}/accounts/{accountId}/probe", s.handleProbeAccountFolders)
api.HandleFunc("PUT /api/tasks/{id}/accounts/{accountId}/folder-mapping", s.handleSetAccountFolderMapping)
mux.Handle("/api/", s.requireAuth(api)) mux.Handle("/api/", s.requireAuth(api))
mux.Handle("/ws", s.requireAuth(http.HandlerFunc(s.handleWS))) mux.Handle("/ws", s.requireAuth(http.HandlerFunc(s.handleWS)))
+35 -15
View File
@@ -16,6 +16,34 @@ import (
var ErrNotTested = errors.New("accounts not fully tested") var ErrNotTested = errors.New("accounts not fully tested")
var ErrAlreadyRunning = errors.New("task already running") var ErrAlreadyRunning = errors.New("task already running")
// folderPlan is one source folder scheduled for copy and its destination name.
type folderPlan struct {
src, dst string
total int64
}
// planFolders decides which of an account's source folders to copy and where.
// Excluded source folders are dropped; a folder absent from mapping keeps its
// own name. Order follows the input folder list.
func planFolders(folders []string, mapping map[string]string, excluded []string) []folderPlan {
skip := make(map[string]struct{}, len(excluded))
for _, e := range excluded {
skip[e] = struct{}{}
}
plan := make([]folderPlan, 0, len(folders))
for _, f := range folders {
if _, ok := skip[f]; ok {
continue
}
df := f
if m, ok := mapping[f]; ok && m != "" {
df = m
}
plan = append(plan, folderPlan{src: f, dst: df})
}
return plan
}
type Orchestrator struct { type Orchestrator struct {
store *store.Store store *store.Store
hub *wshub.Hub hub *wshub.Hub
@@ -269,27 +297,19 @@ func (o *Orchestrator) runAccount(ctx context.Context, task store.Task, runID in
return o.accountFailed(ctx, task.ID, a, srcEP, dstEP, "src", err) return o.accountFailed(ctx, task.ID, a, srcEP, dstEP, "src", err)
} }
// Planning pass: EXAMINE every folder up front to learn the total message // Planning pass: decide folders from the account's own config, then EXAMINE
// count, so the UI can show an accurate overall bar / ETA before copying. // each to learn message counts for the overall progress bar.
type folderPlan struct { plan := planFolders(folders, a.FolderMapping, a.ExcludedFolders)
src, dst string
total int64
}
plan := make([]folderPlan, 0, len(folders))
var grandTotal int64 var grandTotal int64
for _, folder := range folders { for i := range plan {
if actx.Err() != nil { if actx.Err() != nil {
break break
} }
df := folder n, cerr := imapx.FolderMessageCount(src, plan[i].src)
if m, ok := task.FolderMapping[folder]; ok {
df = m
}
n, cerr := imapx.FolderMessageCount(src, folder)
if cerr != nil && actx.Err() == nil { if cerr != nil && actx.Err() == nil {
slog.Warn("count folder failed", "account", a.ID, "folder", folder, "err", cerr) slog.Warn("count folder failed", "account", a.ID, "folder", plan[i].src, "err", cerr)
} }
plan = append(plan, folderPlan{src: folder, dst: df, total: n}) plan[i].total = n
grandTotal += n grandTotal += n
} }
o.hub.Publish(wshub.Event{Type: "plan", TaskID: task.ID, Data: map[string]any{ o.hub.Publish(wshub.Event{Type: "plan", TaskID: task.ID, Data: map[string]any{
+40
View File
@@ -0,0 +1,40 @@
package orchestrator
import "testing"
func names(ps []folderPlan) []string {
out := make([]string, len(ps))
for i, p := range ps {
out[i] = p.src + "->" + p.dst
}
return out
}
func TestPlanFolders(t *testing.T) {
folders := []string{"INBOX", "Спам", "Trash", "Sent"}
// Empty config: every folder syncs to its own name, in order.
got := planFolders(folders, nil, nil)
if len(got) != 4 || got[0].src != "INBOX" || got[0].dst != "INBOX" {
t.Fatalf("empty config: %v", names(got))
}
// Rename + exclusion.
got = planFolders(folders,
map[string]string{"Спам": "Spam"},
[]string{"Trash"})
if len(got) != 3 {
t.Fatalf("want 3 plans (Trash excluded), got %v", names(got))
}
for _, p := range got {
if p.src == "Trash" {
t.Fatalf("excluded folder present: %v", names(got))
}
if p.src == "Спам" && p.dst != "Spam" {
t.Fatalf("rename not applied: %v", names(got))
}
if p.src == "INBOX" && p.dst != "INBOX" {
t.Fatalf("unmapped folder should keep its name: %v", names(got))
}
}
}
+35 -15
View File
@@ -6,19 +6,21 @@ import (
) )
type Account struct { type Account struct {
ID int64 ID int64
TaskID int64 TaskID int64
SrcLogin string SrcLogin string
SrcPassEnc string SrcPassEnc string
DstLogin string DstLogin string
DstPassEnc string DstPassEnc string
TestSrcStatus string TestSrcStatus string
TestDstStatus string TestDstStatus string
Status string Status string
Copied int64 Copied int64
Skipped int64 Skipped int64
Errors int64 Errors int64
LastError string LastError string
FolderMapping map[string]string
ExcludedFolders []string
} }
func (s *Store) CreateAccount(ctx context.Context, a Account) (int64, error) { func (s *Store) CreateAccount(ctx context.Context, a Account) (int64, error) {
@@ -39,7 +41,8 @@ func (s *Store) DeleteAccount(ctx context.Context, id int64) error {
func (s *Store) ListAccountsByTask(ctx context.Context, taskID int64) ([]Account, error) { func (s *Store) ListAccountsByTask(ctx context.Context, taskID int64) ([]Account, error) {
rows, err := s.Pool.Query(ctx, rows, err := s.Pool.Query(ctx,
`SELECT id, task_id, src_login, src_pass_enc, dst_login, dst_pass_enc, `SELECT id, task_id, src_login, src_pass_enc, dst_login, dst_pass_enc,
test_src_status, test_dst_status, status, copied_count, skipped_count, error_count, last_error test_src_status, test_dst_status, status, copied_count, skipped_count,
error_count, last_error, folder_mapping, excluded_folders
FROM accounts WHERE task_id=$1 ORDER BY id`, taskID) FROM accounts WHERE task_id=$1 ORDER BY id`, taskID)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -49,7 +52,8 @@ func (s *Store) ListAccountsByTask(ctx context.Context, taskID int64) ([]Account
for rows.Next() { for rows.Next() {
var a Account var a Account
if err := rows.Scan(&a.ID, &a.TaskID, &a.SrcLogin, &a.SrcPassEnc, &a.DstLogin, &a.DstPassEnc, if err := rows.Scan(&a.ID, &a.TaskID, &a.SrcLogin, &a.SrcPassEnc, &a.DstLogin, &a.DstPassEnc,
&a.TestSrcStatus, &a.TestDstStatus, &a.Status, &a.Copied, &a.Skipped, &a.Errors, &a.LastError); err != nil { &a.TestSrcStatus, &a.TestDstStatus, &a.Status, &a.Copied, &a.Skipped, &a.Errors, &a.LastError,
&a.FolderMapping, &a.ExcludedFolders); err != nil {
return nil, err return nil, err
} }
out = append(out, a) out = append(out, a)
@@ -95,3 +99,19 @@ func (s *Store) IncAccountCounters(ctx context.Context, id, copied, skipped, err
id, copied, skipped, errs) id, copied, skipped, errs)
return err return err
} }
// SetAccountFolderMapping persists an account's per-folder rename map and the
// set of source folders to skip. nil is normalized to empty so JSONB stays
// '{}' / '[]' rather than null.
func (s *Store) SetAccountFolderMapping(ctx context.Context, id int64, mapping map[string]string, excluded []string) error {
if mapping == nil {
mapping = map[string]string{}
}
if excluded == nil {
excluded = []string{}
}
_, err := s.Pool.Exec(ctx,
`UPDATE accounts SET folder_mapping=$2, excluded_folders=$3 WHERE id=$1`,
id, mapping, excluded)
return err
}
+28
View File
@@ -64,3 +64,31 @@ func TestResetAccountCounters(t *testing.T) {
a.Copied, a.Skipped, a.Errors) a.Copied, a.Skipped, a.Errors)
} }
} }
func TestSetAccountFolderMapping(t *testing.T) {
s := testStore(t)
ctx := context.Background()
epSrc, _ := s.CreateEndpoint(ctx, Endpoint{RoleLabel: "src", Host: "a", Port: 993, TLSMode: "ssl"})
epDst, _ := s.CreateEndpoint(ctx, Endpoint{RoleLabel: "dst", Host: "b", Port: 993, TLSMode: "ssl"})
taskID, _ := s.CreateTask(ctx, Task{Name: "t", SrcEndpointID: epSrc, DstEndpointID: epDst})
accID, _ := s.CreateAccount(ctx, Account{TaskID: taskID, SrcLogin: "u", SrcPassEnc: "x", DstLogin: "u2", DstPassEnc: "y"})
// Fresh account defaults: empty map, empty exclusions (not nil after scan).
accs, _ := s.ListAccountsByTask(ctx, taskID)
if len(accs) != 1 || len(accs[0].FolderMapping) != 0 || len(accs[0].ExcludedFolders) != 0 {
t.Fatalf("defaults: map=%v excl=%v", accs[0].FolderMapping, accs[0].ExcludedFolders)
}
if err := s.SetAccountFolderMapping(ctx, accID,
map[string]string{"Спам": "Spam"}, []string{"Trash"}); err != nil {
t.Fatalf("set: %v", err)
}
accs, _ = s.ListAccountsByTask(ctx, taskID)
a := accs[0]
if a.FolderMapping["Спам"] != "Spam" {
t.Fatalf("mapping not persisted: %v", a.FolderMapping)
}
if len(a.ExcludedFolders) != 1 || a.ExcludedFolders[0] != "Trash" {
t.Fatalf("excluded not persisted: %v", a.ExcludedFolders)
}
}
@@ -0,0 +1,2 @@
ALTER TABLE accounts DROP COLUMN excluded_folders;
ALTER TABLE accounts DROP COLUMN folder_mapping;
@@ -0,0 +1,6 @@
ALTER TABLE accounts ADD COLUMN folder_mapping JSONB NOT NULL DEFAULT '{}';
ALTER TABLE accounts ADD COLUMN excluded_folders JSONB NOT NULL DEFAULT '[]';
-- Carry the existing task-wide mapping onto its accounts so behavior is preserved.
UPDATE accounts a SET folder_mapping = t.folder_mapping
FROM tasks t WHERE a.task_id = t.id AND t.folder_mapping <> '{}'::jsonb;
+15 -2
View File
@@ -33,6 +33,8 @@ export interface Account {
skipped: number skipped: number
errors: number errors: number
last_error?: string last_error?: string
folder_mapping?: Record<string, string>
excluded_folders?: string[]
} }
export interface TaskDetail { export interface TaskDetail {
@@ -101,8 +103,19 @@ export const probeFolders = (
creds: { src_login: string; src_pass: string; dst_login: string; dst_pass: string }, creds: { src_login: string; src_pass: string; dst_login: string; dst_pass: string },
) => api<ProbeResult>(`/api/tasks/${taskId}/probe`, jsonBody(creds)) ) => api<ProbeResult>(`/api/tasks/${taskId}/probe`, jsonBody(creds))
export const setFolderMapping = (taskId: number, mapping: Record<string, string>) => export const probeAccountFolders = (taskId: number, accId: number) =>
api(`/api/tasks/${taskId}/folder-mapping`, { ...jsonBody({ mapping }), method: 'PUT' }) api<ProbeResult>(`/api/tasks/${taskId}/accounts/${accId}/probe`, { method: 'POST' })
export const setAccountFolderMapping = (
taskId: number,
accId: number,
mapping: Record<string, string>,
excluded: string[],
) =>
api(`/api/tasks/${taskId}/accounts/${accId}/folder-mapping`, {
...jsonBody({ mapping, excluded }),
method: 'PUT',
})
export const listTasks = () => api<Task[]>('/api/tasks') export const listTasks = () => api<Task[]>('/api/tasks')
+44 -1
View File
@@ -368,11 +368,47 @@
.map-row { .map-row {
display: grid; display: grid;
grid-template-columns: 1fr auto 1fr auto; grid-template-columns: auto 1fr auto 1fr auto;
align-items: center; align-items: center;
gap: 10px; gap: 10px;
} }
.map-row-off .map-src,
.map-row-off .map-arrow {
color: var(--fg-faint);
text-decoration: line-through;
}
.map-check {
display: flex;
align-items: center;
}
.map-check input,
.map-all input {
accent-color: var(--accent);
cursor: pointer;
}
.map-all {
display: flex;
align-items: center;
gap: 8px;
margin-bottom: 12px;
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--fg-dim);
cursor: pointer;
}
.map-excluded {
font-size: 10px;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--fg-faint);
}
.map-src { .map-src {
font-family: var(--font-mono); font-family: var(--font-mono);
font-size: 13px; font-size: 13px;
@@ -623,6 +659,13 @@ table.tbl a.rowlink:hover {
text-align: right; text-align: right;
} }
.row-actions {
display: inline-flex;
gap: 12px;
align-items: center;
justify-content: flex-end;
}
.empty-row td { .empty-row td {
text-align: center; text-align: center;
color: var(--fg-faint); color: var(--fg-faint);
+44 -6
View File
@@ -6,7 +6,8 @@ type Props = {
srcFolders: string[] srcFolders: string[]
dstFolders: string[] dstFolders: string[]
initialMapping: Record<string, string> initialMapping: Record<string, string>
onConfirm: (mapping: Record<string, string>) => void initialExcluded: string[]
onConfirm: (mapping: Record<string, string>, excluded: string[]) => void
onCancel: () => void onCancel: () => void
} }
@@ -18,8 +19,14 @@ function defaultDst(src: string, dstFolders: string[], initial: Record<string, s
return src return src
} }
export function FolderMappingModal({ open, srcFolders, dstFolders, initialMapping, onConfirm, onCancel }: Props) { export function FolderMappingModal({
open, srcFolders, dstFolders, initialMapping, initialExcluded, onConfirm, onCancel,
}: Props) {
const [choice, setChoice] = useState<Record<string, string>>({}) const [choice, setChoice] = useState<Record<string, string>>({})
const [synced, setSynced] = useState<Record<string, boolean>>(() => {
const excl = new Set(initialExcluded)
return Object.fromEntries(srcFolders.map((f) => [f, !excl.has(f)]))
})
// Options per select: all destination folders, plus the source name itself // Options per select: all destination folders, plus the source name itself
// (marked "create") when it does not already exist on the destination. // (marked "create") when it does not already exist on the destination.
@@ -35,13 +42,19 @@ export function FolderMappingModal({ open, srcFolders, dstFolders, initialMappin
const valueFor = (src: string) => choice[src] ?? defaultDst(src, dstFolders, initialMapping) const valueFor = (src: string) => choice[src] ?? defaultDst(src, dstFolders, initialMapping)
function confirm() { function confirm() {
const mapping: Record<string, string> = { ...initialMapping } const mapping: Record<string, string> = {}
const excluded: string[] = []
for (const src of srcFolders) { for (const src of srcFolders) {
if (synced[src] === false) {
excluded.push(src)
delete mapping[src]
continue
}
const dst = valueFor(src) const dst = valueFor(src)
if (dst === src) delete mapping[src] if (dst === src) delete mapping[src]
else mapping[src] = dst else mapping[src] = dst
} }
onConfirm(mapping) onConfirm(mapping, excluded)
} }
return ( return (
@@ -51,12 +64,32 @@ export function FolderMappingModal({ open, srcFolders, dstFolders, initialMappin
Route each source folder to an existing destination folder. Leaving a folder mapped to its own name Route each source folder to an existing destination folder. Leaving a folder mapped to its own name
creates it on the destination if missing (e.g. map <code>Спам</code> <code>Spam</code> to avoid duplicates). creates it on the destination if missing (e.g. map <code>Спам</code> <code>Spam</code> to avoid duplicates).
</p> </p>
<label className="map-all">
<input
type="checkbox"
checked={srcFolders.every((f) => synced[f] !== false)}
onChange={(e) => {
const on = e.target.checked
setSynced(Object.fromEntries(srcFolders.map((f) => [f, on])))
}}
/>
sync all folders
</label>
<div className="map-grid"> <div className="map-grid">
{srcFolders.map((src) => { {srcFolders.map((src) => {
const on = synced[src] !== false
const val = valueFor(src) const val = valueFor(src)
const creates = !dstFolders.includes(val) const creates = !dstFolders.includes(val)
return ( return (
<div className="map-row" key={src}> <div className={`map-row${on ? '' : ' map-row-off'}`} key={src}>
<label className="map-check">
<input
type="checkbox"
checked={on}
aria-label={`Sync folder ${src}`}
onChange={(e) => setSynced((s) => ({ ...s, [src]: e.target.checked }))}
/>
</label>
<span className="map-src" title={src}> <span className="map-src" title={src}>
{src} {src}
</span> </span>
@@ -65,6 +98,7 @@ export function FolderMappingModal({ open, srcFolders, dstFolders, initialMappin
className="map-select" className="map-select"
aria-label={`Destination folder for ${src}`} aria-label={`Destination folder for ${src}`}
value={val} value={val}
disabled={!on}
onChange={(e) => setChoice((c) => ({ ...c, [src]: e.target.value }))} onChange={(e) => setChoice((c) => ({ ...c, [src]: e.target.value }))}
> >
{options(src).map((f) => ( {options(src).map((f) => (
@@ -74,7 +108,11 @@ export function FolderMappingModal({ open, srcFolders, dstFolders, initialMappin
</option> </option>
))} ))}
</select> </select>
{creates && <span className="map-new">new</span>} {on ? (
creates && <span className="map-new">new</span>
) : (
<span className="map-excluded">excluded</span>
)}
</div> </div>
) )
})} })}
+91 -18
View File
@@ -1,5 +1,5 @@
import { useEffect, useRef, useState, type ChangeEvent, type FormEvent } from 'react' import { useEffect, useRef, useState, type ChangeEvent, type FormEvent } from 'react'
import { cancelAccount, createAccount, deleteAccount, getTask, importCSV, probeFolders, runTask, setFolderMapping, testAccounts, type TaskDetail as TaskDetailData } from '../api' import { cancelAccount, createAccount, deleteAccount, getTask, importCSV, probeAccountFolders, probeFolders, runTask, setAccountFolderMapping, testAccounts, type TaskDetail as TaskDetailData } from '../api'
import { connectTaskWS, type TaskEvent } from '../ws' import { connectTaskWS, type TaskEvent } from '../ws'
import { StatusBadge } from '../components/StatusBadge' import { StatusBadge } from '../components/StatusBadge'
import { useConfirm } from '../components/ConfirmProvider' import { useConfirm } from '../components/ConfirmProvider'
@@ -75,6 +75,13 @@ export function TaskDetail({ id }: { id: number }) {
const [form, setForm] = useState(emptyAccount) const [form, setForm] = useState(emptyAccount)
const [busy, setBusy] = useState<'test' | 'run' | 'add' | 'import' | 'delete' | 'probe' | null>(null) const [busy, setBusy] = useState<'test' | 'run' | 'add' | 'import' | 'delete' | 'probe' | null>(null)
const [mapState, setMapState] = useState<{ src: string[]; dst: string[]; creds: typeof emptyAccount } | null>(null) const [mapState, setMapState] = useState<{ src: string[]; dst: string[]; creds: typeof emptyAccount } | null>(null)
const [editMap, setEditMap] = useState<{
accId: number
src: string[]
dst: string[]
mapping: Record<string, string>
excluded: string[]
} | null>(null)
const confirm = useConfirm() const confirm = useConfirm()
const [error, setError] = useState<string | null>(null) const [error, setError] = useState<string | null>(null)
const [live, setLive] = useState<Record<number, LiveProgress>>({}) const [live, setLive] = useState<Record<number, LiveProgress>>({})
@@ -186,13 +193,13 @@ export function TaskDetail({ id }: { id: number }) {
} }
} }
async function confirmMapping(mapping: Record<string, string>) { async function confirmMapping(mapping: Record<string, string>, excluded: string[]) {
if (!mapState) return if (!mapState) return
setBusy('add') setBusy('add')
setError(null) setError(null)
try { try {
await createAccount(id, mapState.creds) const { id: accId } = await createAccount(id, mapState.creds)
await setFolderMapping(id, mapping) await setAccountFolderMapping(id, accId, mapping, excluded)
setForm(emptyAccount) setForm(emptyAccount)
setMapState(null) setMapState(null)
reload() reload()
@@ -203,6 +210,47 @@ export function TaskDetail({ id }: { id: number }) {
} }
} }
async function onEditFolders(a: { id: number; src_login: string; folder_mapping?: Record<string, string>; excluded_folders?: string[] }) {
setBusy('probe')
setError(null)
try {
const res = await probeAccountFolders(id, a.id)
if (!res.src.ok || !res.dst.ok) {
const parts: string[] = []
if (!res.src.ok) parts.push(`source: ${res.src.error ?? 'login failed'}`)
if (!res.dst.ok) parts.push(`destination: ${res.dst.error ?? 'login failed'}`)
setError(`Folder probe failed — ${parts.join('; ')}`)
return
}
setEditMap({
accId: a.id,
src: res.src.folders ?? [],
dst: res.dst.folders ?? [],
mapping: a.folder_mapping ?? {},
excluded: a.excluded_folders ?? [],
})
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to probe folders')
} finally {
setBusy(null)
}
}
async function saveEditMapping(mapping: Record<string, string>, excluded: string[]) {
if (!editMap) return
setBusy('add')
setError(null)
try {
await setAccountFolderMapping(id, editMap.accId, mapping, excluded)
setEditMap(null)
reload()
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to save mapping')
} finally {
setBusy(null)
}
}
function downloadExampleCSV() { function downloadExampleCSV() {
const sample = [ const sample = [
'alice@source.example,SrcPass1,alice@dest.example,DstPass1', 'alice@source.example,SrcPass1,alice@dest.example,DstPass1',
@@ -528,20 +576,32 @@ export function TaskDetail({ id }: { id: number }) {
<td className="num-cell">{live[a.id]?.skipped ?? a.skipped}</td> <td className="num-cell">{live[a.id]?.skipped ?? a.skipped}</td>
<td className="num-cell">{a.errors}</td> <td className="num-cell">{a.errors}</td>
<td className="num-cell"> <td className="num-cell">
{a.status === 'running' ? ( <div className="row-actions">
<button type="button" className="link-btn danger" onClick={() => onCancelAccount(a.id)}> {a.status !== 'running' && data?.task.status !== 'running' && (
cancel <button
</button> type="button"
) : ( className="link-btn"
<button onClick={() => onEditFolders(a)}
type="button" disabled={busy !== null}
className="link-btn danger" >
onClick={() => onDeleteAccount(a.id, a.src_login)} folders
disabled={busy !== null || data?.task.status === 'running'} </button>
> )}
remove {a.status === 'running' ? (
</button> <button type="button" className="link-btn danger" onClick={() => onCancelAccount(a.id)}>
)} cancel
</button>
) : (
<button
type="button"
className="link-btn danger"
onClick={() => onDeleteAccount(a.id, a.src_login)}
disabled={busy !== null || data?.task.status === 'running'}
>
remove
</button>
)}
</div>
</td> </td>
</tr> </tr>
)) ))
@@ -558,10 +618,23 @@ export function TaskDetail({ id }: { id: number }) {
srcFolders={mapState.src} srcFolders={mapState.src}
dstFolders={mapState.dst} dstFolders={mapState.dst}
initialMapping={task.folder_mapping ?? {}} initialMapping={task.folder_mapping ?? {}}
initialExcluded={[]}
onCancel={() => setMapState(null)} onCancel={() => setMapState(null)}
onConfirm={confirmMapping} onConfirm={confirmMapping}
/> />
)} )}
{editMap && (
<FolderMappingModal
key={`edit-${editMap.accId}`}
open
srcFolders={editMap.src}
dstFolders={editMap.dst}
initialMapping={editMap.mapping}
initialExcluded={editMap.excluded}
onCancel={() => setEditMap(null)}
onConfirm={saveEditMapping}
/>
)}
</> </>
) )
} }