docs: spec + plan for per-account folder mapping & exclusion
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,160 @@
|
||||
# Per-account folder mapping & folder exclusion — design
|
||||
|
||||
**Date:** 2026-07-03
|
||||
**Status:** awaiting user review
|
||||
|
||||
## Context
|
||||
|
||||
Two gaps in the current folder-mapping feature:
|
||||
|
||||
1. **No per-account mapping for bulk (CSV) accounts.** Folder mapping is collected
|
||||
only in the single "Add account" flow. Accounts imported from CSV inherit the
|
||||
task-wide mapping and cannot be mapped individually. The user wants a **folders**
|
||||
button per account row (left of remove) that re-probes that account's live
|
||||
folders and opens the mapping modal for it.
|
||||
2. **No exclusion.** Mapping can only rename `src → dst`; it cannot skip folders
|
||||
the user doesn't want to copy (Spam, Trash, per-client junk). The user wants
|
||||
**checkboxes** to choose which source folders are synced.
|
||||
|
||||
### Current architecture (as-is)
|
||||
|
||||
- `tasks.folder_mapping` (JSONB `map[src]dst`) — **one mapping for the whole task**,
|
||||
applied to every account by the orchestrator: `df := task.FolderMapping[folder]`
|
||||
(`internal/orchestrator/orchestrator.go:283-286`).
|
||||
- Single-add flow: `probeFolders` (form creds) → `FolderMappingModal` → `createAccount`
|
||||
then `setFolderMapping` (task-level) (`TaskDetail.tsx:167-204`).
|
||||
- `handleProbeFolders` takes **plaintext** creds from the request body
|
||||
(`internal/httpapi/accounts.go:39-83`). CSV accounts store only encrypted
|
||||
passwords, so re-probing an existing account needs a server-side decrypt path.
|
||||
- `store.Account` has **no** mapping field.
|
||||
|
||||
## Decision: fully per-account model
|
||||
|
||||
Folder configuration moves from the task to the account. `tasks.folder_mapping`
|
||||
is retired as the source of truth (column kept, no longer read) and its data is
|
||||
migrated onto each account. This matches the intent ("each account its own
|
||||
folders") and removes the current oddity where adding one account rewrites the
|
||||
whole task's mapping.
|
||||
|
||||
Exclusion is stored as a **separate `excluded_folders` list** (not a sentinel in
|
||||
the mapping). Rationale: a source folder that is *not configured* (new folder
|
||||
appearing after mapping, or a CSV account never opened) must **default to synced**
|
||||
— a copy tool must never silently drop mail. An exclusion list gives that safe
|
||||
default; an inclusion list would silently skip new folders.
|
||||
|
||||
### Semantics
|
||||
|
||||
For an account, given its live source folders:
|
||||
|
||||
- `excluded_folders: string[]` — source folder names to **skip** entirely.
|
||||
- `folder_mapping: map[src]dst` — rename for synced folders; a src absent from the
|
||||
map syncs to its own name (existing behavior).
|
||||
- Default (empty mapping, empty exclusions) = sync every folder name-matched =
|
||||
current behavior. Backward compatible.
|
||||
|
||||
## Data model
|
||||
|
||||
**Migration `0003_account_folder_mapping`:**
|
||||
|
||||
```sql
|
||||
-- up
|
||||
ALTER TABLE accounts ADD COLUMN folder_mapping JSONB NOT NULL DEFAULT '{}';
|
||||
ALTER TABLE accounts ADD COLUMN excluded_folders JSONB NOT NULL DEFAULT '[]';
|
||||
-- carry the existing task-wide mapping onto its accounts
|
||||
UPDATE accounts a SET folder_mapping = t.folder_mapping
|
||||
FROM tasks t WHERE a.task_id = t.id AND t.folder_mapping <> '{}';
|
||||
-- down
|
||||
ALTER TABLE accounts DROP COLUMN excluded_folders;
|
||||
ALTER TABLE accounts DROP COLUMN folder_mapping;
|
||||
```
|
||||
|
||||
**`store.Account`** gains:
|
||||
- `FolderMapping map[string]string`
|
||||
- `ExcludedFolders []string`
|
||||
|
||||
`ListAccountsByTask` selects/scans both (JSONB). New helper:
|
||||
- `SetAccountFolderMapping(ctx, id int64, mapping map[string]string, excluded []string) error`
|
||||
|
||||
`tasks.folder_mapping` column and `SetTaskFolderMapping` remain but are no longer
|
||||
read by the orchestrator (kept for the migration source and to avoid a destructive
|
||||
change; single-add stops writing it).
|
||||
|
||||
## Orchestrator
|
||||
|
||||
Extract the per-folder plan decision into a small **pure, testable** helper so the
|
||||
exclusion/mapping logic can be unit-tested without IMAP:
|
||||
|
||||
```go
|
||||
// planFolders returns the (src,dst) pairs to copy for an account, dropping
|
||||
// excluded source folders and applying renames.
|
||||
func planFolders(folders []string, mapping map[string]string, excluded []string) []folderPlan
|
||||
```
|
||||
|
||||
`runAccount` replaces the inline `task.FolderMapping[folder]` loop with
|
||||
`planFolders(folders, a.FolderMapping, a.ExcludedFolders)`, then counts messages
|
||||
per surviving folder as today. Excluded folders never enter the plan (not counted,
|
||||
not scanned, not copied).
|
||||
|
||||
## HTTP API
|
||||
|
||||
- **`POST /api/tasks/{id}/accounts/{accId}/probe`** — new. Loads the account,
|
||||
decrypts its stored src/dst passwords, reuses the existing `imapx.TestLogin`
|
||||
probe, returns `{src:{ok,folders,error}, dst:{...}}`. No creds in the request.
|
||||
- **`PUT /api/tasks/{id}/accounts/{accId}/folder-mapping`** — new. Body
|
||||
`{mapping: map[string]string, excluded: string[]}` → `SetAccountFolderMapping`.
|
||||
- **`AccountView` / `accountDTO`** expose `folder_mapping` and `excluded_folders`
|
||||
so the modal can seed saved state when reopened.
|
||||
- Existing task-level `POST /probe` stays (single-add probes before the account
|
||||
exists). Task-level `PUT /folder-mapping` handler stays but is unused by the UI.
|
||||
|
||||
## Frontend
|
||||
|
||||
**`api.ts`**
|
||||
- `Account` interface gains `folder_mapping?: Record<string,string>` and
|
||||
`excluded_folders?: string[]`.
|
||||
- `probeAccountFolders(taskId, accId)` → `ProbeResult`.
|
||||
- `setAccountFolderMapping(taskId, accId, mapping, excluded)`.
|
||||
|
||||
**`FolderMappingModal`** — add exclusion:
|
||||
- New props `initialExcluded: string[]`, and `onConfirm(mapping, excluded)`.
|
||||
- Each row gets a leading **checkbox** (checked = sync). Unchecking greys/disables
|
||||
that row's dst `<select>` and the "new/create" marker, and adds the src to the
|
||||
excluded set.
|
||||
- A header "sync all" checkbox toggles every row.
|
||||
- `confirm()` emits `mapping` (only for synced rows) and `excluded` (unchecked rows).
|
||||
|
||||
**`TaskDetail`**
|
||||
- Actions cell: add a `folders` link-btn **left of** remove/cancel, shown for every
|
||||
account that is not currently running. Click → `probeAccountFolders(accId)` →
|
||||
open the modal in **edit-account** mode seeded with the account's saved
|
||||
`folder_mapping` + `excluded_folders` and the freshly probed folders.
|
||||
- New state `editMap: { accId; src: string[]; dst: string[]; mapping; excluded } | null`,
|
||||
mounted like the existing add-modal with a `key` per account for clean remount.
|
||||
- On confirm in edit mode → `setAccountFolderMapping(accId, mapping, excluded)` → reload.
|
||||
- Single-add `confirmMapping` switches from task-level `setFolderMapping` to
|
||||
`setAccountFolderMapping(newAccId, mapping, excluded)` after `createAccount`.
|
||||
- Probe button shows a busy state (`busy: 'probe'`) while re-probing an account.
|
||||
|
||||
## Error handling
|
||||
|
||||
- Probe-by-account failures (login fails, IMAP down) surface via the existing
|
||||
`error` banner (now `role="alert"`), same as single-add probe failures.
|
||||
- Saving mapping while a run is in progress is blocked: the `folders` button is
|
||||
hidden/disabled for `running` accounts (mapping is read at run start).
|
||||
|
||||
## Testing
|
||||
|
||||
- **store**: `SetAccountFolderMapping` round-trips mapping+excluded via
|
||||
`ListAccountsByTask`; migration 0003 copies `tasks.folder_mapping` onto accounts.
|
||||
- **orchestrator**: unit-test the extracted `planFolders` — excluded folders
|
||||
dropped, renames applied, absent-src keeps its name, empty config = all folders.
|
||||
- **httpapi**: probe-by-account decrypts and returns folders; PUT persists.
|
||||
- **frontend / e2e**: `/run` the app — CSV-import an account, click **folders**,
|
||||
uncheck a folder + rename another, save, run, verify the excluded folder is not
|
||||
copied and the rename lands. Reduced to a manual E2E checklist per project rules.
|
||||
|
||||
## Out of scope (YAGNI)
|
||||
|
||||
- Task-wide "apply to all accounts" bulk mapping editor.
|
||||
- Regex/glob folder matching.
|
||||
- Removing the `tasks.folder_mapping` column (kept to avoid a destructive migration).
|
||||
Reference in New Issue
Block a user