---
type: workflow
title: AI auto-reply workflow
description: End-to-end trace of the AI autoreply — enabling it per chat (F3 on the selected or open chat) or globally (F4 / web panel) for private chats and groups, the ReplyGate rules, the reply delay that batches messages into one reply, the drop when the owner answered by hand, voice-note transcriptions and image descriptions processed once and reused, the startup catch-up, building the AI context from SOUL.md, the no_reply tool, retries and sending the reply.
tags: [ai, autoreply, workflow, vision, transcription, reply-gate, reply-delay, catch-up, image-description]
sources:
  - id: openwiki-source-9fec9d1af408501c9c455161
    resource: repo://src/app.rs
  - id: openwiki-source-043e6252badc57a1ee6bfb48
    resource: repo://src/autoreply.rs
  - id: openwiki-source-d6b2055bbb49804b8a855fd2
    resource: repo://src/config_watch.rs
  - id: openwiki-source-2a737474d86fc75cc9d9694f
    resource: repo://src/config.rs
  - id: openwiki-source-ce0c4ea8cd3b15431fa691ce
    resource: repo://src/history.rs
generated: { by: "opencode", at: "2026-10-24T16:39:50.206Z" }
---

# AI auto-reply workflow

The autoreply answers incoming WhatsApp messages with an AI-generated text,
only for chats where the user turned it on — one by one, or all at once with
the global switch. Decision rules and context building are pure functions in
`src/autoreply.rs` (no I/O); orchestration lives in
[`App`](../architecture/app-state.md); the HTTP calls are done by the
[AI client](../integrations/ai-client.md) (text and vision) and by the
transcription endpoint (`src/transcribe.rs`).

```
WaEvent::Message { message, image, audio }
  -> History::save_message (new only)
  -> App::maybe_autoreply   (enabled = global OR per-chat flag)
       ReplyGate::should_schedule? --yes--> mark_pending + timer(reply_delay_seconds)
       media from an allowed sender?  ----> keep_image / keep_audio
  ... AppEvent::ReplyDue
  -> prepare_reply: reply_context (still enabled? AI? owner not answered last?)
       --no--> finish_reply (drop)
       audios without transcription? --yes--> download + transcribe (parallel, max 3)
                                               -> AppEvent::AudiosTranscribed -> text stored
       images without description?  --yes--> download + describe_image (parallel, max 5)
                                               -> AppEvent::ImagesDescribed -> description stored
  -> start_reply: messages_since(yesterday 00:00, 200) -> mark @[te] -> build_turns
  -> AiClient::complete (tools: [no_reply], up to ai_retries retries)
  -> AppEvent::AiReply
       NoReply -> status "AI: nessuna risposta necessaria"
       Text    -> still enabled AND owner did not answer meanwhile? -> send_text -> stored is_ai = true
  -> finish_reply: newer message arrived meanwhile? -> schedule one new reply
WaEvent::OfflineSyncCompleted (first time only)
  -> catch_up -> mark_pending + prepare_reply (no delay) per due private chat
```

## 1. Enabling

There are two independent switches, and a chat is **enabled** when either is
on (`autoreply_enabled`, `src/app.rs:858-860`):

- **Per chat** — F3 toggles the flag of the focused chat: the selected row
  of the (filtered) list when the list has focus, the open chat when the
  input has focus (`toggle_autoreply` / `toggle_chat_autoreply`,
  `src/app.rs:1348-1378`), so a chat can be enabled without opening it; the
  [web control panel](../integrations/web-control-panel.md) can toggle any
  listed chat the same way. Turning it off is always allowed; turning it on
  requires a configured AI (API key in `config.toml` and a non-blank
  `SOUL.md`, see [Configuration](../operations/configuration-and-data.md))
  and a chat that supports autoreply (`can_enable_autoreply`,
  `src/app.rs:1894-1902`), otherwise the status shows
  `Autoreply disponibile solo nelle chat private e nei gruppi`. The flag is
  persisted in the `chats` table of the
  [history store](../concepts/history-store.md) (default off).
- **Global ("AI tutte")** — F4 or the web panel toggles
  `global_autoreply` (`toggle_global_autoreply`, `src/app.rs:1433-1444`);
  enabling requires an AI client. It is persisted in the `settings` table,
  loaded at startup and does not change any per-chat flag, so turning it off
  restores the per-chat selection. The TUI marks it in the list title and
  footer ([Terminal UI](../architecture/terminal-ui.md)).

The global switch does not bypass the chat-type rule: `meets_content_rules`
still requires a supported chat, so broadcasts, status and newsletters are
never answered.

The AI setup itself follows the configuration files live: when
`config.toml`, `SOUL.md` or `VISION.md` change, the configuration is
reloaded and the AI client rebuilt within about 2 seconds
(`apply_reloaded_config`, `src/app.rs:422-434`, see
[Configuration](../operations/configuration-and-data.md#live-reload)).
Replies already running keep the previous client; later schedules, contexts
and requests use the new prompt, models, `reply_delay_seconds`,
`catch_up_seconds`, `ai_retries` and `transcribe_*` keys. If the new files
leave the AI unconfigured, every autoreply path stops as described below,
but the persisted flags are kept.

`supports_autoreply` (`src/autoreply.rs:105-108`) accepts private chats and
groups:

- private: JID server `s.whatsapp.net` or `lid` (`is_private_chat`,
  `src/autoreply.rs:83-92`);
- group: JID server `g.us` (`is_group_chat`, `src/autoreply.rs:94-103`);
- broadcasts (including `status@broadcast`), newsletters and malformed JIDs
  are excluded.

## 2. Trigger and ReplyGate rules

Only messages that `History::save_message` reports as new reach
`maybe_autoreply` (`on_message`, `src/app.rs:485-499`; `maybe_autoreply`,
`src/app.rs:509-532`), so duplicates and history-sync re-deliveries of
stored messages never trigger. `maybe_autoreply` needs an AI client and a
system prompt (`ai_setup`, `src/app.rs:987-989`), refreshes the own JIDs if
unknown and computes `enabled` (global or per chat).

`ReplyGate::should_schedule(message, enabled)`
(`src/autoreply.rs:189-191`) returns `true` only when **all** hold
(`is_reply_candidate` and `meets_content_rules`,
`src/autoreply.rs:247-265`):

| Rule | Detail |
|---|---|
| Not pending | the chat has no reply already scheduled or in progress |
| Enabled | global autoreply on, or the per-chat flag on |
| Sender allowed | not sent by the user — except human-written messages in the self-chat ("Messaggi a te stesso"); AI messages never trigger (`sender_allowed`, `src/autoreply.rs:178-180`) |
| Supported chat | private chat or group (`supports_autoreply`) |
| No backlog | `timestamp >= started_at`, the process start time (`is_backlog`, `src/autoreply.rs:198-200`) |
| Has content | text not blank and not `[media]`/`[messaggio non supportato]`; voice notes and images do trigger |

The self-chat is detected by comparing the chat JID with the account's own
PN/LID JIDs, ignoring a `:N` device suffix (`is_self_chat`,
`src/autoreply.rs:154-157`; `strip_device_suffix`,
`src/autoreply.rs:299-309`). Groups follow exactly the same rules as private
chats; mentions and quotes no longer change anything.

## 3. Reply delay and batching

When `should_schedule` passes, `schedule_reply` (`src/app.rs:539-542`) marks
the chat pending in the gate (`mark_pending`, `src/autoreply.rs:208-210`)
and `spawn_reply_timer` (`src/app.rs:1586-1591`) emits
`AppEvent::ReplyDue { chat_jid }` after `reply_delay_seconds` (default 30;
`0` answers at once). The chat stays pending until `finish_reply`, so every
message arriving during the wait — and while the AI is generating — gets no
timer of its own: the reply is built from the context loaded when it starts,
so it covers everything received during the wait.

`handle_reply_due` (`src/app.rs:634-641`) takes the images and audios kept
for the chat and, if the chat is still pending (a local deletion removes the
pending mark), calls `prepare_reply` (`src/app.rs:656-679`). Its
`reply_context` (`src/app.rs:868-880`) drops the reply — and `finish_reply`
ends the pending state — when:

- the autoreply was turned off (both switches) during the wait;
- the AI is not configured;
- the chat has no message in the context window;
- the latest message was written by hand by the owner outside the self-chat
  (`answered_by_owner`, `src/autoreply.rs:237-239`): the owner already
  answered, so the AI stays silent.

Otherwise it transcribes the new audios and describes the new images first
(see §5) or calls `start_reply` at once.

### After the reply

`AiReply` is always followed by `finish_reply` (`src/app.rs:837-852`), which
clears the pending state and the `answered_up_to` snapshot. If a reply was
started and the latest stored message is a different one
(`arrived_after`, `src/app.rs:1696-1698`) that itself passes
`should_schedule`, a **new** reply is scheduled (with a fresh delay, keeping
the media received meanwhile); otherwise the kept images and audios are
dropped. So messages that arrive while a reply is being generated are
answered once, afterwards, rather than being lost or triggering parallel
replies.

## 4. Startup catch-up

Messages received while the app was off are delivered on connection and are
older than `started_at`, so the live rules treat them as backlog. When
WhatsApp signals that the offline queue has been delivered
(`WaEvent::OfflineSyncCompleted`), `on_offline_sync_completed`
(`src/app.rs:924-930`) runs `catch_up` once per process (`catch_up_done`;
later reconnections skip it).

`catch_up` / `catch_up_chats` (`src/app.rs:940-981`) look at every enabled
chat — all chats when the global autoreply is on, otherwise those with the
per-chat flag — and take its last stored message.
`ReplyGate::catch_up_due` (`src/autoreply.rs:278-290`) accepts it only when:

- `catch_up_seconds` (default 7200) is positive — `0` disables the catch-up;
- the chat is not a group;
- the message is at most `catch_up_seconds` old and older than the app start
  (newer messages follow the live path);
- it passes `meets_content_rules`, so a last message written by the user
  (already answered) never qualifies.

Each due chat is marked pending and goes through `prepare_reply` **without**
the reply delay, together with the images and audios of its offline messages
(`backlog_images`, `backlog_audios`); offline media of other chats are
dropped. When at least one reply went ahead the status shows
`Risposte di recupero: N`. Offline images that arrive after the catch-up
already ran are described on their own, without a reply
(`describe_late_image`, `src/app.rs:571-581`), and offline audios
transcribed the same way (`transcribe_late_audio`, `src/app.rs:616-626`).

## 5. Media: image descriptions and voice-note transcriptions

Media bytes are never sent to the text model. Each image or audio sent by an
allowed sender in an enabled chat is kept by `keep_image`
(`src/app.rs:550-564`) or `keep_audio` (`src/app.rs:590-609`; audios are
ignored when `transcribe_key` is unset) and, just before the reply that
covers it, processed **once**:

- audios are transcribed first by the configured `audio/transcriptions`
  endpoint (`spawn_transcriptions` → `spawn_audio_transcriptions`,
  `src/app.rs:688-715`, `src/app.rs:1796-1830`), in parallel. The
  transcription is appended to the stored message text as
  `[messaggio vocale] <text>` (`handle_audios_transcribed`,
  `src/app.rs:784-829`, via `History::update_message_text`), so it enters
  every later context; failures stay untranscribed and the status shows
  `Trascrizione vocale fallita (N): <first error>`. When the reply was
  waiting for them, the postponed images are described next, or the reply
  starts;
- images are then described by the vision model (`AiClient::describe_image`,
  request text from `VISION.md` or the built-in one, see
  [AI client](../integrations/ai-client.md)). The vision model sees only
  that request and the image, never the conversation. The description is
  stored in the `image_descriptions` table and reused in every later
  context.

`select_media_to_describe` (`src/app.rs:1655-1676`) keeps only the items
whose message is in the reply context and that are not already processed or
in progress (`describing` / `transcribing`, plus stored descriptions or
transcribed texts, respectively `images_to_describe` and
`audios_to_transcribe`, `src/app.rs:888-918`), and caps them to the most
recent ones: **5 images** (`MAX_IMAGES_PER_REPLY`) or **3 audios**
(`MAX_AUDIOS_PER_REPLY`); older media of the same batch are dropped for
good, to cap the cost of albums and bursts of voice notes. Without
`openai_vision_model` images are not described; without `transcribe_key`
audios are not transcribed (`descriptions_apply`, `src/app.rs:1855-1857`).

Both processing kinds run in parallel with one task per item; each download
(`download_image_with_retry` / `download_audio_with_retry`,
`src/app.rs:1737-1784`) makes up to 3 attempts, each abandoned after 60 s,
waiting 2 s × attempt between them, and the bytes stay in memory.
`handle_images_described` (`src/app.rs:747-770`) stores each description
whose message still exists (`History::message_exists`), so a chat deleted
meanwhile gets no orphan rows; failed images stay without description, the
others are unaffected, and the status shows
`Descrizione immagine fallita (N): <first error>`. If the chat is still
pending, the reply then starts (re-checking `reply_context`).

## 6. Building the context

`start_reply` (`src/app.rs:1009-1028`) loads the context with
`History::messages_since(chat_jid, context_start(now), CONTEXT_MESSAGES)`:

- `context_start` (`src/autoreply.rs:466-468`) is local midnight of
  **yesterday** (falls back to `now` minus two days if local time cannot be
  resolved, `src/autoreply.rs:475-480`);
- `CONTEXT_MESSAGES` = 200 (`src/autoreply.rs:15`) caps the window,
  keeping the most recent messages in ascending order
  ([History store](../concepts/history-store.md)).

It records the last context message in `answered_up_to`, replaces every
`@<own number or LID>` in the texts with `@[te]` (`mark_own_mentions`,
`src/autoreply.rs:165-170`; `replace_mention`,
`src/autoreply.rs:444-456`; a longer number is never matched) and calls
`build_turns(system_prompt, messages, TurnContext)` (`src/autoreply.rs:495-503`)
with the self-chat/group flags, the chat name and the stored image
descriptions:

- first a `System` turn (`system_turn_content`, `src/autoreply.rs:427-436`):
  the `SOUL.md` prompt as written, then — in normal chats and groups — a
  line `Chat privata con «name».` or `Chat di gruppo «name».` and
  `ROLE_NOTE_CHAT` (`assistant` messages were written by the account owner
  it answers for, `user` messages by the contact or, in groups, by the
  participants named in front); in the self-chat only `ROLE_NOTE_SELF_CHAT`
  (`user` is the owner, `assistant` its own previous replies). Parts are
  separated by a blank line;
- then the last `CONTEXT_MESSAGES` messages, skipping blank texts;
- roles (`turn_role`, `src/autoreply.rs:318-321`): in normal chats and
  groups own messages are `Assistant`; in the self-chat, where every message
  is `from_me`, only `is_ai` messages are `Assistant`;
- in groups every `User` turn is prefixed with `"<sender>: "`, where the
  sender is the trimmed push name or else the user part of the sender JID
  (`sender_label`, `message_to_turn`, `src/autoreply.rs:387-418`);
- only `User` turns are rewritten (`user_content`, `src/autoreply.rs:370-379`),
  with plain facts — how to react is left to `SOUL.md`:
  - voice note **without** a transcription →
    `[Il contatto ha inviato un messaggio vocale che non puoi ascoltare]`;
    a transcribed one instead passes through as its stored text
    (`[messaggio vocale] <transcription>`);
  - image with a stored description →
    `[Il contatto ha inviato un'immagine. Descrizione: <text>]`, without one →
    `[Il contatto ha inviato un'immagine che non puoi vedere]`; a caption is
    appended as `\nDidascalia: <caption>`;
  - any other text is kept as is.

## 7. Request, `no_reply` and retries

`spawn_ai_request` (`src/app.rs:1600-1611`) calls `AiClient::complete`, which
offers the model a single parameterless tool, `no_reply`. When the model
calls it, the outcome is `Reply::NoReply`: nothing is sent and the status
shows `AI: nessuna risposta necessaria`. The app never says when to use it —
the cases belong in `SOUL.md`.

Every AI call (text completion and image description) is retried by the
[AI client](../integrations/ai-client.md) up to `ai_retries` times (default
2, waiting 2 s, 4 s, …) on transient errors — HTTP 429, 5xx, timeouts,
transport errors, malformed or empty replies — while other 4xx errors fail
at once; transcription calls perform a single request instead. A reply can
therefore arrive noticeably later than the trigger; the checks in §8 happen
after all retries. The request task is awaited by a second task, so
`AiReply` is always delivered and the chat never stays pending. When
`ai_log = true`, every attempt is also appended to `ai_requests.log`.

## 8. Sending

A text reply is handled by `send_ai_reply_if_enabled`
(`src/app.rs:1064-1077`), which re-checks two things:

- `autoreply_enabled`: if both the global and the per-chat switch are off by
  then, the reply is discarded with status
  `Risposta AI scartata: autoreply disattivato`;
- `owner_answered_meanwhile` (`src/app.rs:1084-1090`): if a hand-written
  owner message follows the last message of the reply context
  (`manual_reply_after`), the reply is discarded with
  `Risposta AI scartata: hai già risposto`.

Otherwise the text is sent with `spawn_send(..., is_ai = true)`; on success
`handle_sent` (`src/app.rs:1096-1120`) stores it as an own message with
`is_ai = true`, which keeps it from re-triggering the self-chat and from
counting as a manual answer. Errors appear in the status line
(`Errore AI: ...`, `Invio fallito: ...`).

## Tests

The gate rules, pending tracking, catch-up, mention marking and turn building
are covered by `src/autoreply.rs:505+`; enable rules, `web_chats`, media
selection, failure statuses, `arrived_after` and `manual_reply_after` by
`src/app.rs:1941+`; image descriptions, text updates and the global flag
storage by `src/history.rs`; the `no_reply` tool, descriptions and the retry
policy by `src/ai.rs`; the transcription client by `src/transcribe.rs`. See
[Unit tests](../testing/unit-tests.md).