type: workflow title: AI auto-reply workflow description: End-to-end trace of the AI autoreply — enabling it per chat (F3 on the selected or open chat) or globally (F4 / web panel) for private chats and groups, the ReplyGate rules, the reply delay that batches messages into one reply, the drop when the owner answered by hand, voice-note transcriptions and image descriptions processed once and reused, the startup catch-up, building the AI context from SOUL.md, the no_reply tool, retries and sending the reply. tags: [ai, autoreply, workflow, vision, transcription, reply-gate, reply-delay, catch-up, image-description] sources:

id: openwiki-source-9fec9d1af4 resource: repo://src/app.rs
id: openwiki-source-043e6252ba resource: repo://src/autoreply.rs
id: openwiki-source-d6b2055bbb resource: repo://src/config_watch.rs
id: openwiki-source-2a737474d8 resource: repo://src/config.rs
id: openwiki-source-ce0c4ea8cd resource: repo://src/history.rs generated: { by: "opencode", at: "2026-10-24T16:39:50.206Z" } ---

AI auto-reply workflow

The autoreply answers incoming WhatsApp messages with an AI-generated text, only for chats where the user turned it on — one by one, or all at once with the global switch. Decision rules and context building are pure functions in src/autoreply.rs (no I/O); orchestration lives in App; the HTTP calls are done by the AI client (text and vision) and by the transcription endpoint (src/transcribe.rs).

WaEvent::Message { message, image, audio }
  -> History::save_message (new only)
  -> App::maybe_autoreply   (enabled = global OR per-chat flag)
       ReplyGate::should_schedule? --yes--> mark_pending + timer(reply_delay_seconds)
       media from an allowed sender?  ----> keep_image / keep_audio
  ... AppEvent::ReplyDue
  -> prepare_reply: reply_context (still enabled? AI? owner not answered last?)
       --no--> finish_reply (drop)
       audios without transcription? --yes--> download + transcribe (parallel, max 3)
                                               -> AppEvent::AudiosTranscribed -> text stored
       images without description?  --yes--> download + describe_image (parallel, max 5)
                                               -> AppEvent::ImagesDescribed -> description stored
  -> start_reply: messages_since(yesterday 00:00, 200) -> mark @[te] -> build_turns
  -> AiClient::complete (tools: [no_reply], up to ai_retries retries)
  -> AppEvent::AiReply
       NoReply -> status "AI: nessuna risposta necessaria"
       Text    -> still enabled AND owner did not answer meanwhile? -> send_text -> stored is_ai = true
  -> finish_reply: newer message arrived meanwhile? -> schedule one new reply
WaEvent::OfflineSyncCompleted (first time only)
  -> catch_up -> mark_pending + prepare_reply (no delay) per due private chat

1. Enabling

There are two independent switches, and a chat is enabled when either is on (autoreply_enabled, src/app.rs:858-860):

Per chat — F3 toggles the flag of the focused chat: the selected row of the (filtered) list when the list has focus, the open chat when the input has focus (toggle_autoreply / toggle_chat_autoreply, src/app.rs:1348-1378), so a chat can be enabled without opening it; the web control panel can toggle any listed chat the same way. Turning it off is always allowed; turning it on requires a configured AI (API key in config.toml and a non-blank SOUL.md, see Configuration) and a chat that supports autoreply (can_enable_autoreply, src/app.rs:1894-1902), otherwise the status shows Autoreply disponibile solo nelle chat private e nei gruppi. The flag is persisted in the chats table of the history store (default off).
Global ("AI tutte") — F4 or the web panel toggles global_autoreply (toggle_global_autoreply, src/app.rs:1433-1444); enabling requires an AI client. It is persisted in the settings table, loaded at startup and does not change any per-chat flag, so turning it off restores the per-chat selection. The TUI marks it in the list title and footer (Terminal UI).

The global switch does not bypass the chat-type rule: meets_content_rules still requires a supported chat, so broadcasts, status and newsletters are never answered.

The AI setup itself follows the configuration files live: when config.toml, SOUL.md or VISION.md change, the configuration is reloaded and the AI client rebuilt within about 2 seconds (apply_reloaded_config, src/app.rs:422-434, see Configuration). Replies already running keep the previous client; later schedules, contexts and requests use the new prompt, models, reply_delay_seconds, catch_up_seconds, ai_retries and transcribe_* keys. If the new files leave the AI unconfigured, every autoreply path stops as described below, but the persisted flags are kept.

supports_autoreply (src/autoreply.rs:105-108) accepts private chats and groups:

private: JID server s.whatsapp.net or lid (is_private_chat, src/autoreply.rs:83-92);
group: JID server g.us (is_group_chat, src/autoreply.rs:94-103);
broadcasts (including status@broadcast), newsletters and malformed JIDs are excluded.

2. Trigger and ReplyGate rules

Only messages that History::save_message reports as new reach maybe_autoreply (on_message, src/app.rs:485-499; maybe_autoreply, src/app.rs:509-532), so duplicates and history-sync re-deliveries of stored messages never trigger. maybe_autoreply needs an AI client and a system prompt (ai_setup, src/app.rs:987-989), refreshes the own JIDs if unknown and computes enabled (global or per chat).

ReplyGate::should_schedule(message, enabled) (src/autoreply.rs:189-191) returns true only when all hold (is_reply_candidate and meets_content_rules, src/autoreply.rs:247-265):

Rule	Detail
Not pending	the chat has no reply already scheduled or in progress
Enabled	global autoreply on, or the per-chat flag on
Sender allowed	not sent by the user — except human-written messages in the self-chat ("Messaggi a te stesso"); AI messages never trigger (`sender_allowed`, `src/autoreply.rs:178-180`)
Supported chat	private chat or group (`supports_autoreply`)
No backlog	`timestamp >= started_at`, the process start time (`is_backlog`, `src/autoreply.rs:198-200`)
Has content	text not blank and not `[media]`/`[messaggio non supportato]`; voice notes and images do trigger

The self-chat is detected by comparing the chat JID with the account's own PN/LID JIDs, ignoring a :N device suffix (is_self_chat, src/autoreply.rs:154-157; strip_device_suffix, src/autoreply.rs:299-309). Groups follow exactly the same rules as private chats; mentions and quotes no longer change anything.

3. Reply delay and batching

When should_schedule passes, schedule_reply (src/app.rs:539-542) marks the chat pending in the gate (mark_pending, src/autoreply.rs:208-210) and spawn_reply_timer (src/app.rs:1586-1591) emits AppEvent::ReplyDue { chat_jid } after reply_delay_seconds (default 30; 0 answers at once). The chat stays pending until finish_reply, so every message arriving during the wait — and while the AI is generating — gets no timer of its own: the reply is built from the context loaded when it starts, so it covers everything received during the wait.

handle_reply_due (src/app.rs:634-641) takes the images and audios kept for the chat and, if the chat is still pending (a local deletion removes the pending mark), calls prepare_reply (src/app.rs:656-679). Its reply_context (src/app.rs:868-880) drops the reply — and finish_reply ends the pending state — when:

the autoreply was turned off (both switches) during the wait;
the AI is not configured;
the chat has no message in the context window;
the latest message was written by hand by the owner outside the self-chat (answered_by_owner, src/autoreply.rs:237-239): the owner already answered, so the AI stays silent.

Otherwise it transcribes the new audios and describes the new images first (see §5) or calls start_reply at once.

After the reply

AiReply is always followed by finish_reply (src/app.rs:837-852), which clears the pending state and the answered_up_to snapshot. If a reply was started and the latest stored message is a different one (arrived_after, src/app.rs:1696-1698) that itself passes should_schedule, a new reply is scheduled (with a fresh delay, keeping the media received meanwhile); otherwise the kept images and audios are dropped. So messages that arrive while a reply is being generated are answered once, afterwards, rather than being lost or triggering parallel replies.

4. Startup catch-up

Messages received while the app was off are delivered on connection and are older than started_at, so the live rules treat them as backlog. When WhatsApp signals that the offline queue has been delivered (WaEvent::OfflineSyncCompleted), on_offline_sync_completed (src/app.rs:924-930) runs catch_up once per process (catch_up_done; later reconnections skip it).

catch_up / catch_up_chats (src/app.rs:940-981) look at every enabled chat — all chats when the global autoreply is on, otherwise those with the per-chat flag — and take its last stored message. ReplyGate::catch_up_due (src/autoreply.rs:278-290) accepts it only when:

catch_up_seconds (default 7200) is positive — 0 disables the catch-up;
the chat is not a group;
the message is at most catch_up_seconds old and older than the app start (newer messages follow the live path);
it passes meets_content_rules, so a last message written by the user (already answered) never qualifies.

Each due chat is marked pending and goes through prepare_reply without the reply delay, together with the images and audios of its offline messages (backlog_images, backlog_audios); offline media of other chats are dropped. When at least one reply went ahead the status shows Risposte di recupero: N. Offline images that arrive after the catch-up already ran are described on their own, without a reply (describe_late_image, src/app.rs:571-581), and offline audios transcribed the same way (transcribe_late_audio, src/app.rs:616-626).

5. Media: image descriptions and voice-note transcriptions

Media bytes are never sent to the text model. Each image or audio sent by an allowed sender in an enabled chat is kept by keep_image (src/app.rs:550-564) or keep_audio (src/app.rs:590-609; audios are ignored when transcribe_key is unset) and, just before the reply that covers it, processed once:

audios are transcribed first by the configured audio/transcriptions endpoint (spawn_transcriptions → spawn_audio_transcriptions, src/app.rs:688-715, src/app.rs:1796-1830), in parallel. The transcription is appended to the stored message text as [messaggio vocale] <text> (handle_audios_transcribed, src/app.rs:784-829, via History::update_message_text), so it enters every later context; failures stay untranscribed and the status shows Trascrizione vocale fallita (N): <first error>. When the reply was waiting for them, the postponed images are described next, or the reply starts;
images are then described by the vision model (AiClient::describe_image, request text from VISION.md or the built-in one, see AI client). The vision model sees only that request and the image, never the conversation. The description is stored in the image_descriptions table and reused in every later context.

select_media_to_describe (src/app.rs:1655-1676) keeps only the items whose message is in the reply context and that are not already processed or in progress (describing / transcribing, plus stored descriptions or transcribed texts, respectively images_to_describe and audios_to_transcribe, src/app.rs:888-918), and caps them to the most recent ones: 5 images (MAX_IMAGES_PER_REPLY) or 3 audios (MAX_AUDIOS_PER_REPLY); older media of the same batch are dropped for good, to cap the cost of albums and bursts of voice notes. Without openai_vision_model images are not described; without transcribe_key audios are not transcribed (descriptions_apply, src/app.rs:1855-1857).

Both processing kinds run in parallel with one task per item; each download (download_image_with_retry / download_audio_with_retry, src/app.rs:1737-1784) makes up to 3 attempts, each abandoned after 60 s, waiting 2 s × attempt between them, and the bytes stay in memory. handle_images_described (src/app.rs:747-770) stores each description whose message still exists (History::message_exists), so a chat deleted meanwhile gets no orphan rows; failed images stay without description, the others are unaffected, and the status shows Descrizione immagine fallita (N): <first error>. If the chat is still pending, the reply then starts (re-checking reply_context).

6. Building the context

start_reply (src/app.rs:1009-1028) loads the context with History::messages_since(chat_jid, context_start(now), CONTEXT_MESSAGES):

context_start (src/autoreply.rs:466-468) is local midnight of yesterday (falls back to now minus two days if local time cannot be resolved, src/autoreply.rs:475-480);
CONTEXT_MESSAGES = 200 (src/autoreply.rs:15) caps the window, keeping the most recent messages in ascending order (History store).

It records the last context message in answered_up_to, replaces every @<own number or LID> in the texts with @[te] (mark_own_mentions, src/autoreply.rs:165-170; replace_mention, src/autoreply.rs:444-456; a longer number is never matched) and calls build_turns(system_prompt, messages, TurnContext) (src/autoreply.rs:495-503) with the self-chat/group flags, the chat name and the stored image descriptions:

first a System turn (system_turn_content, src/autoreply.rs:427-436): the SOUL.md prompt as written, then — in normal chats and groups — a line Chat privata con «name». or Chat di gruppo «name». and ROLE_NOTE_CHAT (assistant messages were written by the account owner it answers for, user messages by the contact or, in groups, by the participants named in front); in the self-chat only ROLE_NOTE_SELF_CHAT (user is the owner, assistant its own previous replies). Parts are separated by a blank line;
then the last CONTEXT_MESSAGES messages, skipping blank texts;
roles (turn_role, src/autoreply.rs:318-321): in normal chats and groups own messages are Assistant; in the self-chat, where every message is from_me, only is_ai messages are Assistant;
in groups every User turn is prefixed with "<sender>: ", where the sender is the trimmed push name or else the user part of the sender JID (sender_label, message_to_turn, src/autoreply.rs:387-418);
only User turns are rewritten (user_content, src/autoreply.rs:370-379), with plain facts — how to react is left to SOUL.md:
- voice note without a transcription → [Il contatto ha inviato un messaggio vocale che non puoi ascoltare]; a transcribed one instead passes through as its stored text ([messaggio vocale] <transcription>);
- image with a stored description → [Il contatto ha inviato un'immagine. Descrizione: <text>], without one → [Il contatto ha inviato un'immagine che non puoi vedere]; a caption is appended as \nDidascalia: <caption>;
- any other text is kept as is.

7. Request, `no_reply` and retries

spawn_ai_request (src/app.rs:1600-1611) calls AiClient::complete, which offers the model a single parameterless tool, no_reply. When the model calls it, the outcome is Reply::NoReply: nothing is sent and the status shows AI: nessuna risposta necessaria. The app never says when to use it — the cases belong in SOUL.md.

Every AI call (text completion and image description) is retried by the AI client up to ai_retries times (default 2, waiting 2 s, 4 s, …) on transient errors — HTTP 429, 5xx, timeouts, transport errors, malformed or empty replies — while other 4xx errors fail at once; transcription calls perform a single request instead. A reply can therefore arrive noticeably later than the trigger; the checks in §8 happen after all retries. The request task is awaited by a second task, so AiReply is always delivered and the chat never stays pending. When ai_log = true, every attempt is also appended to ai_requests.log.

8. Sending

A text reply is handled by send_ai_reply_if_enabled (src/app.rs:1064-1077), which re-checks two things:

autoreply_enabled: if both the global and the per-chat switch are off by then, the reply is discarded with status Risposta AI scartata: autoreply disattivato;
owner_answered_meanwhile (src/app.rs:1084-1090): if a hand-written owner message follows the last message of the reply context (manual_reply_after), the reply is discarded with Risposta AI scartata: hai già risposto.

Otherwise the text is sent with spawn_send(..., is_ai = true); on success handle_sent (src/app.rs:1096-1120) stores it as an own message with is_ai = true, which keeps it from re-triggering the self-chat and from counting as a manual answer. Errors appear in the status line (Errore AI: ..., Invio fallito: ...).

Tests

The gate rules, pending tracking, catch-up, mention marking and turn building are covered by src/autoreply.rs:505+; enable rules, web_chats, media selection, failure statuses, arrived_after and manual_reply_after by src/app.rs:1941+; image descriptions, text updates and the global flag storage by src/history.rs; the no_reply tool, descriptions and the retry policy by src/ai.rs; the transcription client by src/transcribe.rs. See Unit tests.

ai-autoreply.md 18 KB History Raw