What It Does
AI Draft Answers automatically generates draft replies for new Help Scout conversations using AI. It draws from a multi-source knowledge base built from your conversations, documentation, site content, and external websites -- so your team gets a head start on every response.
How to Enable
- Go to DeskPress > AI Agent > AI API and select your AI provider (OpenAI or Google Gemini). Enter your API key and choose your preferred chat and embedding models.
- Go to DeskPress > AI Agent > Knowledge Indexing and configure at least one content source (conversations, docs, site content, or external websites). Run the initial indexing.
- Go to DeskPress > AI Agent > AI Drafts and select the mailbox to monitor, then enable the feature.
Configuration Walkthrough
Step 1: Choose Your AI Provider
Go to DeskPress > AI Agent > AI API and pick a provider:
OpenAI
- Chat models: GPT-4o, GPT-4o-mini, GPT-4 Turbo, GPT-3.5 Turbo
- Embedding models: text-embedding-3-small, text-embedding-3-large, text-embedding-ada-002
- You'll need an OpenAI API key from platform.openai.com
Google Gemini
- Chat models: Gemini 2.0 Flash, Gemini 1.5 Pro, Gemini 1.5 Flash
- Embedding models: text-embedding-004, embedding-001
- You'll need a Gemini API key from Google AI Studio
For most setups, the default model selections work well. More powerful models give better answers but cost more per request.
Step 2: Build Your Knowledge Base
Go to DeskPress > AI Agent > Knowledge Indexing to configure your content sources. You can enable any combination of these four sources:
1. Help Scout Conversations
Fetches past conversations from your Help Scout mailbox and uses them as training data. The system is smart about what it includes:
- Age filtering -- Choose how far back to look: 1, 2, 3, or 5 years, or unlimited.
- Status filtering -- Automatically skips spam and trashed conversations.
- Escalation detection -- Skips conversations that were escalated but never clearly resolved.
- Noise stripping -- Removes greetings, sign-offs, and email signatures so the AI focuses on the actual content.
- Sensitive data redaction -- Credit card numbers, Social Security numbers, passwords, and similar data are automatically replaced with
[REDACTED].
2. Help Scout Docs Articles
Indexes your Help Scout documentation site. Articles are converted from HTML to plain text and organized by collection, category, and article hierarchy.
3. WordPress Site Content
Indexes content from your WordPress site. You choose which post types to include -- posts, pages, WooCommerce products, or any custom post type.
4. External Websites
Crawl external URLs or sitemaps to include content from outside your WordPress site. Set a crawl limit to control how many pages are indexed. HTML is automatically cleaned and normalized.
After configuring your sources, click the indexing button. Content is split into chunks, converted to vector embeddings using your chosen embedding model, and stored locally in your database. When the AI generates a draft, it searches these embeddings to find the most relevant content.
Step 3: Configure the Draft Settings
Go to DeskPress > AI Agent > AI Drafts to fine-tune how drafts are generated:
- Mailbox -- Select which Help Scout mailbox to monitor for new conversations.
- Folder -- Optionally limit monitoring to a specific folder within the mailbox.
- System Prompt -- The template that instructs the AI on how to write drafts. You can customize this to match your team's tone and style.
- Max Context -- How many knowledge base chunks to include when generating a draft. More context can improve accuracy but increases processing time and cost.
- Agent Names -- A list of support agent names (one per line). A random name is assigned to each new conversation and reused for follow-up drafts in the same conversation. This personalizes the sign-off on every draft.
- Signature Template -- A text block appended below the agent name at the end of each draft. Use
{agent_name}as a placeholder for the assigned name. HTML links are supported (e.g.,support@example.com).
Step 4: Set Up a Trigger
The AI needs to know when new conversations arrive. Choose one of these methods:
- Webhook -- Help Scout sends a real-time notification to your site when a new conversation comes in. This is the fastest option. You'll need to set up a webhook in Help Scout and enter the webhook secret in the settings.
- Cron Polling -- WordPress periodically checks Help Scout for new conversations. Configure the interval (e.g., every 5 minutes). This is easier to set up but slightly slower.
Step 5: Test with the Draft Tester
Before going live, use the Draft Tester at the bottom of the AI Drafts tab. Enter a sample ticket subject and message, then click Generate Draft. The tester shows:
- Generation time and the number of context items used
- Knowledge Base Context -- the ranked knowledge base items that informed the draft, with similarity scores, weighted scores, and trust levels
- System Prompt -- the full prompt sent to the AI (collapsible)
- Generated Draft -- the full AI response, which you can copy to clipboard
This helps you verify that your knowledge base is returning relevant context and that the system prompt produces the style of response you want. You can click Regenerate to try again, or Copy Draft to grab the text.
Step 6: Provide Feedback and Teach the AI
The Draft Feedback system lets you rate drafts and teach the AI your preferred writing style through two mechanisms:
Rating Drafts
After generating a draft in the Draft Tester, you can rate it:
- Accept -- marks the draft as a good example. The AI will learn to write similarly in the future.
- Needs Correction -- opens a pre-filled textarea where you can edit the draft to show what the correct response should look like. The corrected version is stored as a high-priority training example.
Production drafts posted to Help Scout appear in the Draft History card on the same tab. You can expand any draft to review it and provide the same Accept/Needs Correction feedback.
How the AI Learns
Feedback is used in two ways:
- Correction Examples (RAG) -- Accepted and corrected drafts are stored as vector embeddings in your knowledge base with the highest priority weight. When generating future drafts, the AI sees these examples labeled as "PREFERRED STYLE" and follows them closely.
- Style Guidelines -- Once you have at least 3 feedback entries, click Generate from Feedback in the Style Guidelines card. The AI analyzes all your feedback and produces a concise list of writing style guidelines. These guidelines are automatically prepended to the system prompt for every future draft. You can also edit them manually.
Settings Reference
AI Settings Tab
| Setting | Description |
|---|---|
| AI Provider | Choose between OpenAI or Google Gemini |
| API Key | Your provider's API key |
| Chat Model | The language model used to generate draft replies |
| Embedding Model | The model used to convert content into searchable vectors |
Knowledge Indexing Tab
| Setting | Description |
|---|---|
| Include Docs | Include Help Scout Docs articles in the knowledge base |
| Include Site Content | Include WordPress content in the knowledge base |
| Site Post Types | Which WordPress post types to index |
| Include External Sites | Enable crawling of external websites |
| External Sitemaps | Sitemap URLs to crawl |
| External URLs | Individual page URLs to crawl |
| Crawl Limit | Maximum number of external pages to crawl |
| Max Conversation Age | Maximum age of conversations to process |
AI Drafts Tab
| Setting | Description |
|---|---|
| Mailbox | Which Help Scout mailbox to monitor for new conversations |
| Folder | Specific folder within the mailbox to monitor (optional) |
| System Prompt | The template that controls how the AI writes drafts |
| Max Context | Number of knowledge base chunks to include per draft |
| Polling Interval | How often WordPress polls Help Scout for new conversations |
| Webhook Secret | Secret key for validating Help Scout webhook requests |
| Agent Names | Support agent names (one per line) — randomly assigned per conversation |
| Signature Template | Text block appended after the agent name; supports {agent_name} placeholder and HTML |
| Style Guidelines | AI-generated writing guidelines learned from your draft feedback; prepended to the system prompt |
Indexing Your Knowledge Base
You manage indexing from DeskPress > AI Agent > Knowledge Indexing:
- Initial index -- Click the index button to build your knowledge base for the first time. This runs in the background, so you can navigate away.
- Daily reindex -- A scheduled cron job automatically reindexes your content once a day to keep things fresh.
- Manual reindex -- Hit the reindex button any time you've made significant content changes and don't want to wait for the daily run.
The indexing status is displayed on the page so you can track progress.
Tips and Common Questions
Which AI provider should I choose?
Both work well. OpenAI's GPT-4o is excellent for nuanced responses. Gemini 2.0 Flash is fast and cost-effective. Try both and see which fits your needs and budget.
How much does it cost to run?
Costs depend on your AI provider's pricing and how many conversations you process. Each draft typically uses a small number of tokens. Start with a smaller knowledge base to estimate costs before scaling up.
Start small, then expand.
Begin with one or two content sources and see how the drafts look. Once you're happy with the quality, add more sources to improve coverage.
The AI works best with clear resolutions.
If your Help Scout conversations tend to end with clear, helpful answers, the AI will produce better drafts. Conversations that trail off or get escalated without resolution are automatically filtered out.
Can I customize the writing style?
Yes. Edit the System Prompt to change how the AI writes. You can specify tone ("friendly and professional"), formatting ("use bullet points for steps"), or restrictions ("never mention competitor products"). You can also use the Draft Feedback system to teach the AI your preferred style over time.
What happens if the knowledge base is empty?
The AI won't have any context to draw from and will produce generic responses. Always index at least one content source before enabling drafts.