Technical white paper · v1.0
Comment Assistant: a human-approved reply system
How Okavyx built an AI-assisted way to answer a YouTube channel's comments, with a person approving every word before it is sent. Architecture, safety model, interface, verification and where it goes next.
Overview
01Executive summary
A channel that publishes regularly collects more comments than its owner can answer well. Answering badly, or not at all, costs goodwill. Answering automatically, in public, under the owner's name, risks something worse. The Comment Assistant sits between those two failures: a model drafts, a person decides, and a gated, auditable pipeline sends only what the person approved.
The service reads new comments from the channel every fifteen minutes, sorts each one into a category, and drafts a short reply only for the few categories where a reply is safe and useful. Every draft is checked by code, not by the model, against a written voice policy. The owner reviews drafts on a mobile-first web page, edits the wording if they like, and approves or rejects. Only then can the poster send the reply, and only after a further series of independent checks.
Everything described here is built, deployed and running. Reading, sorting, drafting, review, approval and the gated poster are live. Sending is installed but switched off by default: the owner turns it on from the page, and the first replies are a supervised trial, capped at five a day. Nothing in this paper depends on the trial having finished; the "Future" section says what comes after it.
This paper describes the technical build for engineers who want to understand it, review it, or reuse the pattern. It deliberately names no channel and quotes no viewer.
Context
02The problem and our principles
Replying to comments is valuable and repetitive. Most comments are praise, a question the video already answers, or a suggestion. A few are abusive, spam, off-limits, or from someone who needs a human. A system that treats them all alike will eventually say something wrong in public, in the owner's voice, and public mistakes are not recoverable.
We set six principles before writing any code. Every design decision traces back to one of them.
| Principle | What it means in the build |
|---|---|
| A person decides | No reply is sent without an explicit approval of the exact wording. There is no automatic mode. |
| Fail closed | When anything is unclear (a missing record, an unreadable answer, a lost connection) the system does nothing. A missed reply costs nothing; a wrong one is public. |
| Untrusted input stays input | Viewer text is data, never instructions. It reaches the model as a quoted value, the model has no tools, and everything it returns is validated in code. |
| Least privilege | The narrowest Google permission that allows replying (which is still broad, so the app adds its own limits on top), one channel pinned by identifier, one upload token scoped to that one job, one admin. |
| Everything is reversible until it is sent | Approvals can be withdrawn. Posting can be switched off in one tap. Stored data can be wiped on request. |
| Honest failure | A failure is shown, logged and never mistaken for success. A reply that may or may not have gone out is flagged for a human and never retried. |
Design
03System architecture
The Comment Assistant is one Node.js service in the Okavyx Labs platform. It runs as a container behind the platform's reverse proxy, stores its state in a shared PostgreSQL instance, and has no inbound connection from the internet except through the platform's authenticated edge.
Components
| Component | Responsibility |
|---|---|
| Ingest API | Receives the video's own context (script, sources, small print, dated claims, things never to say) from the content workstation through a narrow, separately authenticated route. |
| Google connection | OAuth with PKCE. The refresh token is stored encrypted (AES-256-GCM) with a key held outside the database. The connection is pinned to the one expected channel identifier; any other channel is refused and nothing is stored. |
| Poller | Reads new comment threads on a fifteen-minute cycle under a daily quota ceiling far below Google's allowance, so a bug hits our ceiling first. A high-water mark, kept even through data wipes, stops old comments being re-read. |
| Triage and drafting | A small, fast model sorts each comment into one category. For the four replyable categories only, a larger model drafts a short reply from the video's own context. |
| Voice checks | Deterministic code enforces the written voice policy on every draft and on every edit the owner makes. |
| Review queue | A server-rendered web page. The owner reads the whole comment, edits the wording, and approves or rejects. |
| Poster | Sends an approved reply to YouTube only if every gate holds at the moment of sending. |
| Notifier | A push alert that carries counts only (never comment text), at most once an hour, in waking hours. |
| Retention | Comments are kept only as long as needed and are refreshed or deleted in line with YouTube's API policies; spam and skipped comments are deleted sooner, and everything can be wiped on request. |
Behaviour
04From comment to draft
Triage: most comments never reach a model
Cheap, obvious cases are decided in code before any model is called. A comment on a video that has no context record is flagged and never sent to a model, because there is nothing safe to answer from. The rest go to the small model with one instruction: choose a single category from a fixed list. Anything the model is unsure about must be labelled unsure, and anything outside the approved list is treated as unsure too.
| Outcome | Categories | What happens |
|---|---|---|
| Reply (a draft is written) | praise, question, suggestion, criticism | A short draft goes to the review queue. |
| Flag for the owner | safety concern, sensitive topic, advice request, business enquiry, personal details, private matter, correction claim, prompt injection, other language, questions about the channel itself, vendor comparisons, off-topic, no context, unsure | Shown on the Flagged tab, safety concerns first. No draft, or a held-back draft with the reason shown. |
| Skip | abuse, off-limits, spam | Listed, never answered. |
The policy table is a plain document in the repository. The same table drives the prompt, the code's decision map and the owner's understanding of why a comment was flagged.
Drafting: short, plain, from the video only
The drafting model receives the video's own context and one comment, both as quoted data, with the approved voice rules in the system prompt. It has no tools and cannot browse. Its answer must parse into a fixed shape; if it does not, the comment is flagged. A draft is accepted only if it also passes the checks below.
Voice checks: the model is not trusted to police itself
Every draft, and every edit the owner types, is normalised and then checked in code:
- Three sentences at most, one exclamation mark at most, one emoji at most.
- No links in any form, including disguised ones (a dot in brackets, a full-width full stop, an address split across words).
- No
@mentions and no hashtags. - No phrase on the banned list, such as stock enthusiasm or calls to subscribe.
- No number that does not appear in the video's own context, so the assistant cannot invent a statistic.
- Text is normalised first (compatibility folding, and invisible and direction-changing characters removed), so what is checked, stored and sent are the same characters.
Spend control
Model spend is metered per call and capped monthly. When the budget is used, model calls stop; there is no retry loop. A failed call is simply tried again on the next cycle.
Safety
05The posting path and its gates
Sending is the only irreversible step, so it is the most heavily defended. A reply goes out only when every one of the following holds at the moment of sending. Each gate is independent; passing one says nothing about the next.
- Posting is installedThe server was started with posting enabled. If it was not, the stored switch is forced off at boot.
- The owner's switch is onA runtime switch on the page, behind a confirmation screen. Switching off is a single tap.
- Polling is onThe existing kill-switch also stops sending.
- The channel is the expected oneThe connection is pinned to one channel identifier.
- The approval is fresh and deliberateMade after the switch was last turned on, and less than twenty-four hours old, so nothing approved earlier goes out in a rush.
- The comment is unchangedThe approval is bound to a digest of the comment text and the draft. If the viewer edits their comment, the approval is void.
- The live thread still agreesImmediately before sending, the thread is fetched again: same text, same author, no reply already from the channel, and every reply visible. If the owner answered by hand, the assistant stands down.
- The last-moment recheckThe switches and the approval are read once more, so a withdrawal a second ago still wins.
- The rate limitsOne reply a minute, a daily cap (five during the trial) and the quota ceiling.
Outcomes are explicit, and "unknown" is never retried
| Outcome | Meaning | Next step |
|---|---|---|
| Posted | Google confirmed the reply. | Recorded with its identifier. |
| Failed | Certainly not sent (for example, refused before the call). | The comment can be approved again. |
| Unknown | The call may have reached Google but no answer came back. | Never retried. Shown to the owner as "check YouTube" before anything else is done. |
| Rejected | The owner declined. | Nothing is sent. |
Every attempt writes to a separate ledger of answered comments, with no link back to the comment record. Wiping data, or the retention job deleting a comment, therefore cannot cause a second reply to the same comment. A wipe is also refused while a send is unresolved, so the warning for a reply that may have gone out is never deleted with it.
Defence
06Security model
The page is internet-facing, reads text written by the public, and can speak publicly for its owner. We assumed hostile input everywhere and built in layers, so that no single mistake is enough.
| Threat | Defence |
|---|---|
| Someone reaches the review page | Passkey sign-in at the edge, then a second admin check inside the application. The identity provider is a single, central one. |
| A comment tries to instruct the model | Viewer text is data in a quoted value, never in the system prompt. The model has no tools. An attempt to instruct it is its own category and is flagged, not followed. |
| A model draft says something it should not | Checks run in code against the voice policy. A draft that fails is held back with the reason shown, not sent. |
| Script or markup in a comment or a name | Everything from the public is escaped on output. Invisible and direction-changing characters are stripped. The "you" marker on our own comments comes only from a database flag, so a name cannot fake it. |
| A forged form submission | Every state-changing action is a form post carrying a per-user signed token, checked on every request. No client-side script ships at all. |
| A stale or substituted approval | Approvals are bound to a digest of what the owner was looking at, expire after a day, and are void if the comment changes. |
| A double send, or a send to the wrong place | The ledger, the live thread recheck, the channel pin, and "unknown is never retried". |
| Token theft | The refresh token is encrypted at rest with a key held outside the database. Secrets are mounted as files, not environment variables, and never appear in source, logs or the page. |
| A hijacked connect flow | PKCE, a single-use state value bound to the person who started it and expiring in ten minutes, and a browser-supplied header check that refuses a connect started by another site. |
| Abuse of the upload door | A narrow, separately authenticated route, with input validation and size and rate limits. |
| A compromised container | A fixed base image version, non-root user, dropped capabilities, no privilege escalation, a read-only filesystem and resource limits. |
| Data kept too long | Automatic retention limits, a typed-confirmation wipe, and data minimisation: alerts carry counts only. |
Experience
07The review interface
The owner works from a phone as often as a desk, so the page is designed mobile-first and scales up. The visual design was produced in Claude Design, then built as a server-rendered page with no client-side script.
Four tabs, one job each
- Review shows drafts waiting. Each draft shows the whole comment, the video, the confidence, and an editable reply on an inverse panel marked "sent exactly as written". Approve and Reject are small and sit under it.
- Flagged shows what the assistant will not answer and why, safety concerns first, with an icon and a word, never colour alone.
- Decided is the history, with plain-language status and a prominent "check this one on YouTube" for any send that may have gone out.
- More holds connection, polling, posting, appearance and data controls, and links to the full records.
Design decisions that matter
| Decision | Reason |
|---|---|
| Switching posting on needs a confirm page; switching off is one tap | Friction belongs on the risky direction only. |
| One primary action per screen | Approving is the single hero moment; everything else is quiet. |
| Long lists are paged | Twenty a page, read from the database a page at a time, so the page stays usable with thousands of comments. |
| Layout by container width, not device | The same markup lays out for a phone, tablet and desktop. A real-browser check at four widths found no sideways scroll. |
| Light, dark and automatic themes | Stored as a preference cookie; every colour pair is defined once and verified for contrast. |
| Error colours always carry an icon and a word | Colour is never the only signal. |
| Stylesheet addresses carry a content hash | A deploy that changes a stylesheet changes its address (see lessons). |
Assurance
08Verification
We treat "it works on my machine" as no evidence. The service ships with its own probe, a single command that starts a throwaway database, applies the migrations to an empty one, runs every test, and destroys it. It refuses to run against a live database.
- Real everything. Tests run against a real PostgreSQL and mint real signed session cookies through the production authentication middleware. There is no authentication bypass in the test build.
- Mutation checks. A script breaks each guard in turn (the channel pin, the approval age, the ledger, the escaping) and asserts that the suite fails. A guard that nothing notices missing is not a guard.
- Separate review. A separate AI reviewer with no prior context examined the finished posting path twice. Its findings were fixed before sending was ever installed: showing the whole comment, refusing hand-answered threads, forcing the switch off at boot, and more.
- A repository-wide guardrail scans every change for known-bad patterns and fails the build on a new one.
- Real-browser checks at phone, tablet and desktop widths, in light and dark, for layout and overflow.
- A supervised trial. The first live replies are approved one at a time by the owner on a test comment, with a daily cap, before any wider use.
Practice
09Delivery and operations
- Short-lived branches and pull requests. Work never lands on the production branch directly. A pull request is merged only after the probe passes and a review finds nothing.
- Per-service deploys. A deploy rebuilds and recreates this one container. The rest of the platform is never touched by it.
- Approvals. A change deploys only after its probe and review pass; anything that changes sign-in, or switches posting on, needs the owner's explicit typed approval.
- Health and monitoring. The container reports health; the platform's monitoring watches it and the poller records any failure on the page and in the log, with the reason Google gave.
- Rollback. Switch posting off on the page (immediate), then revert the deploy if needed. Migrations apply themselves on start and are additive.
- Phased delivery. The build shipped in stages, each verified before the next: video-context upload; read-only polling; triage and drafts as a dry run; approval and gated sending; the redesigned interface.
Hindsight
10Lessons learned
The platform's proxy replaces an application's content security policy with its own, including a rule that limits where forms may submit. A form that redirected to Google's sign-in was blocked without any error, so the button appeared to do nothing. The fix was to make "Connect" a plain link, guarded by a header check, rather than a form post. We recorded it so nobody turns it back.
Revoking the old token when reconnecting also kills the new one, because they share a grant. Only an explicit disconnect revokes; reconnecting just replaces the stored token.
A poll failure that is not surfaced looks exactly like success. Every poll failure is now kept with Google's own status and reason, shown on the page, and logged.
The most useful rule we applied was that an unclear state is flagged to a person, never guessed at. It cost nothing and removed a whole class of incidents.
One tap to switch posting off, a deliberate confirmation to switch it on. The asymmetry is the feature.
The interface was designed in a dedicated design tool and built to match it. It looked right on the first deploy and needed only layout feedback.
Direction
11The future
Everything below is a proposal drawn from our discussions, or simply a good idea worth recording. None of it is committed, and none of it would be built without the owner choosing it. They are grouped by how close they are.
Next, once the supervised trial has run
Use real approvals and any rejected replies to decide whether five a day is right, and whether any gate is too strict or too loose.
A button for flagged comments the owner answered by hand on YouTube, so the queue shows what is genuinely open. The design pack already reserves its place.
Today the content workstation uploads context when it is run. Triggering it as a video goes live would close the gap where a fresh comment arrives on a video with no context yet.
A standard, honest answer for "is this AI?" (drafted with AI, approved by a person), so the owner can answer it in one tap.
Moderation actions on the worst comments, behind their own gate and confirmation, if the owner wants it.
Approve and move on from a keyboard, and a brief grace period before a reply is sent so a slip can be taken back.
Medium term
Record how the owner changes drafts before approving, and use the pattern to refine the voice rules and the worked examples. The system learns the owner's voice from corrections, with no model training involved.
A labelled bank of past comments that every prompt or model change must be re-run against, so improvements are measured and regressions are caught before they ship.
Time to reply, approval rate, category mix, spend, and a weekly digest of what viewers are asking. The questions people ask most are the best source of the next video.
Replies in the commenter's language with the same checks, once the voice policy exists for each.
Extending the model from top-level comments to short conversations, with the same approval and gates.
An immediate alert, outside the hourly limit, when someone appears to be in distress, so a human sees it quickly.
More of the video's own context shaping the draft, such as sensitivities or running jokes.
Bigger ideas
The question of an "auto mode" came up and the answer was no: a person approves every reply. If that ever changed, the safe shape would be narrow and earned: only praise-type comments, only at high confidence, only after a long record of unedited approvals, with a hard daily cap, sampled human review, and an instant kill-switch. It is recorded here so the trade-offs are written down, not as a recommendation.
The pattern (read, sort, draft, check in code, human approval, gated send) is not specific to one channel. Packaged with its own model key and its own voice policy, it could serve other channels, and the same skeleton could handle email or support replies.
Turn the most common questions into a draft pinned comment or a short community post, again for approval, so repeated questions are answered once, well.
Feed question themes and corrections claimed by viewers back to the video's context record, so a genuine error found in a comment is fixed at the source, and later videos start better informed.
A read-only page showing every reply ever sent with the approval behind it. Trust in an assistant speaking for someone is easier to keep when anyone they authorise can check it.
Reference
AAppendix: limits and settings
| Setting | Value |
|---|---|
| Comment read cycle | Every 15 minutes |
| Triage and drafting cycle | Every 5 minutes |
| Send rate | One reply a minute |
| Daily send cap (trial) | Five replies a day |
| Approval lifetime | Twenty-four hours, and only if made after posting was last switched on |
| Google quota | A ceiling well below Google's allowance |
| Monthly model spend | Capped |
| Comment length seen by a model | 2,000 characters |
| Reply length | Three sentences, 1,000 characters at most |
| Retention | Kept only as long as needed, in line with YouTube's API policies; spam and skipped comments sooner |
| Alerts | Counts only, at most one an hour |
| Page size | Twenty items a page |
Names of the channel, its videos and its commenters are deliberately omitted from this document.
Our work is always evolving, and so are the AI vendors and their products. Nothing described here is set in concrete. Each paper describes its system as it stood in October 2026, when it was written; the date shown is when the app was first built.