Skip to content
Edit on GitHub

AI provider

Implement AiProvider against your own backend or GoTipe Cloud — streaming, cancellation, errors and result validation — without exposing model keys to the browser.

@gotipe/aiMIT

An AiProvider is the only part of the AI layer that knows where a model runs. It turns a request and its context into text, or into structured proofreading and review results. Write one against your backend, or against the AI endpoint of GoTipe Cloud.

Caution:

Never ship a model API key to the browser. A key in your bundle, in local storage or in a request header set by page code can be read by anyone who opens the page. Keep the key on your server or in GoTipe Cloud, and let the browser send a short-lived token for the signed-in user.

The contract

Method Required Called for Returns
name yes — A label for the provider.
generateCompletion(context) yes Ghost text, continueWriting AsyncIterable<string>
generateSuggestions(prompt, context) yes AiManager.getSuggestions() Promise<AiSuggestion[]>
executeCommand(command, context) yes AiManager.executeCommand(), and the fallbacks below Promise<string>
streamRewrite(text, context) no rewrite (Improve). Fallback: executeCommand("rewrite") with selectedText AsyncIterable<string>
generateParagraph(prompt, context) no askAi, addParagraph, freeText. Fallback: executeCommand("addParagraph:" + prompt) AsyncIterable<string>
proofread(context) no proofread Promise<AiProofreadSuggestion[]>
reviewComments(context) no review Promise<AiComment[]>

Without proofread or reviewComments, those commands fail with "This AI provider does not support proofreading" or "This AI provider does not support review comments".

Rules

  • Streaming. Return an async iterable, such as an async generator, and yield text chunks. A chunk that is not a string fails the request. Ghost text shows chunks as they arrive; inserted text waits for the end of the stream.
  • Cancellation. Pass context.signal to fetch, or stop your work when it aborts. The manager drops a late result in any case, and an AbortError is never reported.
  • Errors. Throw an Error with a message for the person. The editor reports it as EXTENSION_ERROR with details.extension === "ai", and the AI command bar shows it.
  • Context. The context holds detached copies of blocks. Send your server only what the model needs.

A provider for your backend

ai-provider.ts
import type { AiContext, AiProofreadSuggestion, AiProvider, AiSuggestion } from "@gotipe/ai";

async function post(path: string, context: AiContext, extra: object = {}): Promise<Response> {
  const response = await fetch(path, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      selectedText: context.selectedText,
      blocks: context.surroundingBlocks,
      fullText: context.fullText,
      ...extra,
    }),
    signal: context.signal, // GoTipe aborts it when the request is cancelled
  });
  if (!response.ok) throw new Error(`The assistant is unavailable (${response.status})`);
  return response;
}

async function* stream(path: string, context: AiContext, extra?: object): AsyncIterable<string> {
  const response = await post(path, context, extra);
  if (!response.body) throw new Error("The assistant sent no text");
  const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
  for (;;) {
    const { value, done } = await reader.read();
    if (done) return;
    yield value;
  }
}

export const backendProvider: AiProvider = {
  name: "my-backend",
  generateCompletion: (context) => stream("/api/ai/continue", context),
  streamRewrite: (text, context) => stream("/api/ai/rewrite", context, { text }),
  generateParagraph: (prompt, context) => stream("/api/ai/paragraph", context, { prompt }),
  async executeCommand(command, context) {
    return (await post("/api/ai/command", context, { command })).text();
  },
  async generateSuggestions(prompt, context): Promise<AiSuggestion[]> {
    return (await post("/api/ai/suggestions", context, { prompt })).json();
  },
  async proofread(context): Promise<AiProofreadSuggestion[]> {
    return (await post("/api/ai/proofread", context)).json();
  },
};

Each route authenticates the user, adds the model key on the server, calls the model, and stops that call when the browser disconnects. The streaming routes send plain UTF-8 text as it is generated; /api/ai/command sends plain text; the other two send JSON.

Proofreading and review results

The manager checks structured results against the document before anything reaches the UI. One invalid item fails the whole result.

Result Must hold
AiProofreadSuggestion A unique id; the blockId of an existing block; from and to as UTF-16 offsets into the block's text that do not split a character; originalText equal to the text between them; type insert (with from === to), delete or replace; a string suggestedText; no overlap with another suggestion in the block.
AiComment A unique id, an existing blockId, a string body, valid from and to, and quotedText equal to the text between them. category is optional.

A block's text is the concatenation of its spans' text. Compute offsets in that text, not in Markdown or HTML.

GoTipe Cloud

GoTipe Cloud keeps the model keys (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_AI_API_KEY) on the server and exposes one endpoint, POST /v1/ai/complete. Call it with the user's token; the token needs the ai:use scope (see Authentication).

Field Type Description
command string Required: complete, rewrite, summarize, translate, fix_grammar or change_tone.
prompt string The instruction. prompt or context is required.
context JSON The text the model works on.
document_id string Optional. The user needs viewer access to it.
provider string openai, anthropic or google. Default: AI_DEFAULT_PROVIDER (openai).
model string Default: AI_DEFAULT_MODEL (gpt-4o-mini). Required with a provider other than the default.
max_tokens number 1 to AI_MAX_TOKENS (4096 by default).
temperature number 0 to 2.
stream boolean true for server-sent events.

With stream: false the answer is { content, model, provider, usage }. With stream: true it is a text/event-stream of a start event, content events with { text } and a final done event with { usage }; a failure after the start arrives as an error event with { code, message }. Errors before the stream are JSON { error: { code, message } }: 400 VALIDATION_ERROR, 402 QUOTA_EXCEEDED when the plan's monthly AI requests are used up, 429 RATE_LIMITED above RATE_LIMIT_AI (60 per minute by default), 503 SERVICE_UNAVAILABLE when no key is configured or the model fails.

cloud-ai-provider.ts
import type { AiContext, AiProvider } from "@gotipe/ai";
import { createId } from "@gotipe/core";

const textOf = (context: AiContext): string =>
  context.selectedText ??
  context.fullText ??
  (context.surroundingBlocks ?? [])
    .map((block) =>
      typeof block.content === "string"
        ? block.content
        : (block.content ?? []).map((span) => span.text).join(""),
    )
    .join("\n");

export function cloudAiProvider(apiUrl: string, getToken: () => Promise<string>): AiProvider {
  async function* complete(command: string, prompt: string, context: AiContext) {
    const response = await fetch(`${apiUrl}/v1/ai/complete`, {
      method: "POST",
      headers: { "Content-Type": "application/json", Authorization: `Bearer ${await getToken()}` },
      body: JSON.stringify({ command, prompt, context: textOf(context), stream: true }),
      signal: context.signal,
    });
    if (!response.ok || !response.body) {
      const failure = await response.json().catch(() => null);
      throw new Error(failure?.error?.message ?? `AI request failed (${response.status})`);
    }
    const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
    let buffer = "";
    for (;;) {
      const { value, done } = await reader.read();
      if (done) throw new Error("The AI stream ended early");
      buffer += value;
      for (let end = buffer.indexOf("\n\n"); end >= 0; end = buffer.indexOf("\n\n")) {
        const message = buffer.slice(0, end);
        buffer = buffer.slice(end + 2);
        const event = /^event: (.+)$/m.exec(message)?.[1];
        const data = JSON.parse(/^data: (.+)$/m.exec(message)?.[1] ?? "{}");
        if (event === "content") yield String(data.text);
        else if (event === "error") throw new Error(data.message);
        else if (event === "done") return;
      }
    }
  }
  const collect = async (chunks: AsyncIterable<string>) => {
    let text = "";
    for await (const chunk of chunks) text += chunk;
    return text;
  };
  return {
    name: "gotipe-cloud",
    generateCompletion: (context) => complete("complete", "", context),
    streamRewrite: (text, context) =>
      complete("rewrite", "Improve the text.", { ...context, selectedText: text }),
    generateParagraph: (prompt, context) =>
      complete("complete", `Write one new paragraph: ${prompt}`, context),
    executeCommand: (command, context) => collect(complete("rewrite", command, context)),
    generateSuggestions: async (prompt, context) => [
      {
        id: createId(),
        type: "rewrite",
        content: await collect(complete("rewrite", prompt, context)),
        confidence: 1,
      },
    ],
  };
}

The endpoint returns text only. Proofreading and review need structured results, so this provider leaves them out; serve them from your own backend if you need them.

Limitations

  • @gotipe/ai has no vendor SDK and no tool or function calling.
  • Requests to GoTipe Cloud count against the plan's AI quota. A model call times out after two minutes, and a streamed answer cannot be replayed with an idempotency key.
  • The PII check of the manager runs in the browser before your provider is called. Apply your own server-side policy as well.