文本流式生成

实时流式输出生成文本。

基本用法

import { NovelAI } from 'novelai-sdk-unofficial';

const client = new NovelAI({ apiKey: 'your-api-key' });

const stream = client.text.generateStream({
  input: 'Once upon a time',
  maxLength: 100,
});

for await (const chunk of stream) {
  process.stdout.write(chunk);
}

console.log(); // 结尾换行

工作原理

generateStream 方法返回一个异步生成器，在生成时逐块产出文本。每个块是包含一个或多个 token 的字符串。

const stream = client.text.generateStream({
  input: 'Hello',
  maxLength: 50,
});

// 数据块在生成时到达
for await (const chunk of stream) {
  console.log('收到:', JSON.stringify(chunk));
}

带所有参数

流式生成支持所有标准生成参数：

const stream = client.text.generateStream({
  input: 'The story begins',
  model: 'llama-3-erato-v1',
  temperature: 1.1,
  maxLength: 100,
  topP: 0.95,
  repetitionPenalty: 1.1,
  stopSequences: ['\n\n'],
});

for await (const chunk of stream) {
  process.stdout.write(chunk);
}

说明：使用 stopSequences / banSequences 时，会在开始流式生成前额外调用 token-count 端点进行 token 化。

收集完整输出

async function generateText(input: string): Promise<string> {
  const stream = client.text.generateStream({ input, maxLength: 100 });
  
  let fullText = '';
  for await (const chunk of stream) {
    fullText += chunk;
  }
  
  return fullText;
}

const result = await generateText('Once upon a time');
console.log(result);

进度回调

async function generateWithCallback(
  input: string,
  onChunk: (chunk: string, accumulated: string) => void
): Promise<string> {
  const stream = client.text.generateStream({ input, maxLength: 100 });
  
  let accumulated = '';
  for await (const chunk of stream) {
    accumulated += chunk;
    onChunk(chunk, accumulated);
  }
  
  return accumulated;
}

// 使用
await generateWithCallback('Hello', (chunk, total) => {
  console.log(`新增: "${chunk}" | 总长度: ${total.length}`);
});

错误处理

import { NetworkError, AuthenticationError } from 'novelai-sdk-unofficial';

try {
  const stream = client.text.generateStream({
    input: 'Hello',
    maxLength: 100,
  });

  for await (const chunk of stream) {
    process.stdout.write(chunk);
  }
} catch (error) {
  if (error instanceof NetworkError) {
    console.error('流式传输过程中连接丢失');
  } else if (error instanceof AuthenticationError) {
    console.error('API Key 无效');
  } else {
    console.error('生成失败:', error);
  }
}

使用场景

聊天界面

async function streamResponse(userMessage: string) {
  const context = `User: ${userMessage}\nAssistant:`;
  
  const stream = client.text.generateStream({
    input: context,
    stopSequences: ['\nUser:', '\n\n'],
    maxLength: 150,
  });

  process.stdout.write('Assistant: ');
  for await (const chunk of stream) {
    process.stdout.write(chunk);
  }
  console.log();
}

打字机效果

async function typewriterEffect(input: string, delayMs: number = 50) {
  const stream = client.text.generateStream({ input, maxLength: 100 });
  
  for await (const chunk of stream) {
    for (const char of chunk) {
      process.stdout.write(char);
      await new Promise(resolve => setTimeout(resolve, delayMs));
    }
  }
}

Web 服务器（Express）

import express from 'express';

const app = express();

app.get('/generate', async (req, res) => {
  const input = req.query.input as string;
  
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');

  const stream = client.text.generateStream({ input, maxLength: 100 });
  
  for await (const chunk of stream) {
    res.write(`data: ${JSON.stringify({ text: chunk })}\n\n`);
  }
  
  res.write('data: [DONE]\n\n');
  res.end();
});

取消

使用 AbortController 显式取消流式生成：

const controller = new AbortController();

const stream = client.text.generateStream(
  { input: 'Once upon a time', maxLength: 100 },
  controller.signal,
);

// 5 秒后取消
setTimeout(() => controller.abort(), 5000);

try {
  for await (const chunk of stream) {
    process.stdout.write(chunk);
  }
} catch (error) {
  if (error.name === 'AbortError') {
    console.log('\n生成已取消');
  } else {
    throw error;
  }
}

提示

1. 流式传输适合用户体验 - 用户可以立即看到输出，而不是等待。

2. 处理断开连接 - 网络问题可能中断流。

3. 内存高效 - 流式传输不会缓冲整个响应。

4. 相同参数 - 所有生成参数都适用于流式传输。