Text Generation Parameters

Complete reference for all text generation parameters.

Basic Parameters

Parameter	Type	Default	Range	Description
input	string	required	-	Input text to continue
model	TextModel	'llama-3-erato-v1'	-	Model to use
maxLength	number	40	1-2048	Maximum tokens to generate
minLength	number	1	1-2048	Minimum tokens to generate
generateUntilSentence	boolean	-	-	Generate until sentence end

Sampling Parameters

Temperature

Controls randomness. Higher values produce more varied output.

temperature: 1.0  // Default, balanced
temperature: 0.5  // More focused
temperature: 5.0  // More creative

Value	Effect
0.1-1.0	Focused, deterministic
1.0-2.0	Balanced
2.0+	Creative, varied

Range: 0.1-100

Top-K Sampling

Limits vocabulary to top K most likely tokens.

topK: 0    // Disabled (default)
topK: 50   // Consider only top 50 tokens
topK: 100  // Consider only top 100 tokens

Top-P (Nucleus) Sampling

Limits vocabulary to tokens comprising top P probability mass.

topP: 1.0  // Disabled (default)
topP: 0.9  // Consider tokens in top 90% probability
topP: 0.5  // More restrictive

Range: 0.0-1.0

Top-A Sampling

Alternative sampling method based on probability ratios.

topA: 0    // Disabled (default)
topA: 0.1  // Light filtering

Typical P Sampling

Filters tokens based on "typicality" of their probability.

typicalP: 1.0  // Disabled (default)
typicalP: 0.9  // Typical sampling enabled

Tail-Free Sampling (TFS)

Removes low-probability tail tokens.

tailFreeSampling: 1.0  // Disabled (default)
tailFreeSampling: 0.9  // Light tail removal

Range: 0.0-1.0

Min-P Sampling

Sampling method based on minimum probability threshold.

minP: 0    // Disabled (default)
minP: 0.1  // Filter tokens below 10% of top probability

Range: 0.0-1.0

Top-G Sampling

Group-based sampling method.

topG: 0  // Disabled (default)

Range: 0-65536

Mirostat Parameters

Mirostat is an adaptive sampling algorithm that maintains stable perplexity in output.

Parameter	Type	Description
mirostatTau	number	Target perplexity. 0 = disabled
mirostatLr	number	Learning rate (0.0-1.0)

{
  mirostatTau: 5.0,  // Target perplexity
  mirostatLr: 0.1,   // Learning rate
}

Unified Sampling Parameters

Unified sampling is an advanced sampling method.

Parameter	Type	Description
unifiedLinear	number	Linear parameter
unifiedQuad	number	Quadratic parameter
unifiedConf	number	Entropy scale parameter

Repetition Penalty

Basic Penalty

Penalizes repeated tokens to reduce repetition.

repetitionPenalty: 1.0  // Disabled (default)
repetitionPenalty: 1.1  // Light penalty
repetitionPenalty: 1.5  // Strong penalty

Penalty Range

How far back to look for repetitions.

repetitionPenaltyRange: 0     // Full context (default)
repetitionPenaltyRange: 512   // Last 512 tokens
repetitionPenaltyRange: 1024  // Last 1024 tokens

Range: 0-8192

Frequency Penalty

Penalizes based on how often a token appears.

repetitionPenaltyFrequency: 0    // Disabled (default)
repetitionPenaltyFrequency: 0.1  // Light frequency penalty

Range: -16 to 16

Presence Penalty

Penalizes tokens that have appeared at all.

repetitionPenaltyPresence: 0    // Disabled (default)
repetitionPenaltyPresence: 0.1  // Light presence penalty

Range: -16 to 16

Penalty Slope

Controls how quickly penalty decays with distance.

repetitionPenaltySlope: 0    // Disabled (default)
repetitionPenaltySlope: 0.5  // Light slope

Range: 0-10

Phrase Repetition Penalty

Preset phrase repetition penalty modes.

phraseRepPen: 'off'        // Disabled
phraseRepPen: 'very_light'
phraseRepPen: 'light'
phraseRepPen: 'medium'
phraseRepPen: 'aggressive'
phraseRepPen: 'very_aggressive'

CFG (Classifier-Free Guidance)

CFG can enhance consistency between generated content and the prompt.

Parameter	Type	Description
cfgScale	number	CFG strength. 0 = disabled
cfgUc	string	Negative prompt

{
  cfgScale: 1.5,
  cfgUc: 'boring, repetitive',
}

Sequence Control

Stop Sequences

Generation stops when any of these sequences is produced.

stopSequences: ['\n\n', 'THE END', '***']

Note: This triggers tokenization via the token-count endpoint before the request, which adds extra network calls and a small delay. The model parameter is used for tokenization.

Ban Sequences

These sequences are prevented from appearing in output.

banSequences: ['forbidden', 'unwanted']

Note: This also triggers token-count tokenization (extra requests).

Logit Bias

Adjust the probability of specific tokens.

logitBias: [
  { token: 'the', bias: -0.5 },   // Less likely
  { token: 'magic', bias: 1.0 },  // More likely
]

Other Parameters

Parameter	Type	Description
numLogprobs	number	Number of logprobs to return (0-30)
bracketBan	boolean	Ban bracket tokens
prefix	string	Prefix/module to use
order	number[]	Sampler order

Complete Example

const text = await client.text.generate({
  input: 'The wizard raised his staff and',
  model: 'llama-3-erato-v1',
  
  // Length
  maxLength: 100,
  minLength: 20,
  
  // Sampling
  temperature: 1.1,
  topK: 50,
  topP: 0.95,
  typicalP: 1.0,
  tailFreeSampling: 0.95,
  minP: 0.05,
  
  // Repetition
  repetitionPenalty: 1.15,
  repetitionPenaltyRange: 512,
  repetitionPenaltyFrequency: 0.02,
  repetitionPenaltyPresence: 0.0,
  
  // Sequences
  stopSequences: ['\n\n', '***'],
  banSequences: [],
  
  // Bias
  logitBias: [
    { token: 'magic', bias: 0.5 },
  ],
});

Recommended Presets

Creative Writing

{
  temperature: 1.2,
  topP: 0.95,
  repetitionPenalty: 1.1,
  repetitionPenaltyRange: 512,
}

Focused/Factual

{
  temperature: 0.7,
  topK: 40,
  repetitionPenalty: 1.0,
}

Dialogue

{
  temperature: 1.0,
  topP: 0.9,
  repetitionPenalty: 1.05,
  stopSequences: ['\n\n'],
}