Prompt tuning

A soft prompt is not text. It is a short sequence of vectors living in the model's embedding space, prepended to the real input and trained by backpropagation through a frozen model. Because the vectors are not constrained to be embeddings of actual vocabulary items, the search space is larger than any amount of prompt engineering can reach, and because gradients flow only into those vectors, nothing in the model moves.

Prefix-tuning (2021-01-01) came first and put learned vectors in every layer, not just at the input, so later layers see a prefix too. Applied to GPT-2 for table-to-text and BART for summarization, it reported that learning only 0.1% of the parameters, prefix-tuning obtains comparable performance in the full data setting, outperforms fine-tuning in low-data settings, and extrapolates better to examples with topics unseen during trainingsource, accessed 2026-08-28. That last clause is the one to notice — the constraint is doing work, not just saving memory.

Prompt tuning (2021-04-18) stripped the method to input embeddings alone, and its accounting is stark: 409,600 trainable parameters for a 100-token prompt on T5-XXL — 0.00368% of the model, over 20,000 times fewer task-specific parameters than tuning all 11.1Bsource, accessed 2026-08-28. A task's entire specialization is a file you would not notice on disk.

The finding is about scale, not about the method. The paper's own title says so: prompt tuning becomes more competitive with scale: as models exceed billions of parameters the method closes the gap and matches the strong performance of tuning all model weightssource, accessed 2026-08-28. Read that in the direction that costs something. On Small, Base, Large and XL, prompt tuning is substantially behind tuning the weights; only at 11B does it draw level. The technique does not make a small model cheap to adapt — it becomes viable exactly where full fine-tuning stops being affordable, which is convenient, and is a property of the models rather than of the idea.

The scale dependence shows up as a change in how much the details matter: at smaller model sizes there are large gaps between the different prompt initializations, and once the model is scaled to XXL those differences disappearsource, accessed 2026-08-28. A method whose hyperparameters stop mattering as the model grows is a method whose difficulty was optimization difficulty all along.

Where it still wins outright. A learned prompt is part of the input, so different tasks' prompts can travel in one batch through one set of weights — enables mixed-task inference using the original pre-trained model, so one generalist model can simultaneously serve many taskssource, accessed 2026-08-28. Low-rank adapters can only match that by staying unmerged, which reintroduces the per-token overhead that merging exists to remove. And the frozen-model constraint pays a second dividend the authors measured rather than asserted: conditioning a frozen model with soft prompts confers benefits in robustness to domain transfer, compared with tuning all model weightssource, accessed 2026-08-28.