|
| | | | |
| | | |
| Stat |
Members: 3665
Articles: 2'599'751
Articles rated: 2609
17 January 2025 |
|
| | | | |
|
Article overview
| |
|
|
BatchPrompt: Accomplish more with less | | Jianzhe Lin
; Maurice Diesendruck
; Liang Du
; Robin Abraham
; | | Date: |
1 Sep 2023 | | Abstract: | Many LLMs are trained to perform zero-shot or few-shot inference using
instruction-based prompts. Crafting prompts for these LLMs typically requires
the user to provide a detailed task description, examples of context and
completion, and single example of context for inference. This regular prompt
baseline is referred to as SinglePrompt in this paper. However, for NLP tasks
where each data point for inference is not necessarily lengthy, the token count
for instructions and few-shot examples in the prompt may be considerably larger
than that of the data point, resulting in lower token-resource utilization
compared with encoder-based models like fine-tuned BERT. This cost-efficiency
issue, affecting inference speed and compute budget, counteracts the many
benefits LLMs have to offer. This paper aims to alleviate the preceding problem
by batching multiple data points into a single prompt, a prompting strategy we
refer to as BatchPrompt. This strategy increases the density of data points,
which in turn leads to improved token utilization. Applying BatchPrompt
naively, however, is very challenging due to significant performance
degradation, as observed in our experiments. We also noticed varying inference
outcomes for the same data point appearing in different positions within a
prompt. To address the quality issue while remain high token-resource
utilization, we introduce Batch Permutation and Ensembling for BatchPrompt, a
simple way that recovers labeling quality through majority votes from data
points placed in varying positions in a batch at the price of more token usage.
To counterbalance the additional token usage caused by the voting process, we
further propose Self-reflection-guided EArly Stopping, which can terminate the
voting process early for data points the LLM confidently handles. | | Source: | arXiv, 2309.00384 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|
REGISTER