Papers

The following are the latest papers (sorted by release date) on prompt engineering for large language models (LLMs). We update the list of papers on a daily/weekly basis.

Overviews

Prompt Design and Engineering: Introduction and Advanced Methods (opens in a new tab) (January 2024)
A Survey on Hallucination in Large Language Models: Principles,Taxonomy, Challenges, and Open Questions (opens in a new tab) (November 2023)
An RL Perspective on RLHF, Prompting, and Beyond (opens in a new tab) (October 2023)
Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation (opens in a new tab) (May 2023)
Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study (opens in a new tab) (May 2023)
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond (opens in a new tab) (April 2023)
Tool Learning with Foundation Models (opens in a new tab) (April 2023)
One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Era (opens in a new tab) (April 2023)
A Bibliometric Review of Large Language Models Research from 2017 to 2023 (opens in a new tab) (April 2023)
A Survey of Large Language Models (opens in a new tab) (April 2023)
Nature Language Reasoning, A Survey (opens in a new tab) (March 2023)
Augmented Language Models: a Survey (opens in a new tab) (February 2023)
A Survey for In-context Learning (opens in a new tab) (December 2022)
Towards Reasoning in Large Language Models: A Survey (opens in a new tab) (December 2022)
Reasoning with Language Model Prompting: A Survey (opens in a new tab) (December 2022)
Emergent Abilities of Large Language Models (opens in a new tab) (June 2022)
A Taxonomy of Prompt Modifiers for Text-To-Image Generation (opens in a new tab) (April 2022)
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (opens in a new tab) (July 2021)

Approaches

Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4 (opens in a new tab) (December 2023)
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading (opens in a new tab) (October 2023)
Large Language Models as Analogical Reasoners (opens in a new tab) (October 2023)
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models (opens in a new tab) (October 2023)
Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL (opens in a new tab) (September 2023)
Chain-of-Verification Reduces Hallucination in Large Language Models (opens in a new tab) (September 2023)
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers (opens in a new tab) (September 2023)
From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting (opens in a new tab) (September 2023)
Re-Reading Improves Reasoning in Language Models (opens in a new tab) (September 2023)
Graph of Thoughts: Solving Elaborate Problems with Large Language Models (opens in a new tab) (August 2023)
Skeleton-of-Thought: Large Language Models Can Do Parallel Decoding (opens in a new tab) (July 2023)
Focused Prefix Tuning for Controllable Text Generation (opens in a new tab) (June 2023)
Exploring Lottery Prompts for Pre-trained Language Models (opens in a new tab) (May 2023)
Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses (opens in a new tab) (May 2023)
Let's Verify Step by Step (opens in a new tab) (May 2023)
Universality and Limitations of Prompt Tuning (opens in a new tab) (May 2023)
MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Prompting (opens in a new tab) (May 2023)
PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents (opens in a new tab) (May 2023)
Reasoning with Language Model is Planning with World Model (opens in a new tab) (May 2023)
Self-Critique Prompting with Large Language Models for Inductive Instructions (opens in a new tab) (May 2023)
Better Zero-Shot Reasoning with Self-Adaptive Prompting (opens in a new tab) (May 2023)
Hierarchical Prompting Assists Large Language Model on Web Navigation (opens in a new tab) (May 2023)
Interactive Natural Language Processing (opens in a new tab) (May 2023)
Can We Edit Factual Knowledge by In-Context Learning? (opens in a new tab) (May 2023)
In-Context Learning of Large Language Models Explained as Kernel Regression (opens in a new tab) (May 2023)
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models (opens in a new tab) (May 2023)
Meta-in-context learning in large language models (opens in a new tab) (May 2023)
Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning with LLMs (opens in a new tab) (May 2023)
Post Hoc Explanations of Language Models Can Improve Language Models (opens in a new tab) (May 2023)
Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt (opens in a new tab) (May 2023)
TreePrompt: Learning to Compose Tree Prompts for Explainable Visual Grounding (opens in a new tab) (May 2023)
TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks (opens in a new tab) (May 2023)
Efficient Prompting via Dynamic In-Context Learning (opens in a new tab) (May 2023)
The Web Can Be Your Oyster for Improving Large Language Models (opens in a new tab) (May 2023)
Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency (opens in a new tab) (May 2023)
Tree of Thoughts: Deliberate Problem Solving with Large Language Models (opens in a new tab) (May 2023)
ZeroPrompt: Streaming Acoustic Encoders are Zero-Shot Masked LMs (opens in a new tab) (May 2023)
Chain-of-Symbol Prompting Elicits Planning in Large Langauge Models (opens in a new tab) (May 2023)
CooK: Empowering General-Purpose Language Models with Modular and Collaborative Knowledge (opens in a new tab) (May 2023)
What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning (opens in a new tab) (May 2023)
Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Sampling (opens in a new tab) (May 2023)
Satisfiability-Aided Language Models Using Declarative Prompting (opens in a new tab) (May 2023)
Pre-Training to Learn in Context (opens in a new tab) (May 2023)
Boosted Prompt Ensembles for Large Language Models (opens in a new tab) (April 2023)
Global Prompt Cell: A Portable Control Module for Effective Prompt (opens in a new tab) (April 2023)
Why think step-by-step? Reasoning emerges from the locality of experience (opens in a new tab) (April 2023)
Revisiting Automated Prompting: Are We Actually Doing Better? (opens in a new tab) (April 2023)
REFINER: Reasoning Feedback on Intermediate Representations (opens in a new tab) (April 2023)
Reflexion: an autonomous agent with dynamic memory and self-reflection (opens in a new tab) (March 2023)
CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society (opens in a new tab) (March 2023)
Self-Refine: Iterative Refinement with Self-Feedback (opens in a new tab) (March 2023)
kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference (opens in a new tab) (March 2023)
Visual-Language Prompt Tuning with Knowledge-guided Context Optimization (opens in a new tab) (March 2023)
Fairness-guided Few-shot Prompting for Large Language Models (opens in a new tab) (March 2023)
Context-faithful Prompting for Large Language Models (opens in a new tab) (March 2023)
Is Prompt All You Need? No. A Comprehensive and Broader View of Instruction Learning (opens in a new tab) (March 2023)
UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation (opens in a new tab) (March 2023)
Model-tuning Via Prompts Makes NLP Models Adversarially Robust (opens in a new tab) (March 2023)
Structure Pretraining and Prompt Tuning for Knowledge Graph Transfer (opens in a new tab) (March 2023)
CoTEVer: Chain of Thought Prompting Annotation Toolkit for Explanation Verification (opens in a new tab) (March 2023)
Larger language models do in-context learning differently (opens in a new tab) (March 2023)
OpenICL: An Open-Source Framework for In-context Learning (opens in a new tab) (March 2023)
Dynamic Prompting: A Unified Framework for Prompt Tuning (opens in a new tab) (March 2023)
ART: Automatic multi-step reasoning and tool-use for large language models (opens in a new tab) (March 2023)
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning (opens in a new tab) (March 2023)
Effectiveness of Data Augmentation for Prefix Tuning with Limited Data (opens in a new tab) (March 2023)
Mixture of Soft Prompts for Controllable Data Generation (opens in a new tab) (March 2023)
Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners (opens in a new tab) (March 2023)
How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks (opens in a new tab) (March 2023)
Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT (opens in a new tab) (February 2023)
EvoPrompting: Language Models for Code-Level Neural Architecture Search (opens in a new tab) (February 2023)
In-Context Instruction Learning (opens in a new tab) (February 2023)
Chain of Hindsight Aligns Language Models with Feedback (opens in a new tab) (February 2023)
Language Is Not All You Need: Aligning Perception with Language Models (opens in a new tab) (February 2023)
Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data (opens in a new tab) (February 2023)
Active Prompting with Chain-of-Thought for Large Language Models (opens in a new tab) (February 2023)
More than you've asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models (opens in a new tab) (February 2023)
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT (opens in a new tab) (February 2023)
Guiding Large Language Models via Directional Stimulus Prompting (opens in a new tab) (February 2023)
How Does In-Context Learning Help Prompt Tuning? (opens in a new tab) (February 2023)
Scalable Prompt Generation for Semi-supervised Learning with Language Models (opens in a new tab) (February 2023)
Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints (opens in a new tab) (February 2023)
À-la-carte Prompt Tuning (APT): Combining Distinct Data Via Composable Prompting (opens in a new tab) (February 2023)
GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks (opens in a new tab) (February 2023)
The Capacity for Moral Self-Correction in Large Language Models (opens in a new tab) (February 2023)
SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domains (opens in a new tab) (February 2023)
Evaluating the Robustness of Discrete Prompts (opens in a new tab) (February 2023)
Compositional Exemplars for In-context Learning (opens in a new tab) (February 2023)
Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery (opens in a new tab) (February 2023)
Multimodal Chain-of-Thought Reasoning in Language Models (opens in a new tab) (February 2023)
Large Language Models Can Be Easily Distracted by Irrelevant Context (opens in a new tab) (February 2023)
Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models (opens in a new tab) (February 2023)
Progressive Prompts: Continual Learning for Language Models (opens in a new tab) (January 2023)
Batch Prompting: Efficient Inference with LLM APIs (opens in a new tab) (January 2023)
Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP (opens in a new tab) (December 2022)
On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning (opens in a new tab) (December 2022)
Constitutional AI: Harmlessness from AI Feedback (opens in a new tab) (December 2022)
Successive Prompting for Decomposing Complex Questions (opens in a new tab) (December 2022)
Large Language Models are reasoners with Self-Verification (opens in a new tab) (December 2022)
Discovering Language Model Behaviors with Model-Written Evaluations (opens in a new tab) (December 2022)
Structured Prompting: Scaling In-Context Learning to 1,000 Examples (opens in a new tab) (December 2022)
PAL: Program-aided Language Models (opens in a new tab) (November 2022)
Large Language Models Are Human-Level Prompt Engineers (opens in a new tab) (November 2022)
Ignore Previous Prompt: Attack Techniques For Language Models (opens in a new tab) (November 2022)
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods (opens in a new tab) (November 2022)
Teaching Algorithmic Reasoning via In-context Learning (opens in a new tab) (November 2022)
Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference (opens in a new tab) (November 2022)
Ask Me Anything: A simple strategy for prompting language models (opens in a new tab) (October 2022)
Recitation-Augmented Language Models (opens in a new tab) (October 2022)
ReAct: Synergizing Reasoning and Acting in Language Models (opens in a new tab) (October 2022)
Prompting GPT-3 To Be Reliable (opens in a new tab) (October 2022)
Decomposed Prompting: A Modular Approach for Solving Complex Tasks (opens in a new tab) (October 2022)
Automatic Chain of Thought Prompting in Large Language Models (opens in a new tab) (October 2022)
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought (opens in a new tab) (October 2022)
Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examples (opens in a new tab) (September 2022)
Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning (opens in a new tab) (September 2022)
Promptagator: Few-shot Dense Retrieval From 8 Examples (opens in a new tab) (September 2022)
Atlas: Few-shot Learning with Retrieval Augmented Language Models (opens in a new tab) (November 2022)
DocPrompting: Generating Code by Retrieving the Docs (opens in a new tab) (July 2022)
On the Advance of Making Language Models Better Reasoners (opens in a new tab) (June 2022)
Large Language Models are Zero-Shot Reasoners (opens in a new tab) (May 2022)
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations (opens in a new tab) (May 2022)
MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning (opens in a new tab) (May 2022)
PPT: Pre-trained Prompt Tuning for Few-shot Learning (opens in a new tab) (Mqy 2022)
Toxicity Detection with Generative Prompt-based Inference (opens in a new tab) (May 2022)
Learning to Transfer Prompts for Text Generation (opens in a new tab) (May 2022)
The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning (opens in a new tab) (May 2022)
A Taxonomy of Prompt Modifiers for Text-To-Image Generation (opens in a new tab) (April 2022)
PromptChainer: Chaining Large Language Model Prompts through Visual Programming (opens in a new tab) (March 2022)
Self-Consistency Improves Chain of Thought Reasoning in Language Models (opens in a new tab) (March 2022)
Training language models to follow instructions with human feedback (opens in a new tab)
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (opens in a new tab) (February 2022)
Chain of Thought Prompting Elicits Reasoning in Large Language Models (opens in a new tab) (January 2022)
Show Your Work: Scratchpads for Intermediate Computation with Language Models (opens in a new tab) (November 2021)
AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts (opens in a new tab) (October 2021)
Generated Knowledge Prompting for Commonsense Reasoning (opens in a new tab) (October 2021)
Multitask Prompted Training Enables Zero-Shot Task Generalization (opens in a new tab) (October 2021)
Reframing Instructional Prompts to GPTk's Language (opens in a new tab) (September 2021)
Design Guidelines for Prompt Engineering Text-to-Image Generative Models (opens in a new tab) (September 2021)
Making Pre-trained Language Models Better Few-shot Learners (opens in a new tab) (August 2021)
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity (opens in a new tab) (April 2021)
BERTese: Learning to Speak to BERT (opens in a new tab) (April 2021)
The Power of Scale for Parameter-Efficient Prompt Tuning (opens in a new tab) (April 2021)
Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm (opens in a new tab) (February 2021)
Calibrate Before Use: Improving Few-Shot Performance of Language Models (opens in a new tab) (February 2021)
Prefix-Tuning: Optimizing Continuous Prompts for Generation (opens in a new tab) (January 2021)
Learning to Generate Task-Specific Adapters from Task Description (opens in a new tab) (January 2021)
Making Pre-trained Language Models Better Few-shot Learners (opens in a new tab) (December 2020)
Learning from Task Descriptions (opens in a new tab) (November 2020)
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts (opens in a new tab) (October 2020)
Language Models are Few-Shot Learners (opens in a new tab) (May 2020)
How Can We Know What Language Models Know? (opens in a new tab) (July 2020)
Scaling Laws for Neural Language Models (opens in a new tab) (January 2020)

Applications

Collections

What is Groq?Tools