Prompt engineering is a concept in

artificial intelligence Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machine A machine is a physical system using Power (physics), power to apply Force, forces and control Motion, moveme ...

, particularly

natural language processing Natural language processing (NLP) is an interdisciplinary subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to proc ...

(NLP). In prompt engineering, the description of the task is embedded in the input, e.g., as a question instead of it being implicitly given. Prompt engineering typically works by converting one or more tasks to a prompt-based dataset and training a

language model A language model is a probability distribution over sequences of words. Given any sequence of words of length , a language model assigns a probability P(w_1,\ldots,w_m) to the whole sequence. Language models generate probabilities by training on ...

with what has been called "prompt-based learning" or just "prompt learning". Prompt engineering may work from a large "frozen" pretrained language model and where only the representation of the prompt is learned (i.e., optimized), using methods such as "prefix-tuning" or "prompt tuning". The GPT-2 and

GPT-3 Generative Pre-trained Transformer 3 (GPT-3) is an autoregressive language model that uses deep learning to produce human-like text. Given an initial text as prompt, it will produce text that continues the prompt. The architecture is a standa ...

language models were important steps in prompt engineering. In 2021, multitask prompt engineering using multiple NLP datasets showed good performance on new tasks. Prompts that include a chain of thought in few-shot learning examples show better indication of

reasoning Reason is the capacity of consciously applying logic by drawing conclusions from new or existing information, with the aim of seeking the truth. It is closely associated with such characteristically human activities as philosophy, science, lang ...

in language models. In

zero-shot learning Zero-shot learning (ZSL) is a problem setup in machine learning where, at test time, a learner observes samples from classes which were not observed during training, and needs to predict the class that they belong to. Zero-shot methods generally w ...

prepending text to the prompt that encourages a chain of thought (e.g. "Let's think step by step") may improve the performance of a language model in multi-step reasoning problems. The broad accessibility of these tools were driven by the publication of several open-source notebooks and community-led projects for image synthesis. A description for handling prompts reported that over 2,000 public prompts for around 170 datasets were available in February 2022. In 2022, machine learning models like DALL-E 2, Stable Diffusion, and Midjourney were released to the public. These models take text prompts as input and use them to generate images, which introduced a new category of prompt engineering related to

text-to-image A text-to-image model is a machine learning model which takes as input a natural language description and produces an image matching that description. Such models began to be developed in the mid-2010s, as a result of advances in deep neural netwo ...

prompting.

Malicious

''Prompt injection'' is a family of related

computer security exploit An exploit (from the English verb ''to exploit'', meaning "to use something to one’s own advantage") is a piece of software, a chunk of data, or a sequence of commands that takes advantage of a bug or vulnerability to cause unintended or unan ...

s carried out by getting machine learning models (such as large language model) which were trained to follow human-given instructions to follow instructions provided by a malicious user, which stands in contrast to the intended operation of instruction-following systems, wherein the ML model is intended only to follow trusted instructions (prompt) provided by the ML model's operator. Prompt injection can be viewed as a code injection attack using adversarial prompt engineering. In 2022, the

NCC Group NCC Group (LSE: NCC) is an information assurance firm headquartered in Manchester, United Kingdom. Its service areas cover software escrow and verification, cyber security consulting and managed services. NCC Group claims over 15,000 clients worldw ...

has characterized prompt injection as a new class of vulnerability of AI/ML systems. Around 2023, prompt injection was seen "in the wild" in minor exploits against ChatGPT and similar chatbots, for example to reveal the hidden initial prompts of the systems, or to trick the chatbot into participating in conversations that violate the chatbot's content policy.

References

{{Scholia, topic Machine learning Natural language processing