Visual inputs

GPT-4 can accept a prompt of text and images, which—parallel to the text-only setting—lets the user specify any vision or language task. Specifically, it generates text outputs (natural language, code, etc.) given inputs consisting of interspersed text and images. Over a range of domains—including documents with text and photographs, diagrams, or screenshots—GPT-4 exhibits similar capabilities as it does on text-only inputs. Furthermore, it can be augmented with test-time techniques that were developed for text-only language models, including few-shot and chain-of-thought prompting. Image inputs are still a research preview and not publicly available.


Limitations

Interestingly, the base pre-trained model is highly calibrated (its predicted confidence in an answer generally matches the probability of being correct) .  However, through our current post-training process, the calibration is reduced.


How Could AI Affect Influence Operations?

When researchers evaluate influence operations, they consider the actors, behaviors, and content. The widespread availability of technology powered by language models has the potential to impact all three facets:Actors: Language models could drive down the cost of running influence operations, placing them within reach of new actors and actor types. Likewise, propagandists-for-hire that automate production of text may gain new competitive advantages.Behavior: Influence operations with language models will become easier to scale, and tactics that are currently expensive (e.g., generating personalized content) may become cheaper. Language models may also enable new tactics to emerge—like real-time content generation in chatbots.Content: Text creation tools powered by language models may generate more impactful or persuasive messaging compared to propagandists, especially those who lack requisite linguistic or cultural knowledge of their target. They may also make influence operations less discoverable, since they repeatedly create new content without needing to resort to copy-pasting and other noticeable time-saving behaviors.Our bottom-line judgment is that language models will be useful for propagandists and will likely transform online influence operations. Even if the most advanced models are kept private or controlled through application programming interface (API) access, propagandists will likely gravitate towards open-source alternatives and nation states may invest in the technology themselves.


Critical Unknowns

Many factors impact whether, and the extent to which, language models will be used in influence operations. Our report dives into many of these considerations. For example:What new capabilities for influence will emerge as a side effect of well-intentioned research or commercial investment? Which actors will make significant investments in language models?When will easy-to-use tools to generate text become publicly available? Will it be more effective to engineer specific language models for influence operations, rather than apply generic ones?Will norms develop that disincentivize actors who wage AI-enabled influence operations? How will actor intentions develop?While we expect to see diffusion of the technology as well as improvements in the usability, reliability, and efficiency of language models, many questions about the future remain unanswered. Because these are critical possibilities that can change how language models may impact influence operations, additional research to reduce uncertainty is highly valuable.


A Framework for Mitigations

To chart a path forward, the report lays out key stages in the language model-to-influence operation pipeline. Each of these stages is a point for potential mitigations.To successfully wage an influence operation leveraging a language model, propagandists would require that: (1) a model exists, (2) they can reliably access it, (3) they can disseminate content from the model, and (4) an end user is affected. Many possible mitigation strategies fall along these four steps, as shown below. Stage in the pipeline 1. Model Construction 2. Model Access 3. Content Dissemination 4. Belief Formation Illustrative Mitigations AI developers build models that are more fact-sensitive. AI providers impose stricter usage restrictions on language models. Platforms and AI providers coordinate to identify AI content. Institutions engage in media literacy campaigns. Developers spread radioactive data to make generative models detectable. AI providers develop new norms around model release. Platforms require 'proof of personhood' to post. Developers provide consumer focused AI tools. Governments impose restrictions on data collection. AI providers close security vulnerabilities. Entities that rely on public input take steps to reduce their exposure to misleading AI content. Governments impose access controls on AI hardware. Digital provenance standards are widely adopted.


If a Mitigation Exists, is it Desirable?

Just because a mitigation could reduce the threat of AI-enabled influence operations does not mean that it should be put into place. Some mitigations carry their own downside risks. Others may not be feasible. While we do not explicitly endorse or rate mitigations, the paper provides a set of guiding questions for policymakers and others to consider:Technical Feasibility: Is the proposed mitigation technically feasible? Does it require significant changes to technical infrastructure?Social Feasibility: Is the mitigation feasible from a political, legal, and institutional perspective? Does it require costly coordination, are key actors incentivized to implement it, and is it actionable under existing law, regulation, and industry standards?Downside Risk: What are the potential negative impacts of the mitigation, and how significant are they?Impact: How effective would a proposed mitigation be at reducing the threat?We hope this framework will spur ideas for other mitigation strategies, and that the guiding questions will help relevant institutions begin to consider whether various mitigations are worth pursuing.This report is far from the final word on AI and the future of influence operations. Our aim is to define the present environment and to help set an agenda for future research. We encourage anyone interested in collaborating or discussing relevant projects to connect with us. For more, read the full report here.


Reducing graphic and explicit training data

Since training data shapes the capabilities of any learned model, data filtering is a powerful tool for limiting undesirable model capabilities.  We applied this approach to two categories—images depicting graphic violence and sexual content—by using classifiers to filter images in these categories out of the dataset before training DALL·E 2.  We trained these image classifiers in-house and are continuing to study the effects of dataset filtering on our trained model.To train our image classifiers, we reused an approach that we had previously employed to filter training data for GLIDE.  The basic steps to this approach are as follows: first, we create a specification for the image categories we would like to label; second, we gather a few hundred positive and negative examples for each category; third, we use an active learning procedure to gather more data and improve the precision/recall trade-off; and finally, we run the resulting classifier on the entire dataset with a conservative classification threshold to favor recall over precision.  To set these thresholds, we prioritized filtering out all of the bad data over leaving in all of the good data.  This is because we can always fine-tune our model with more data later to teach it new things, but it's much harder to make the model forget something that it has already learned.We start with a small dataset of labeled images (top of figure).  We then train a classifier on this data.  The active learning process then uses the current classifier to select a handful of unlabeled images that are likely to improve classifier performance.  Finally, humans produce labels for these images, adding them to the labeled dataset.  The process can be repeated to iteratively improve the classifier's performance.During the active learning phase, we iteratively improved our classifiers by gathering human labels for potentially difficult or misclassified images.  Notably, we used two active learning techniques to choose images from our dataset (which contains hundreds of millions of unlabeled images) to present to humans for labeling.  First, to reduce our classifier's false positive rate (i.e., the frequency with which it misclassifies a benign image as violent or sexual), we assigned human labels to images that the current model classified as positive.  For this step to work well, we tuned our classification threshold for nearly 100% recall but a high false-positive rate; this way, our labelers were mostly labeling truly negative cases.  While this technique helps to reduce false positives and reduces the need for labelers to look at potentially harmful images, it does not help find more positive cases that the model is currently missing.To reduce our classifier's false negative rate, we employed a second active learning technique: nearest neighbor search.  In particular, we ran many-fold cross-validation to find positive samples in our current labeled dataset which the model tended to misclassify as negative (to do this, we literally trained hundreds of versions of the classifier with different train-validation splits)


We then scanned our large collection of unlabeled images for nearest neighbors of these samples in a perceptual feature space, and assigned human labels to the discovered images .  Thanks to our compute infrastructure, it was trivial to scale up both classifier training and nearest neighbor search to many GPUs, allowing the active learning step to take place over a number of minutes rather than hours or days.To verify the effectiveness of our data filters, we trained two GLIDE models with the same hyperparameters: one on unfiltered data, and one on the dataset after filtering.  We refer to the former model as the unfiltered model, and the latter as the filtered model.  As expected, we found that the filtered model generally produced less explicit or graphic content in response to requests for this kind of content.  However, we also found an unexpected side-effect of data filtering: it created or amplified the model's biases towards certain demographics.UnfilteredFiltered


Fixing bias introduced by data filters

Generative models attempt to match the distribution of their training data, including any biases therein.  As a result, filtering the training data has the potential to create or amplify biases in downstream models.  In general, fixing biases in the original dataset is a difficult sociotechnical task that we continue to study, and is beyond the scope of this post.  The problem we address here is the amplification of biases caused specifically by data filtering itself.  With our approach, we aim to prevent the filtered model from being more biased than the unfiltered model, essentially reducing the distribution shift caused by data filtering.As a concrete example of bias amplification due to filtering, consider the prompt 'a ceo'.  When our unfiltered model generated images for this prompt, it tended to produce more images of men than women, and we expect that most of this bias is a reflection of our current training data.  However, when we ran the same prompt through our filtered model, the bias appeared to be amplified; the generations were almost exclusively images of men.We hypothesize that this particular case of bias amplification comes from two places: first, even if women and men have roughly equal representation in the original dataset, the dataset may be biased toward presenting women in more sexualized contexts; and second, our classifiers themselves may be biased either due to implementation or class definition, despite our efforts to ensure that this was not the case during the data collection and validation phases.  Due to both of these effects, our filter may remove more images of women than men, which changes the gender ratio that the model observes in training.To investigate filter-induced bias more thoroughly, we wanted a way to measure how much our data filters were affecting the bias towards various concepts.  Notably, our violence and sexual content filters are purely image-based, but the multimodal nature of our dataset allows us to directly measure the effects of these filters on text.  Since every image is accompanied by a text caption, we were able to look at the relative frequency of hand-selected keywords across the filtered and unfiltered dataset to estimate how much the filters were affecting any given concept.To put this into practice, we used Apache Spark to compute the frequencies of a handful of keywords (e.g., 'parent', 'woman', 'kid') over all of the captions in both our filtered and unfiltered datasets.  Even though our dataset contains hundreds of millions of text-image pairs, computing these keyword frequencies only took a few minutes using our compute cluster.After computing keyword frequencies, we were able to confirm that our dataset filters had indeed skewed the frequencies of certain keywords more than others.  For example, the filters reduced the frequency of the word 'woman' by 14%, while the frequency of the word 'man' was only reduced by 6%.  This confirmed, on a large scale, what we had already observed anecdotally by sampling from GLIDE models trained on both datasets.An illustration of dataset reweighting


Fixing bias introduced by data filters

This can be derived using the definition of the sigmoid:1/(1+e−f(x))/(1−1/(1+e−f(x))) 1/(1+e^−f(x))/(1−1/(1+e^−f(x))) 1/(1+e−f(x))/(1−1/(1+e−f(x)))=1/(1+e−f(x))/((1+e−f(x)−1)/(1+e−f(x))) = 1/(1+e^{-f(x)}) / ((1+e^{-f(x)} - 1)/(1+e^{-f(x)})) =1/(1+e−f(x))/((1+e−f(x)−1)/(1+e−f(x)))=1/(1+e−f(x))/((e−f(x))/(1+e−f(x))) = 1/(1+e^{-f(x)}) / ((e^{-f(x)})/(1+e^{-f(x)})) =1/(1+e−f(x))/((e−f(x))/(1+e−f(x)))=(1+e−f(x))/(1+e−f(x))/(e−f(x)) = (1+e^-f(x))/(1+e^-f(x)) / (e^-f(x)) =(1+e−f(x))/(1+e−f(x))/(e−f(x))=1/(e−f(x))=ef(x) = 1 / (e^{-f(x)}) = e^{f(x)} =1/(e−f(x))=ef(x)How well does this reweighting scheme actually mitigate the amplified bias? When we fine-tuned our previous filtered model with the new weighting scheme, the fine-tuned model's behavior much more closely matched the unfiltered model on the biased examples we had previously found .  While this was encouraging, we also wanted to evaluate this mitigation more thoroughly using our keyword-based bias heuristic.  To measure keyword frequencies while taking our new weighting scheme into account, we can simply weight every instance of a keyword in the filtered dataset by the weight of the sample that contains it.  Doing this, we get a new set of keyword frequencies that reflect the sample weights in the filtered dataset.Across most of the keywords we checked, the reweighting scheme reduced the frequency change induced by filtering.  For our previous examples of 'man' and 'woman', the relative frequency reductions became 1% and –1%, whereas their previous values were 14% and 6%, respectively.  While this metric is just a proxy for actual filtering bias, it is reassuring that our image-based reweighting scheme actually improves a text-based metric so significantly.We are continuing to investigate remaining biases in DALL·E 2, in part through larger evaluations of the model's behavior and investigations of how filtering impacted bias and capability development.


Fixing bias introduced by data filters

We start with a balanced dataset (left) .  If our filter affects one category more than another, it can create a biased dataset (middle).  Using reweighting, we effectively 'repeat' some data more than others, allowing us to rebalance the bias caused by the filters (right).Now that we had a proxy for measuring filter-induced bias, we needed a way to mitigate it.  To tackle this problem, we aimed to re-weight the filtered dataset so that its distribution better matched the distribution of unfiltered images.  As a toy example to illustrate this idea, suppose our dataset consists of 50% cat photos and 50% dog photos, but our data filters remove 75% of dogs but only 50% of cats.  The final dataset would be ⅔ cats and ⅓ dogs, and a likelihood-based generative model trained on this dataset would likely generate more images of cats than dogs.  We can fix this imbalance by multiplying the training loss of every image of a dog by 2, emulating the effect of repeating every dog image twice.  It turns out that we can scale this approach to our real datasets and models in a way that is largely automatic–that is, we needn't hand-select the features that we want to reweight.We compute weights for images in the filtered dataset using probabilities from a special classifier, similar to the approach used by Choi et al.  (2019).  To train this classifier, we uniformly sample images from both datasets and predict which dataset the image came from.  In particular, this model predicts P(unfiltered|image), given a prior P(unfiltered) = 0.5.  In practice, we don't want this model to be too powerful, or else it might learn the exact function implemented by our filters in the first place.  Instead, we want the model to be smoother than our original data filters, capturing broad categories that are affected by the filters while still being unsure about whether a particular image would be filtered or not.  To this end, we trained a linear probe on top of a small CLIP model.Once we have a classifier which predicts the probability that an image is from the unfiltered dataset, we still need to convert this prediction into a weight for the image.  For example, suppose that P(unfiltered|image) = 0.8.  This means that the sample is 4 times more likely to be found in the unfiltered data than the filtered data, and a weight of 4 should correct the imbalance.  More generally, we can use the weight P(unfiltered|image)/P(filtered|image).A[A]When we parametrize P(unfiltered|image) as sigmoid(f(x)), the weight is then exp(f(x))


Preventing image regurgitation

We observed that our internal predecessors to DALL·E 2 would sometimes reproduce training images verbatim.  This behavior was undesirable, since we would like DALL·E 2 to create original, unique images by default and not just 'stitch together' pieces of existing images.  Additionally, reproducing training images verbatim can raise legal questions around copyright infringement, ownership, and privacy (if people's photos were present in training data).To better understand the issue of image regurgitation, we collected a dataset of prompts that frequently resulted in duplicated images.  To do this, we used a trained model to sample images for 50,000 prompts from our training dataset, and sorted the samples by perceptual similarity to the corresponding training image.  Finally, we inspected the top matches by hand, finding only a few hundred true duplicate pairs out of the 50k total prompts.  Even though the regurgitation rate appeared to be less than 1%, we felt it was necessary to push the rate down to 0 for the reasons stated above.When we studied our dataset of regurgitated images, we noticed two patterns.  First, the images were almost all simple vector graphics, which were likely easy to memorize due to their low information content.  Second, and more importantly, the images all had many near-duplicates in the training dataset.  For example, there might be a vector graphic which looks like a clock showing the time 1 o'clock—but then we would discover a training sample containing the same clock showing 2 o'clock, and then 3 o'clock, etc.  Once we realized this, we used a distributed nearest neighbor search to verify that, indeed, all of the regurgitated images had perceptually similar duplicates in the dataset. Other works have observed a similar phenomenon in large language models, finding that data duplication is strongly linked to memorization.The above finding suggested that, if we deduplicated our dataset, we might solve the regurgitation problem.  To achieve this, we planned to use a neural network to identify groups of images that looked similar, and then remove all but one image from each group.B[B]To achieve this, we can compute a feature vector viv_ivi for every training image iii, and then remove all images jjj such that there exists an i<ji < ji<j where ∣∣vi−vj∣∣||v_i - v_j||∣∣vi−vj∣∣ <threshold.  To solve this problem naively, we would need to compute every pairwise distance ∣∣vi−vj∣∣||v_i - v_j||∣∣vi−vj∣∣, a task that scales quadratically with the size of our dataset.However, this would require checking, for each image, whether it is a duplicate of every other image in the dataset.  Since our whole dataset contains hundreds of millions of images, we would naively need to check hundreds of quadrillions of image pairs to find all the duplicates.  While this is technically within reach, especially on a large compute cluster, we found a much more efficient alternative that works almost as well at a small fraction of the cost.Consider what happens if we cluster our dataset before performing deduplication


Preventing image regurgitation

Since nearby samples often fall into the same cluster, most of the duplicate pairs would not cross cluster decision boundaries .  We could then deduplicate samples within each cluster without checking for duplicates outside of the cluster, while only missing a small fraction of all duplicate pairs.  This is much faster than the naive approach, since we no longer have to check every single pair of images.C[C]Letting K K K represent the number of clusters and N N N the dataset size, this approach only requires O(K∗(N/K)2)=O(N2/K) O(K*(N/K)^2) = O(N^2/K) O(K∗(N/K)2)=O(N2/K) pairwise distance calculations, rather than the full O(N2) O(N^2) O(N2).  Meanwhile, we are still guaranteed that no image will have more than K K K near-duplicates in the worst possible case.When we tested this approach empirically on a small subset of our data, it found 85% of all duplicate pairs when usingK=1024 clusters.To improve the success rate of the above algorithm, we leveraged one key observation: when you cluster different random subsets of a dataset, the resulting cluster decision boundaries are often quite different.  Therefore, if a duplicate pair crosses a cluster boundary for one clustering of the data, the same pair might fall inside a single cluster in a different clustering.  The more clusterings you try, the more likely you are to discover a given duplicate pair.  In practice, we settled on using five clusterings, which means that we search for duplicates of each image in the union of five different clusters.  In practice, this found 97% of all duplicate pairs on a subset of our data.Surprisingly, almost a quarter of our dataset was removed by deduplication.  When we looked at the near-duplicate pairs that were found, many of them included meaningful changes.  Recall the clock example from above: the dataset might include many images of the same clock at different times of day.  While these images are likely to make the model memorize this particular clock's appearance, they might also help the model learn to distinguish between times of day on a clock.  Given how much data was removed, we were worried that removing images like this might have hurt the model's performance.To test the effect of deduplication on our models, we trained two models with identical hyperparameters: one on the full dataset, and one on the deduplicated version of the dataset.  To compare the models, we used the same human evaluations we used to evaluate our original GLIDE model.  Surprisingly, we found that human evaluators slightly preferred the model trained on deduplicated data, suggesting that the large amount of redundant images in the dataset was actually hurting performance.Once we had a model trained on deduplicated data, we reran the regurgitation search we had previously done over 50k prompts from the training dataset.  We found that the new model never regurgitated a training image when given the exact prompt for the image from the training dataset


Next steps

While all of the mitigations discussed above represent significant progress towards our goal of reducing the risks associated with DALL·E 2, each mitigation still has room to improve:Better pre-training filters could allow us to train DALL·E 2 on more data and potentially further reduce bias in the model. Our current filters are tuned for a low miss-rate at the cost of many false positives. As a result, we filtered out roughly 5% of our entire dataset even though most of these filtered images do not violate our content policy at all. Improving our filters could allow us to reclaim some of this training data.Bias is introduced and potentially amplified at many stages of system development and deployment. Evaluating and mitigating the bias in systems like DALL·E 2 and the harm induced by this bias is an important interdisciplinary problem that we continue to study at OpenAI as part of our broader mission. Our work on this includes building evaluations to better understand the problem, curating new datasets, and applying techniques like human feedback and fine-tuning to build more robust and representative technologies.It is also crucial that we continue to study memorization and generalization in deep learning systems. While deduplication is a good first step towards preventing memorization, it does not tell us everything there is to learn about why or how models like DALL·E 2 memorize training data.


Preventing image regurgitation

To take this test another step further, we also performed a nearest neighbor search over the entire training dataset for each of the 50k generated images .  This way, we thought we might catch the model regurgitating a different image than the one associated with a given prompt.  Even with this more thorough check, we never found a case of image regurgitation.


VPT zero-shot results

We chose to validate our method in Minecraft because it (1) is one of the most actively played video games in the world and thus has a wealth of freely available video data and (2) is open-ended with a wide variety of things to do, similar to real-world applications such as computer usage. Unlike prior works in Minecraft that use simplified action spaces aimed at easing exploration, our AI uses the much more generally applicable, though also much more difficult, native human interface: 20Hz frame rate with the mouse and keyboard.Trained on 70,000 hours of IDM-labeled online video, our behavioral cloning model (the 'VPT foundation model') accomplishes tasks in Minecraft that are nearly impossible to achieve with reinforcement learning from scratch. It learns to chop down trees to collect logs, craft those logs into planks, and then craft those planks into a crafting table; this sequence takes a human proficient in Minecraft approximately 50 seconds or 1,000 consecutive game actions. Sequence of items required to craft a crafting table, labeled with the median time it takes proficient humans to reach each step Additionally, the model performs other complex skills humans often do in the game, such as swimming, hunting animals for food, and eating that food. It also learned the skill of 'pillar jumping', a common behavior in Minecraft of elevating yourself by repeatedly jumping and placing a block underneath yourself.


Fine-tuning with behavioral cloning

Foundation models are designed to have a broad behavior profile and be generally capable across a wide variety of tasks. To incorporate new knowledge or allow them to specialize on a narrower task distribution, it is common practice to fine-tune these models to smaller, more specific datasets. As a case study into how well the VPT foundation model can be fine-tuned to downstream datasets, we asked our contractors to play for 10 minutes in brand new Minecraft worlds and build a house from basic Minecraft materials. We hoped that this would amplify the foundation model's ability to reliably perform 'early game' skills such as building crafting tables. When fine-tuning to this dataset, not only do we see a massive improvement in reliably performing the early game skills already present in the foundation model, but the fine-tuned model also learns to go even deeper into the technology tree by crafting both wooden and stone tools. Sometimes we even see some rudimentary shelter construction and the agent searching through villages, including raiding chests. Sequence of items required to craft a stone pickaxe, labeled with the median time it takes proficient humans to reach each step Improved early game behavior from BC fine-tuning


Fine-tuning with reinforcement learning

When it is possible to specify a reward function, reinforcement learning (RL) can be a powerful method for eliciting high, potentially even super-human, performance. However, many tasks require overcoming hard exploration challenges, and most RL methods tackle these with random exploration priors, e.g. models are often incentivized to act randomly via entropy bonuses. The VPT model should be a much better prior for RL because emulating human behavior is likely much more helpful than taking random actions. We set our model the challenging task of collecting a diamond pickaxe, an unprecedented capability in Minecraft made all the more difficult when using the native human interface.Crafting a diamond pickaxe requires a long and complicated sequence of subtasks. To make this task tractable, we reward agents for each item in the sequence. Sequence of items required to craft a stone pickaxe, labeled with the median time it takes proficient humans to reach each step We found that an RL policy trained from a random initialization (the standard RL method) barely achieves any reward, never learning to collect logs and only rarely collecting sticks. In stark contrast, fine-tuning from a VPT model not only learns to craft diamond pickaxes (which it does in 2.5% of 10-minute Minecraft episodes), but it even has a human-level success rate at collecting all items leading up to the diamond pickaxe. This is the first time anyone has shown a computer agent capable of crafting diamond tools in Minecraft, which takes humans over 20 minutes (24,000 actions) on average.Reward over episodes


Conclusion

VPT paves the path toward allowing agents to learn to act by watching the vast numbers of videos on the internet. Compared to generative video modeling or contrastive methods that would only yield representational priors, VPT offers the exciting possibility of directly learning large scale behavioral priors in more domains than just language. While we only experiment in Minecraft, the game is very open-ended and the native human interface (mouse and keyboard) is very generic, so we believe our results bode well for other similar domains, e.g. computer usage.For more information, please see our paper. We are also open sourcing our contractor data, Minecraft environment, model code, and model weights, which we hope will aid future research into VPT. Furthermore, we have partnered with the MineRL NeurIPS competition this year. Contestants can use and fine-tune our models to try to solve many difficult tasks in Minecraft. Those interested can check out the competition webpage and compete for a blue-sky prize of $100,000 in addition to a regular prize pool of $20,000. Grants are available to self-identified underrepresented groups and individuals.


Data scaling

Perhaps the most important hypothesis of our work is that it is far more effective to use labeled contractor data to train an IDM (as part of the VPT pipeline) than it is to directly train a BC foundation model from that same small contractor dataset. To validate this hypothesis we train foundation models on increasing amounts of data from 1 to 70,000 hours. Those trained on under 2,000 hours of data are trained on the contractor data with ground-truth labels that were originally collected to train the IDM, and those trained on over 2,000 hours are trained on internet data labeled with our IDM. We then take each foundation model and fine-tune it to the house building dataset described in the previous section.Effect of foundation model training data on fine-tuningAs foundation model data increases, we generally see an increase in crafting ability, and only at the largest data scale do we see the emergence of stone tool crafting.


Experiments with AI assistance

We compare human ratings of AI-written summaries between a control group receiving no assistance and an assisted group who get to see 8 AI-written critiques. Summaries are picked from 3 different sources. Assisted humans find about 50% more flaws in summaries than unassisted raters, using model critiques directly for most of the critiques they find.To see how useful our models are for evaluation assistance, we show labelers 8 model-written critiques of each summary, with a control group that receives no assistance. We use topic-based summaries from three sources: written by our models, written by humans, and written by humans deliberately to have important yet subtle flaws.


Passage

What does the article say about the storm's effects on daily life?


Critiques

UnassistedAI-assisted


Human-written

Even though summarization isn't actually a difficult task for humans and our models aren't more capable than humans, they already provide meaningful assistance: when asked to evaluate model-written summaries, the assisted group finds 50% more flaws than the control group. For deliberately misleading summaries, assistance increases how often humans spot the intended flaw from 27% to 45%.


Scaling properties of critiques

Assistance on model-written summaries only works if they are able to critique themselves. We ask humans to rate the helpfulness of model-written self-critiques, and find larger models are better at self-critiquing.Larger models are better at self-critiquing in our topic-based summarization domain: Even though larger models have answers that are more difficult to critique, they generate more helpful critiques of their own outputs. In this plot, model scale is measured in log loss (nats) after fine-tuning. Helpfulness is determined by a human judging whether the model-generated critique of the model-generated answer is valid and useful for understanding summary quality. We filter for summaries that humans found a critique for.We also find that large models are able to directly improve their outputs, using their self-critiques, which small models are unable to do. Using better critiques helps models make better improvements than they do with worse critiques, or with no critiques.


Do models tell us everything they know?

To provide the best evaluation assistance on difficult tasks, we would like models to communicate all problems that they 'know about.' Whenever a model correctly predicts that an answer is flawed, can the model also produce a concrete critique that humans understand?This is particularly important for supervising models that could attempt to mislead human supervisors or hide information. We would like to train equally smart assistance models to point out what humans don't notice.Unfortunately, we found that models are better at discriminating than at critiquing their own answers, indicating they know about some problems that they can't or don't articulate. Furthermore, the gap between discrimination and critique ability did not appear to decrease for larger models. Reducing this gap is an important priority for our alignment research.


Next steps

An important limitation of this work is that topic-based summarization is not actually a difficult task: humans understand it quite well and it takes them only about 10 minutes to evaluate a summary. To understand the limits of AI-assisted evaluation better, we need to work with tasks that are much more difficult for humans to evaluate.Nevertheless, these results make us optimistic that we can train models to provide humans with meaningful feedback assistance. This is an important pillar of our alignment strategy, starting with the work on debate and recursive reward modeling. In the long run, we want to build assistants that can be trusted to take on all of the cognitive labor needed for evaluation, so humans can focus on communicating their preferences.If you're interested in this line of research, we're hiring Research Engineers and Research Scientists!


Summary

HumanHuman misleadingAI


No parallelism

Data ParallelismPipeline ParallelismTensor ParallelismExpert Parallelism An illustration of various parallelism strategies on a three-layer model. Each color refers to one layer and dashed lines separate different GPUs. Training a neural network is an iterative process. In every iteration, we do a pass forward through a model's layers to compute an output for each training example in a batch of data. Then another pass proceeds backward through the layers, propagating how much each parameter affects the final output by computing a gradient with respect to each parameter. The average gradient for the batch, the parameters, and some per-parameter optimization state is passed to an optimization algorithm, such as Adam, which computes the next iteration's parameters (which should have slightly better performance on your data) and new per-parameter optimization state. As the training iterates over batches of data, the model evolves to produce increasingly accurate outputs.Various parallelism techniques slice this training process across different dimensions, including:Data parallelism—run different subsets of the batch on different GPUs;Pipeline parallelism—run different layers of the model on different GPUs;Tensor parallelism—break up the math for a single operation such as a matrix multiplication to be split across GPUs;Mixture-of-Experts—process each example by only a fraction of each layer.(In this post, we'll assume that you are using GPUs to train your neural networks, but the same ideas apply to those using any other neural network accelerator.)


Pipeline parallelism

With Pipeline Parallel training, we partition sequential chunks of the model across GPUs. Each GPU holds only a fraction of parameters, and thus the same model consumes proportionally less memory per GPU.It's straightforward to split a large model into chunks of consecutive layers. However, there's a sequential dependency between inputs and outputs of layers, so a naive implementation can lead to a large amount of idle time while a worker waits for outputs from the previous machine to be used as its inputs. These waiting time chunks are known as 'bubbles,' wasting the computation that could be done by the idling machines.ForwardBackwardUpdateIdle Illustration of a naive pipeline parallelism setup where the model is vertically split into 4 partitions by layer. Worker 1 hosts model parameters of the first layer of the network (closest to the input), while worker 4 hosts layer 4 (which is closest to the output). 'F', 'B', and 'U' represent forward, backward and update operations, respectively. The subscripts indicate on which worker an operation runs. Data is processed by one worker at a time due to the sequential dependency, leading to large 'bubbles' of idle time. We can reuse the ideas from data parallelism to reduce the cost of the bubble by having each worker only process a subset of data elements at one time, allowing us to cleverly overlap new computation with wait time. The core idea is to split one batch into multiple microbatches; each microbatch should be proportionally faster to process and each worker begins working on the next microbatch as soon as it's available, thus expediting the pipeline execution. With enough microbatches the workers can be utilized most of the time with a minimal bubble at the beginning and end of the step. Gradients are averaged across microbatches, and updates to the parameters happen only once all microbatches have been completed.The number of workers that the model is split over is commonly known as pipeline depth.During the forward pass, workers only need to send the output (called activations) of its chunk of layers to the next worker; during the backward pass, it only sends the gradients on those activations to the previous worker. There's a big design space of how to schedule these passes and how to aggregate the gradients across microbatches. GPipe has each worker process forward and backward passes consecutively and then aggregates gradients from multiple microbatches synchronously at the end. PipeDream instead schedules each worker to alternatively process forward and backward passes.ForwardBackwardUpdateIdleGPipePipeDream Comparison of GPipe and PipeDream pipelining schemes, using 4 microbatches per batch. Microbatches 1-8 correspond to two consecutive data batches. In the image, '(number)' indicates on which microbatch an operation is performed and the subscript marks the worker ID. Note that PipeDream gets more efficiency by performing some computations with stale parameters.


PipeDream

Comparison of GPipe and PipeDream pipelining schemes, using 4 microbatches per batch. Microbatches 1-8 correspond to two consecutive data batches. In the image, '(number)' indicates on which microbatch an operation is performed and the subscript marks the worker ID. Note that PipeDream gets more efficiency by performing some computations with stale parameters.


Tensor parallelism

Pipeline parallelism splits a model 'vertically' by layer. It's also possible to 'horizontally' split certain operations within a layer, which is usually called Tensor Parallel training. For many modern models (such as the Transformer), the computation bottleneck is multiplying an activation batch matrix with a large weight matrix. Matrix multiplication can be thought of as dot products between pairs of rows and columns; it's possible to compute independent dot products on different GPUs, or to compute parts of each dot product on different GPUs and sum up the results. With either strategy, we can slice the weight matrix into even-sized 'shards', host each shard on a different GPU, and use that shard to compute the relevant part of the overall matrix product before later communicating to combine the results.One example is Megatron-LM, which parallelizes matrix multiplications within the Transformer's self-attention and MLP layers. PTD-P uses tensor, data, and pipeline parallelism; its pipeline schedule assigns multiple non-consecutive layers to each device, reducing bubble overhead at the cost of more network communication.Sometimes the input to the network can be parallelized across a dimension with a high degree of parallel computation relative to cross-communication. Sequence parallelism is one such idea, where an input sequence is split across time into multiple sub-examples, proportionally decreasing peak memory consumption by allowing the computation to proceed with more granularly-sized examples.


Other memory saving designs

There are many other computational strategies to make training increasingly large neural networks more tractable. For example:To compute the gradient, you need to have saved the original activations, which can consume a lot of device RAM. Checkpointing (also known as activation recomputation) stores any subset of activations, and recomputes the intermediate ones just-in-time during the backward pass. This saves a lot of memory at the computational cost of at most one additional full forward pass. One can also continually trade off between compute and memory cost by selective activation recomputation, which is checkpointing subsets of the activations that are relatively more expensive to store but cheaper to compute.Mixed Precision Training is to train models using lower-precision numbers (most commonly FP16). Modern accelerators can reach much higher FLOP counts with lower-precision numbers, and you also save on device RAM. With proper care, the resulting model can lose almost no accuracy.Offloading is to temporarily offload unused data to the CPU or amongst different devices and later read it back when needed. Naive implementations will slow down training a lot, but sophisticated implementations will pre-fetch data so that the device never needs to wait on it. One implementation of this idea is ZeRO which splits the parameters, gradients, and optimizer states across all available hardware and materializes them as needed.Memory Efficient Optimizers have been proposed to reduce the memory footprint of the running state maintained by the optimizer, such as Adafactor.Compression also can be used for storing intermediate results in the network. For example, Gist compresses activations that are saved for the backward pass; DALL·E compresses the gradients before synchronizing them.At OpenAI, we are training and improving large models from the underlying infrastructure all the way to deploying them for real-world problems. If you'd like to put the ideas from this post into practice—especially relevant for our Scaling and Applied Research teams—we're hiring!


Data parallelism

Data Parallel training means copying the same parameters to multiple GPUs (often called 'workers') and assigning different examples to each to be processed simultaneously. Data parallelism alone still requires that your model fits into a single GPU's memory, but lets you utilize the compute of many GPUs at the cost of storing many duplicate copies of your parameters. That being said, there are strategies to increase the effective RAM available to your GPU, such as temporarily offloading parameters to CPU memory between usages.As each data parallel worker updates its copy of the parameters, they need to coordinate to ensure that each worker continues to have similar parameters. The simplest approach is to introduce blocking communication between workers: (1) independently compute the gradient on each worker; (2) average the gradients across workers; and (3) independently compute the same new parameters on each worker. Step (2) is a blocking average which requires transferring quite a lot of data (proportional to the number of workers times the size of your parameters), which can hurt your training throughput. There are various asynchronous synchronization schemes to remove this overhead, but they hurt learning efficiency; in practice, people generally stick with the synchronous approach.


Mixture-of-Experts (MoE)

With the Mixture-of-Experts (MoE) approach, only a fraction of the network is used to compute the output for any one input. One example approach is to have many sets of weights and the network can choose which set to use via a gating mechanism at inference time. This enables many more parameters without increased computation cost. Each set of weights is referred to as 'experts,' in the hope that the network will learn to assign specialized computation and skills to each expert. Different experts can be hosted on different GPUs, providing a clear way to scale up the number of GPUs used for a model.Illustration of a mixture-of-experts (MoE) layer. Only 2 out of the n experts are selected by the gating network. (Image adapted from: Shazeer et al., 2017)GShard scales an MoE Transformer up to 600 billion parameters with a scheme where only the MoE layers are split across multiple TPU devices and other layers are fully duplicated. Switch Transformer scales model size to trillions of parameters with even higher sparsity by routing one input to a single expert.


Better understanding users' businesses

To better serve users and provide the right type of support, Stripe tries to understand exactly how each business uses the platform and customizes support accordingly. It's an obvious and important step, but one that requires a lot of human hours. 'Many businesses, such as nightclubs, keep their websites sparse and mysterious so it can take a great deal of searching and clicking to understand what's going on,' says Mann. Now, Stripe uses GPT-4 to scan these sites and deliver a summary, which outperforms those written by people.'When we started hand-checking the results, we realized, ‘Wait a minute, the humans were wrong and the model was right.'' Mann says. 'GPT-4 was basically better than human reviewers.'


Answering support questions about documentation

Another critical way Stripe supports developers is through extensive technical documentation and a robust developer support team to answer technical questions or troubleshoot issues. GPT-4 is able to digest, understand and become that virtual assistant—almost instantly.'GPT just works out of the box,' he says. 'That's not how you'd expect software for large language models to work.'GPT is able to understand the user's question, read detailed documentation on their behalf, identify the relevant section and summarize the solution in one place.GPT-4-powered Stripe Docs00:11


Fraud detection on community platforms

There's also the need to manage malicious or bad actors. Stripe maintains a robust community on forums like Discord, which not only crowdsources help for niche technical questions but raises the visibility of developers for future work. However—because it is on the internet—malicious actors find their way into these forums, often trying to gain critical information from community members or regain credibility with Stripe's community team after being kicked off the platform.Just by analyzing the syntax of posts in Discord, GPT-4 has been flagging accounts where Stripe's fraud team should follow up and be sure it isn't, in fact, a fraudster playing nice. GPT-4 can help scan inbound communications, identifying coordinated activity from malicious actors.


Looking forward

Now the Stripe team is thinking about the next round of features. GPT could be deployed as a business coach that can understand revenue models or advise businesses on strategies. With the potential applications only broadening as GPT gets smarter over time.Mann says he and his team are essentially painting with a new canvas every day.'Some of these features,' he says after listing all the ways Stripe has had success so far, 'feel like magic.'


English to Icelandic

PromptHvað heitir Donald Duck á íslensku?What is Donald Duck called in Icelandic?GPT-3 completionDonald Duck heitir á íslensku Donaldi KjániDonald Duck is called in Icelandic Donaldi the FoolIncorrectChatGPT completionDonald Duck heitir sama nafni á íslensku og enskuDonald Duck has the same name in Icelandic and EnglishIncorrectGPT-4 completionDonald Duck heitir Andrés Önd á íslenskuDonald Duck is called Andrés Önd in IcelandicCorrectEven GPT-4, while much better at Icelandic than GPT-3.5, produces Icelandic with grammatical errors, 'translationese', and incorrect cultural knowledge. To change this, Vilhjálmur Þorsteinsson, chief executive at Miðeind ehf, a language technology company involved in the initiative, assembled a team of 40 volunteers to train GPT-4 on proper Icelandic grammar and cultural knowledge. In a process called Reinforcement Learning from Human Feedback, or RLHF, human testers give GPT-4 a prompt, and four possible completions are generated. Testers then select the best answer from the four responses and edit it to create an ideal completion. The data from this process is then used to further train GPT-4 to produce better responses in the future.Prior to RLHF, the process of fine-tuning a model was labor and data-intensive. Þorsteinsson's team attempted to fine-tune a GPT-3 model with 300,000 Icelandic language examples, but the results were disappointing.The capability to generate grammatical or correct Icelandic just wasn't there with GPT-3.Vilhjálmur Þorsteinsson, Chief Executive, Miðeind ehf'Now we can just jump directly to the general capabilities of the large models,' Þorsteinsson says, 'and enable things with our language that used to require a lot of manual labor, data preparation and resource collection for each use case.' RLHF produces results with just 100 examples, which makes it more feasible for other low-resource languages, with less digital language data available, to replicate the process. Any language model, depending on the strength and breadth of data training, has a scale of tasks it can perform. Simple language models are limited to simple tasks like assessing the sentiment of a tweet or identifying company names. More complex models are multi-taskers, able to work with abstract concepts, account for social and environment contexts and even exhibit degrees of creativity. GPT-4, for instance, can generate a poem about the vagaries of modern life in the style of Völuspá, an ancient Icelandic poem from the Poetic Edda of Norse mythology. Miðeind's team of AI researchers has been working on training GPT-4 on Icelandic.


Cultural context

GPT-4 gives a different answer to the same question depending on which language you ask in.Prompt (Icelandic)Hvað sitja margir á þingi?CompletionÁ Alþingi, sem er lögþing Íslands, sitja 63 þingmenn.In Alþingi, which is the Icelandic parliament, there are 63 parliamentarians.Prompt (English)How many parliamentarians are there?CompletionIt depends on which country you are referring to. For example: / • In the United Kingdom, the Parliament has 650 members in the House of Commons and approximately 800 members in the House of Lords. / • In Canada, there are 338 members of Parliament in the House of Commons and 105 senators in the Senate. / • In India, there are 545 members in the Lok Sabha (House of the People) and 245 members in the Rajya Sabha (Council of States). / • In Australia, there are 151 members of the House of Representatives and 76 senators in the Senate. Prompt (Icelandic)Hver er forseti núna?CompletionForseti Íslands núna (2021) er Guðni Th. Jóhannesson.The current (2021) President of Iceland is Guðni Th. Jóhannesson.Prompt (English)Who is president now?CompletionAs of 2021, the President of the United States is Joe Biden.With one round of RLHF complete, the model still has some room for improvement. This is the ongoing work for the Iceland team: continue to train GPT-4 with enough examples that the model can power the most complex and creative applications in Icelandic, rather than defaulting to English, so that Icelandic companies can soon deploy GPT-4 in Icelandic in their interactive applications.


Room for improvement

PromptEr það algengt að íslensk börn séu látin sofa úti í vagni?Is it common for Icelandic babies to be put down for a nap outside in a pram?CompletionJá, það er algengt að íslensk börn séu látin sofa úti í vagni, sérstaklega á vori og sumri. Margir foreldrar telja að útiloft og fersk hrein vindur séu góðir fyrir börnin og auki séu þau oftast rolegri við að sofna úti. Æskilegt er að fylgjast vel með veðurfari og hitastigi, og passa að barnið sé nægur klætt eftir aðstæðum.Completion corrected for grammar and spellingJá, það er algengt að íslensk börn séu látin sofa úti í vagni, sérstaklega á vori og sumri. Margir foreldrar telja að útiloft og ferskur hreinn vindur sé góður fyrir börnin og að auki séu þau oftast rólegri við að sofna úti. Æskilegt er að fylgjast vel með veðurfari og hitastigi, og passa að barnið sé nægur klætt eftir aðstæðum.Yes, it is common for Icelandic babies to nap outside in a pram, especially in spring and summer. Many parents think that outside air and fresh clean wind are good for the babies and, furthermore, that they are usually calmer when they fall asleep outside. It is best to pay close attention to weather and temperature and to make sure that the baby is enough dressed according to circumstances.Not only will Þorsteinsson's team have trained the world's most powerful language engine in their language, but they will have provided an entire country with the opportunity to interact with OpenAI's models in their own language. Thus, using GPT-4 as its backend, Embla—Miðeind's voice assistant app—will be able to have conversations with users in fluent Icelandic, as well as offering translations to other languages. As another example, Icelandic companies won't have to rely on an English-speaking chatbot on their website, but can have one in Icelandic instead.We're excited for the impact of our work on the greater model—and happy to have OpenAI's support in helping the Icelandic language survive in a digital age.Vilhjálmur Þorsteinsson, Chief Executive, Miðeind ehf


Pretraining

Core contributorsChristopher Berner Supercomputing leadGreg Brockman Infrastructure leadTrevor Cai Throughput leadDavid Farhi Manager of optimization teamChris Hesse Infrastructure usability co-leadShantanu Jain Infrastructure usability co-leadKyle Kosic Uptime and stability leadJakub Pachocki Overall lead, optimization leadAlex Paino Architecture & data vice leadMikhail Pavlov Software correctness leadMichael Petrov Hardware correctness leadNick Ryder Architecture & data leadSzymon Sidor Optimization vice leadNikolas Tezak Execution leadPhil Tillet Triton leadAmin Tootoonchian Model distribution, systems & networking leadQiming Yuan Dataset sourcing and processing leadWojciech Zaremba Manager of dataset teamCompute cluster scalingChristopher Berner, Oleg Boiko, Andrew Cann, Ben Chess, Christian Gibson, Mateusz Litwin, Emy Parparita, Henri Roussez, Eric Sigler, Akila WelihindaDataSandhini Agarwal, Suchir Balaji, Mo Bavarian, Che Chang, Sheila Dunning, Leo Gao, Jonathan Gordon, Peter Hoeschele, Shawn Jain, Shantanu Jain, Roger Jiang, Heewoo Jun, Łukasz Kaiser, Nitish Shirish Keskar, Jong Wook Kim, Aris Konstantinidis, Chak Li, Todor Markov, Bianca Martin, David Mély, Oleg Murk, Hyeonwoo Noh, Long Ouyang, Alex Paino, Vitchyr Pong, Alec Radford, Nick Ryder, John Schulman, Daniel Selsam, Ian Sohl, Chelsea Voss, Lilian Weng, Clemens Winter, Tao Xu, Qiming Yuan, Wojciech ZarembaDistributed training infrastructureGreg Brockman, Trevor Cai, Chris Hesse, Shantanu Jain, Yongjik Kim, Kyle Kosic, Mateusz Litwin, Jakub Pachocki, Mikhail Pavlov, Szymon Sidor, Nikolas Tezak, Madeleine Thompson, Amin Tootoonchian, Qiming YuanHardware correctnessGreg Brockman, Shantanu Jain, Kyle Kosic, Michael Petrov, Nikolas Tezak, Amin Tootoonchian, Chelsea Voss, Qiming YuanOptimization & architectureIgor Babuschkin, Mo Bavarian, Adrien Ecoffet, David Farhi, Jesse Han, Ingmar Kanitscheider, Daniel Levy, Jakub Pachocki, Alex Paino, Mikhail Pavlov, Nick Ryder, Szymon Sidor, Jie Tang, Jerry Tworek, Tao XuTraining run babysittingSuchir Balaji, Mo Bavarian, Greg Brockman, Trevor Cai, Chris Hesse, Shantanu Jain, Roger Jiang, Yongjik Kim, Kyle Kosic, Mateusz Litwin, Jakub Pachocki, Alex Paino, Mikhail Pavlov, Michael Petrov, Nick Ryder, Szymon Sidor, Nikolas Tezak, Madeleine Thompson, Phil Tillet, Amin Tootoonchian, Chelsea Voss, Ben Wang, Tao Xu, Qiming Yuan


Long context

Core contributorsGabriel Goh Long context co-leadŁukasz Kaiser Long context leadBen Wang Attention architecture leadClemens Winter Long context co-leadLong context researchMo Bavarian, Gabriel Goh, Heewoo Jun, Łukasz Kaiser, Chak Li, Ben Wang, Clemens WinterLong context kernelsPhil Tillet


Vision

Core contributorsTrevor Cai Execution leadMark Chen Vision team co-lead, Deployment leadCasey Chu Initial prototype leadChris Hesse Data load balancing & developer tooling leadShengli Hu Vision Safety Evaluations leadYongjik Kim GPU performance leadJamie Kiros Overall vision co-lead, deployment research & evaluation leadDaniel Levy Overall vision co-lead, optimization leadChristine McLeavey Vision team leadDavid Mély Data leadHyeonwoo Noh Overall vision co-lead, research leadMikhail Pavlov Scaling engineering leadRaul Puri Overall vision co-lead, engineering leadAmin Tootoonchian Model distribution, systems & networking leadArchitecture researchCasey Chu, Jamie Kiros, Christine McLeavey, Hyeonwoo Noh, Raul Puri, Alec Radford, Aditya RameshCompute cluster scalingAndrew Cann, Rory Carmichael, Christian Gibson, Henri Roussez, Akila WelihindaDistributed training infrastructureTrevor Cai, Yunxing Dai, Chris Hesse, Brandon Houghton, Yongjik Kim, Łukasz Kondraciuk, Hyeonwoo Noh, Mikhail Pavlov, Raul Puri, Nikolas Tezak, Amin Tootoonchian, Tianhao ZhengHardware correctnessOleg Boiko, Trevor Cai, Michael Petrov, Alethea PowerDataJong Wook Kim, David Mély, Reiichiro Nakano, Hyeonwoo Noh, Long Ouyang, Raul Puri, Pranav Shyam, Tao XuAlignment DataLong OuyangTraining run babysittingTrevor Cai, Kyle Kosic, Daniel Levy, David Mély, Reiichiro Nakano, Hyeonwoo Noh, Mikhail Pavlov, Raul Puri, Amin TootoonchianDeployment & post-trainingIlge Akkaya, Mark Chen, Jamie Kiros, Rachel Lim, Reiichiro Nakano, Raul Puri, Jiayi Weng


Evaluation & analysis

Core contributorsSandhini Agarwal System Card co-leadLama Ahmad Expert red teaming & adversarial testing program leadMo Bavarian Capability prediction co-leadTyna Eloundou Safety evaluations co-leadAndrew Kondrich OpenAI Evals open-sourcing co-leadGretchen Krueger System Card co-leadMichael Lampe Privacy and PII evaluations leadPamela Mishkin Economic impact & overreliance evaluations leadBenjamin Sokolowsky Capability prediction co-leadJack Rae Research benchmark execution leadChelsea Voss Eval execution leadAlvin Wang OpenAI Evals leadKai Xiao Safety evaluations co-leadMarvin Zhang OpenAI Evals open-sourcing co-leadOpenAI Evals libraryShixiang Shane Gu, Angela Jiang, Logan Kilpatrick, Andrew Kondrich, Pamela Mishkin, Jakub Pachocki, Ted Sanders, Jessica Shieh, Alvin Wang, Marvin ZhangModel-graded evaluation infrastructureLiam Fedus, Rapha Gontijo-Lopes, Shixiang Shane Gu, Andrew Kondrich, Michael (Rai) Pokorny, Wojciech Zaremba, Chong Zhang, Marvin Zhang, Shengjia Zhao, Barret ZophAcceleration forecastingAlan Hickey, Daniel Kokotajlo, Cullen O'Keefe, Sarah ShokerChatGPT evaluationsJuan Felipe Cerón Uribe, Hyung Won Chung, Rapha Gontijo-Lopes, Liam Fedus, Luke Metz, Michael Rai Pokorny, Jason Wei, Shengjia Zhao, Barret ZophCapability evaluationsSully Chen, Tyna Eloundou, Shengli Hu, Roger Jiang, Jamie Kiros, Teddy Lee, Scott Mayer McKinney, Jakub Pachocki, Alex Paino, Giambattista Parascandolo, Boris Power, Raul Puri, Jack Rae, Nick Ryder, Ted Sanders, Szymon Sidor, Benjamin Sokolowsky, Chelsea Voss, Alvin Wang, Rowan Zellers, Juntang ZhuangCoding evaluationsIlge Akkaya, Mo Bavarian, Jonathan Gordon, Shawn Jain, Chak Li, Oleg Murk, Vitchyr Pong, Benjamin Sokolowsky, Jerry Tworek, Kevin Yu, Wojciech ZarembaReal-world use case evaluationsAndrew Kondrich, Joe Palermo, Boris Power, Ted SandersContamination investigationsAdrien Ecoffet, Roger Jiang, Ingmar Kanitscheider, Scott Mayer McKinney, Alex Paino, Giambattista Parascandolo, Jack Rae, Qiming YuanInstruction following and API evalsDiogo Almeida, Carroll Wainwright, Marvin ZhangNovel capability discoveryFilipe de Avila Belbute Peres, Kevin Button, Fotis Chantzis, Mike Heaton, Wade Hickey, Xin Hu, Andrew Kondrich, Matt Knight, Andrew Mayne, Jake McNeil, Vinnie Monaco, Joe Palermo, Joel Parish, Boris Power, Bob Rotsted, Ted SandersVision evaluationsShixiang Shane Gu, Shengli Hu, Jamie Kiros, Hyeonwoo Noh, Raul Puri, Rowan ZellersEconomic impact evaluationTyna Eloundou, Sam Manning, Aalok Mehta, Pamela MishkinNon-proliferation, international humanitarian law & national security red teamingSarah ShokerOverreliance analysisMiles Brundage, Michael Lampe, Pamela MishkinPrivacy and PII evaluationsMichael Lampe, Vinnie Monaco, Ashley PantulianoSafety and policy evaluationsJosh Achiam, Sandhini Agarwal, Lama Ahmad, Jeff Belgum, Tyna Eloundou, Johannes Heidecke, Shengli Hu, Joost Huizinga, Jamie Kiros, Gretchen Krueger, Michael Lampe, Stephanie Lin, Ryan Lowe, Todor Markov, Vinnie Monaco, Tong Mu, Raul Puri, Girish Sastry, Andrea Vallone, Carroll Wainwright, CJ Weinmann, Lilian Weng, Kai Xiao, Chong ZhangOpenAI adversarial testersJosh Achiam, Steven Adler, Lama Ahmad, Shyamal Anadkat, Red Avila, Gabriel Bernadett-Shapiro, Anna-Luisa Brakman, Tim Brooks, Miles Brundage, Chelsea Carlson, Derek Chen, Hyung Won Chung, Jeremiah Currier, Daniel Kokotajlo, David Dohan, Adrien Ecoffet, Juston Forte, Vik Goel, Ryan Greene, Johannes Heidecke, Alan Hickey, Shengli Hu, Joost Huizinga, Janko, Tomer Kaftan, Ali Kamali, Nitish Shirish Keskar, Tabarak Khan, Hendrik Kirchner, Daniel Kokotajlo, Gretchen Krueger, Michael Lampe, Teddy Lee, Molly Lin, Ryan Lowe, Todor Markov, Jake McNeil, Pamela Mishkin, Vinnie Monaco, Daniel Mossing, Tong Mu, Oleg Murk, Cullen O'Keefe, Joe Palermo, Giambattista Parascandolo, Joel Parish, Boris Power, Alethea Power, Cameron Raymond, Francis Real, Bob Rotsted, Mario Salterelli, Sam Wolrich, Ted Sanders, Girish Sasty, Sarah Shoker, Shyamal Anadkat, Yang Song, Natalie Staudacher, Madeleine Thompson, Elizabeth Tseng, Chelsea Voss, Jason Wei, Chong ZhangSystem card & broader impacts analysisSteven Adler, Sandhini Agarwal, Lama Ahmad, Janko Altenschmidt, Jeff Belgum, Gabriel Bernadett-Shapiro, Miles Brundage, Derek Chen, Tyna Eloundou, Liam Fedus, Leo Gao, Vik Goel, Johannes Heidecke, Alan Hickey, Shengli Hu, Joost Huizinga, Daniel Kokotajlo, Gretchen Krueger, Michael Lampe, Jade Leung, Stephanie Lin, Ryan Lowe, Kim Malfacini, Todor Markov, Bianca Martin, Aalok Mehta, Pamela Mishkin, Tong Mu, Richard Ngo, Cullen O'Keefe, Joel Parish, Rai Pokorny, Bob Rotsted, Girish Sastry, Sarah Shoker, Andrea Vallone, Carroll Wainwright, CJ Weinmann, Lilian Weng, Dave Willner, Kai Xiao, Chong Zhang


RL & alignment

Core contributorsGreg Brockman Core infrastructure authorArka Dhar Human data product managerLiam Fedus Data flywheel leadTarun Gogineni Model creativityRapha Gontijo-Lopes Synthetic dataJoshua Gross Data collection engineering co-leadJohannes Heidecke Refusals & model safety co-leadJoost Huizinga Initial fine-tuning deriskingTeddy Lee Human data product managerJan Leike Alignment co-leadRyan Lowe Alignment co-leadLuke Metz Infrastructure lead, ChatML format leadLong Ouyang IF data collection leadJohn Schulman Overall leadJerry Tworek Code leadCarroll Wainwright IF data infrastructure leadJonathan Ward Data collection engineering co-leadJiayi Weng RL Infrastructure authorSarah Yoo Human data operations managerWojciech Zaremba Human data leadChong Zhang Refusals & model safety co-leadShengjia Zhao Reward model leadBarret Zoph Overall training leadDataset contributionsDiogo Almeida, Mo Bavarian, Juan Felipe Cerón Uribe, Tyna Eloundou, Liam Fedus, Tarun Gogineni, Rapha Gontijo-Lopes, Jonathan Gordon, Joost Huizinga, Shawn Jain, Roger Jiang, Łukasz Kaiser, Christina Kim, Jan Leike, Chak Li, Stephanie Lin, Ryan Lowe, Jacob Menick, Luke Metz, Pamela Mishkin, Tong Mu, Oleg Murk, Ashvin Nair, Long Ouyang, Alex Passos, Michael (Rai) Pokorny, Vitchyr Pong, Shibani Santurkar, Daniel Selsam, Sarah Shoker,, Carroll Wainwright, Matt Wiethoff, Jeff Wu, Kai Xiao, Kevin Yu, Marvin Zhang, Chong Zhang, William Zhuk, Barret ZophData infrastructureIrwan Bello, Lenny Bogdonoff, Juan Felipe Cerón Uribe, Joshua Gross, Shawn Jain, Haozhun Jin, Christina Kim, Aris Konstantinidis, Teddy Lee, David Medina, Jacob Menick, Luke Metz, Ashvin Nair,Long Ouyang, Michael (Rai) Pokorny, Vitchyr Pong, John Schulman, Jonathan Ward, Jiayi Weng, Matt Wiethoff, Sarah Yoo, Kevin Yu, Wojciech Zaremba, William Zhuk, Barret ZophChatML formatIlge Akkaya, Christina Kim, Chak Li, Rachel Lim, Jacob Menick, Luke Metz, Andrey Mishchenko, Vitchyr Pong, John Schulman, Carroll Wainwright, Barret ZophModel safetyJosh Achiam, Steven Adler, Juan Felipe Cerón Uribe, Hyung Won Chung, Tyna Eloundou, Rapha Gontijo-Lopes, Shixiang Shane Gu, Johannes Heidecke, Joost Huizinga, Teddy Lee, Jan Leike, Stephanie Lin, Ryan Lowe, Todor Markov, Luke Metz, Tong Mu, Shibani Santurkar, John Schulman, Andrea Vallone, Carroll Wainwright, Jason Wei, Lilian Weng, Kai Xiao, Chong Zhang, Marvin Zhang, Barret ZophRefusalsJuan Felipe Cerón Uribe, Tyna Eloundou, Johannes Heidecke, Joost Huizinga, Jan Leike, Stephanie Lin, Ryan Lowe, Pamela Mishkin, Tong Mu, Carroll Wainwright, Lilian Weng, Kai Xiao, Chong Zhang, Barret ZophFoundational RLHF and InstructGPT workDiogo Almeida, Joost Huizinga, Roger Jiang, Jan Leike, Stephanie Lin, Ryan Lowe, Pamela Mishkin, Dan Mossing, Long Ouyang, Katarina Slama, Carroll Wainwright, Jeff Wu, Kai Xiao, Marvin ZhangFlagship training runsGreg Brockman, Liam Fedus, Johannes Heidecke, Joost Huizinga, Roger Jiang, Kyle Kosic, Luke Metz, Ashvin Nair, Jiayi Weng, Chong Zhang, Shengjia Zhao, Barret ZophCode capabilityIlge Akkaya, Mo Bavarian, Jonathan Gordon, Shawn Jain, Haozhun Jin, Teddy Lee, Chak Li, Oleg Murk, Ashvin Nair, Vitchyr Pong, Benjamin Sokolowsky, Jerry Tworek, Matt Wiethoff, Sarah Yoo, Kevin Yu, Wojciech Zaremba, William Zhuk


Additional contributions

Sam Altman, Katie Mayer, Bob McGrew, Mira Murati, Ilya Sutskever, Peter WelinderBlog post & paper contentSandhini Agarwal, Greg Brockman, Miles Brundage, Adrien Ecoffet, Tyna Eloundou, David Farhi, Johannes Heidecke, Shengli Hu, Joost Huizinga, Roger Jiang, Gretchen Krueger, Jan Leike, Daniel Levy, Stephanie Lin, Ryan Lowe, Tong Mu, Hyeonwoo Noh, Jakub Pachocki, Jack Rae, Kendra Rimbach, Shibani Santurkar, Szymon Sidor, Benjamin Sokolowsky, Jie Tang, Chelsea Voss, Kai Xiao, Rowan Zellers, Chong Zhang, Marvin ZhangCommunicationsRuby Chen, Cory Decareaux, Thomas Degry, Steve Dowling, Niko Felix, Elie Georges, Anna Makanju, Andrew Mayne, Aalok Mehta, Elizabeth Proehl, Kendra Rimbach, Natalie Summers, Justin Jay Wang, Hannah WongCompute allocation supportTheresa Lopez, Elizabeth TsengContracting, revenue, pricing & finance supportBrooke Chan, Denny Jin, Billie Jonn, Patricia Lue, Kyla Sheppard, Lauren WorkmanLaunch partners & product operationsFilipe de Avila Belbute Peres, Brittany Carey, Simón Posada Fishman, Isabella Fulford, Teddy Lee, Yaniv Markovski, Tolly Powell, Toki Sherbakov, Jessica Shieh, Natalie Staudacher, Preston TuggleLegalJake Berdine, Che Chang, Sheila Dunning, Ashley PantulianoSecurity & privacy engineeringKevin Button, Fotis Chantzis, Wade Hickey, Xin Hu, Shino Jomoto, Matt Knight, Jake McNeil, Vinnie Monaco, Joel Parish, Bob RotstedSystem administration & on-call supportMorgan Grafstein, Francis Real, Mario SaltarelliAuthorship & credit attributionDavid FarhiWe also acknowledge and thank every OpenAI team member not explicitly mentioned above, including the amazing people on the executive assistant, finance, go to market, human resources, legal, operations and recruiting teams. From hiring everyone in the company, to making sure we have an amazing office space, to building the administrative, HR, legal, and financial structures that allow us to do our best work, everyone at OpenAI has contributed to GPT-4.We thank Microsoft for their partnership, especially Microsoft Azure for supporting model training with infrastructure design and management, and the Microsoft Bing team and Microsoft's safety teams for their partnership on safe deployment.We are grateful to our expert adversarial testers and red teamers who helped test our models at early stages of development and informed our risk assessments as well as the system card. Participation in this red teaming process is not an endorsement of the deployment plans of OpenAI or OpenAI's policies: Steven Basart, Sophie Duba, Cèsar Ferri, Heather Frase, Gavin Hartnett, Jake J. Hecla, Dan Hendrycks, Jose Hernandez-Orallo, Alice Hunsberger, Rajiv W. Jain, Boru Gollo Jattani, Lauren Kahn, Dan Kaszeta, Sara Kingsley, Noam Kolt, Nathan Labenz, Eric Liddick, Andrew J. Lohn, Andrew MacPherson, Sam Manning, Mantas Mazeika, Anna Mills, Yael Moros, Jimin Mun, Aviv Ovadya, Roya Pakzad, Yifan Peng, Ciel Qi, Alex Rosenblatt, Paul Röttger, Maarten Sap, Wout Schellaert, George Shih, Muhammad Shoker, Melanie Subbiah, Bryan West, Andrew D. White, Anna Katariina Wisakanto, Akhila Yerukola, Lexin Zhou, Xuhui Zhou.Contributors listed in alphabetized order.


Deployment

Core contributorsSteven Adler Early stage program management leadSandhini Agarwal Launch safety leadDerek Chen Monitoring & response leadAtty Eleti GPT-4 API co-leadJoanne Jang GPT-4 product co-leadAngela Jiang GPT-4 product co-leadTomer Kaftan Inference infrastructure & deployment leadRachel Lim GPT-4 API co-leadKim Malfacini Usage policy leadBianca Martin Release program management leadEvan Morikawa Engineering leadHenrique Ponde de Oliveira Pinto Inference workflow leadHeather Schmidt GPT-4 infrastructure managementMaddie Simens Design leadFelipe Petroski Such Inference optimization & reliability leadAndrea Vallone Detection & refusals policy leadLilian Weng Applied research leadDave Willner Trust & safety leadMichael Wu Inference research leadInference researchPaul Baltescu, Scott Gray, Yuchen He, Arvind Neelakantan, Michael WuGPT-4 API & ChatML deploymentGreg Brockman, Brooke Chan, Chester Cho, Atty Eleti, Rachel Lim, Andrew Peng, Michelle Pokrass, Sherwin WuGPT-4 web experienceValerie Balcom, Lenny Bogdonoff, Jason Chen, Dave Cummings, Noah Deutsch, Mike Heaton, Paul McMillan, Rajeev Nayak, Joel Parish, Adam Perelman, Eric Sigler, Nick Turley, Arun Vijayvergiya, Chelsea VossInference infrastructureBrooke Chan, Scott Gray, Chris Hallacy, Kenny Hsu, Tomer Kaftan, Rachel Lim, Henrique Ponde de Oliveira Pinto, Raul Puri, Heather Schmidt, Felipe Petroski SuchReliability engineeringHaiming Bao, Madelaine Boyd, Ben Chess, Damien Deville, Yufei Guo, Vishal Kuo, Ikai Lan, Michelle Pokrass, Carl Ross, David Schnurr, Jordan Sitkin, Felipe Petroski SuchTrust & safety engineeringJeff Belgum, Madelaine Boyd, Vik GoelTrust & safety monitoring and responseJanko Altenschmidt, Anna-Luisa Brakman, Derek Chen, Florencia Leoni Aleman, Molly Lin, Cameron Raymond, CJ Weinmann, Dave Willner, Samuel WolrichTrust & safety policyRosie Campbell, Kim Malfacini, Andrea Vallone, Dave WillnerDeployment computePeter Hoeschele, Evan MorikawaProduct managementJeff Harris, Joanne Jang, Angela Jiang


Latest updates

View all updatesDALL·E API now available in public betaNov 3, 2022November 3, 2022DALL·E 2: Extending creativityJul 14, 2022July 14, 2022DALL·E now available without waitlistSep 28, 2022September 28, 2022DALL·E: Introducing outpaintingAug 31, 2022August 31, 2022DALL·E 2 can create original, realistic images and art from a text description. It can combine concepts, attributes, and styles.


Tabs

Image generationOutpaintingInpaintingVariationsDALL·E 2 can create original, realistic images and art from a text description. It can combine concepts, attributes, and styles.Try DALL·E


Output Navigation

Go to slide 1Go to slide 2Go to slide 3Go to slide 4Go to slide 5DALL·E 2 can expand images beyond what's in the original canvas, creating expansive new compositions.Try DALL·EDALL·E 2 can make realistic edits to existing images from a natural language caption. It can add and remove elements while taking shadows, reflections, and textures into account.Try DALL·E


Output Navigation

Go to slide 1Go to slide 2Go to slide 3Go to slide 4Go to slide 5DALL·E 2 can take an image and create different variations of it inspired by the original.Try DALL·E


Output Navigation

Go to slide 1Go to slide 2Go to slide 3Go to slide 4Go to slide 5In January 2021, OpenAI introduced DALL·E. One year later, our newest system, DALL·E 2, generates more realistic and accurate images with 4x greater resolution.DALL·E 1DALL·E 2


DALL·E 2 is preferred over DALL·E 1 when evaluators compared each model.

preferred for caption matchingpreferred for photorealism


Related research

View all researchHierarchical text-conditional image generation with CLIP latentsApr 13, 2022April 13, 2022DALL·E: Creating images from textJan 5, 2021January 5, 2021DALL·E 2 pre-training mitigationsJun 28, 2022June 28, 2022CLIP: Connecting text and imagesJan 5, 2021January 5, 2021


Preventing harmful generations

Read about improving safety


Curbing misuse

Read content policy


Credits

Research AdvancementsAditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, Mark ChenEngineering, Design, Product, and PrototypingJeff Belgum, Dave Cummings, Jonathan Gordon, Chris Hallacy, Shawn Jain, Joanne Jang, Fraser Kelton, Vishal Kuo, Joel Lehman, Rachel Lim, Bianca Martin, Evan Morikawa, Rajeev Nayak, Glenn Powell, Krijn Rijshouwer, David Schnurr, Maddie Simens, Kenneth Stanley, Felipe Such, Chelsea Voss, Justin Jay WangComms, Policy, Legal, Ops, Safety, and SecuritySteven Adler, Lama Ahmad, Miles Brundage, Kevin Button, Che Chang, Fotis Chantzis, Derek Chen, Frances Choi, Steve Dowling, Elie Georges, Shino Jomoto, Aris Konstantinidis, Gretchen Krueger, Andrew Mayne, Pamela Mishkin, Bob Rotsted, Natalie Summers, Dave Willner, Hannah WongAcknowledgmentsThanks to those who helped with and provided feedback on this release: Sandhini Agarwal, Sam Altman, Chester Cho, Peter Hoeschele, Jacob Jackson, Jong Wook Kim, Matt Knight, Jason Kwon, Anna Makanju, Katie Mayer, Bob McGrew, Luke Miller, Mira Murati, Adam Nace, Hyeonwoo Noh, Cullen O'Keefe, Long Ouyang, Michael Petrov, Henrique Ponde de Oliveira Pinto, Alec Radford, Girish Sastry, Pranav Shyam, Aravind Srinivas, Ilya Sutskever, Preston Tuggle, Arun Vijayvergiya, Peter Welinder


Phased deployment based on learning

View risks and limitationsOur hope is that DALL·E 2 will empower people to express themselves creatively. DALL·E 2 also helps us understand how advanced AI systems see and understand our world, which is critical to our mission of creating AI that benefits humanity.


Text generation models

New capabilities launched at DevDayJSON modeReproducible outputsAssistants APIOpenAI's text generation models (often called generative pre-trained transformers or large language models) have been trained to understand natural language, code, and images. The models provide text outputs in response to their inputs. The inputs to these models are also referred to as 'prompts'. Designing a prompt is essentially how you 'program' a large language model model, usually by providing instructions or some examples of how to successfully complete a task.Using OpenAI's text generation models, you can build applications to:Draft documentsWrite computer codeAnswer questions about a knowledge baseAnalyze textsGive software a natural language interfaceTutor in a range of subjectsTranslate languagesSimulate characters for gamesWith the release of gpt-4-vision-preview, you can now build systems that also process and understand images.Explore GPT-4 with image inputsCheck out the vision guide for more detail.To use one of these models via the OpenAI API, you'll send a request containing the inputs and your API key, and receive a response containing the model's output. Our latest models, gpt-4 and gpt-3.5-turbo, are accessed through the chat completions API endpoint.https://api.openai.com/v1/chat/completionshttps://api.openai.com/v1/completionshttps://api.openai.com/v1/completionsYou can experiment with various models in the chat playground. If you're not sure which model to use, then use gpt-3.5-turbo or gpt-4.


Chat Completions API

Chat models take a list of messages as input and return a model-generated message as output. Although the chat format is designed to make multi-turn conversations easy, it's just as useful for single-turn tasks without any conversation.An example Chat Completions API call looks like the following:Select librarypythonnode.jsCopy1 2 3 4 5 6 7 8 9 10 11 12 from openai import OpenAI client = OpenAI() response = client.chat.completions.create( model='gpt-3.5-turbo', messages=[ {'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Who won the world series in 2020?'}, {'role': 'assistant', 'content': 'The Los Angeles Dodgers won the World Series in 2020.'}, {'role': 'user', 'content': 'Where was it played?'} ] To learn more, you can view the full API reference documentation for the Chat API.The main input is the messages parameter. Messages must be an array of message objects, where each object has a role (either 'system', 'user', or 'assistant') and content. Conversations can be as short as one message or many back and forth turns.Typically, a conversation is formatted with a system message first, followed by alternating user and assistant messages.The system message helps set the behavior of the assistant. For example, you can modify the personality of the assistant or provide specific instructions about how it should behave throughout the conversation. However note that the system message is optional and the model's behavior without a system message is likely to be similar to using a generic message such as 'You are a helpful assistant.'The user messages provide requests or comments for the assistant to respond to. Assistant messages store previous assistant responses, but can also be written by you to give examples of desired behavior.Including conversation history is important when user instructions refer to prior messages. In the example above, the user's final question of 'Where was it played?' only makes sense in the context of the prior messages about the World Series of 2020. Because the models have no memory of past requests, all relevant information must be supplied as part of the conversation history in each request. If a conversation cannot fit within the model's token limit, it will need to be shortened in some way.stream


Managing tokens

Language models read and write text in chunks called tokens. In English, a token can be as short as one character or as long as one word (e.g., a or apple), and in some languages tokens can be even shorter than one character or even longer than one word.For example, the string 'ChatGPT is great!' is encoded into six tokens: ['Chat', 'G', 'PT', ' is', ' great', '!'].The total number of tokens in an API call affects:How much your API call costs, as you pay per tokenHow long your API call takes, as writing more tokens takes more timeWhether your API call works at all, as total tokens must be below the model's maximum limit (4097 tokens for gpt-3.5-turbo)Both input and output tokens count toward these quantities. For example, if your API call used 10 tokens in the message input and you received 20 tokens in the message output, you would be billed for 30 tokens. Note however that for some models the price per token is different for tokens in the input vs. the output (see the pricing page for more information).To see how many tokens are used by an API call, check the usage field in the API response (e.g., response['usage']['total_tokens']).Chat models like gpt-3.5-turbo and gpt-4 use tokens in the same way as the models available in the completions API, but because of their message-based formatting, it's more difficult to count how many tokens will be used by a conversation.To see how many tokens are in a text string without making an API call, use OpenAI's tiktoken Python library. Example code can be found in the OpenAI Cookbook's guide on how to count tokens with tiktoken.Each message passed to the API consumes the number of tokens in the content, role, and other fields, plus a few extra for behind-the-scenes formatting. This may change slightly in the future.If a conversation has too many tokens to fit within a model's maximum limit (e.g., more than 4097 tokens for gpt-3.5-turbo), you will have to truncate, omit, or otherwise shrink your text until it fits. Beware that if a message is removed from the messages input, the model will lose all knowledge of it.Note that very long conversations are more likely to receive incomplete replies. For example, a gpt-3.5-turbo conversation that is 4090 tokens long will have its reply cut off after just 6 tokens.


Parameter details

Frequency and presence penaltiesThe frequency and presence penalties found in the Chat Completions API and Legacy Completions API can be used to reduce the likelihood of sampling repetitive sequences of tokens. They work by directly modifying the logits (un-normalized log-probabilities) with an additive contribution.mu[j] -> mu[j] - c[j] * alpha_frequency - float(c[j] > 0) * alpha_presenceWhere:mu[j] is the logits of the j-th tokenc[j] is how often that token was sampled prior to the current positionfloat(c[j] > 0) is 1 if c[j] > 0 and 0 otherwisealpha_frequency is the frequency penalty coefficientalpha_presence is the presence penalty coefficientAs we can see, the presence penalty is a one-off additive contribution that applies to all tokens that have been sampled at least once and the frequency penalty is a contribution that is proportional to how often a particular token has already been sampled.Reasonable values for the penalty coefficients are around 0.1 to 1 if the aim is to just reduce repetitive samples somewhat. If the aim is to strongly suppress repetition, then one can increase the coefficients up to 2, but this can noticeably degrade the quality of samples. Negative values can be used to increase the likelihood of repetition.


JSON mode New

A common way to use Chat Completions is to instruct the model to always return JSON in some format that makes sense for your use case, by providing a system message. This works well, but occasionally the models may generate output that does not parse to valid JSON.To prevent these errors and improve model performance, when calling gpt-4-1106-preview or gpt-3.5-turbo-1106, you can set response_format to { 'type': 'json_object' } to enable JSON mode. When JSON mode is enabled, the model is constrained to only generate strings that parse into valid JSON.Important notes:When using JSON mode, always instruct the model to produce JSON via some message in the conversation, for example via your system message. If you don't include an explicit instruction to generate JSON, the model may generate an unending stream of whitespace and the request may run continually until it reaches the token limit. To help ensure you don't forget, the API will throw an error if the string 'JSON' does not appear somewhere in the context.The JSON in the message the model returns may be partial (i.e. cut off) if finish_reason is length, which indicates the generation exceeded max_tokens or the conversation exceeded the token limit. To guard against this, check finish_reason before parsing the response.JSON mode will not guarantee the output matches any specific schema, only that it is valid and parses without errors.Note that JSON mode is always enabled when the model is generating arguments as part of function calling.


Completions API Legacy

The completions API endpoint received its final update in July 2023 and has a different interface than the new chat completions endpoint. Instead of the input being a list of messages, the input is a freeform text string called a prompt.An example API call looks as follows:Select librarypythonnode.jsCopy1 2 3 4 5 6 7 from openai import OpenAI client = OpenAI() response = client.completions.create( model='gpt-3.5-turbo-instruct', prompt='Write a tagline for an ice cream shop.' See the full API reference documentation to learn more.


Chat Completions response format

An example Chat Completions API response looks as follows:1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 { 'choices': [ { 'finish_reason': 'stop', 'index': 0, 'message': { 'content': 'The 2020 World Series was played in Texas at Globe Life Field in Arlington.', 'role': 'assistant' } } ], 'created': 1677664795, 'id': 'chatcmpl-7QyqpwdfhqwajicIEznoc6Q47XAyW', 'model': 'gpt-3.5-turbo-0613', 'object': 'chat.completion', 'usage': { 'completion_tokens': 17, 'prompt_tokens': 57, 'total_tokens': 74 } The assistant's reply can be extracted with:Select librarypythonnode.jsCopyresponse['choices'][0]['message']['content']Every response will include a finish_reason. The possible values for finish_reason are:stop: API returned complete message, or a message terminated by one of the stop sequences provided via the stop parameterlength: Incomplete model output due to max_tokens parameter or token limitfunction_call: The model decided to call a functioncontent_filter: Omitted content due to a flag from our content filtersnull: API response still in progress or incompleteDepending on input parameters, the model response may include different information.


Completions response format

An example completions API response looks as follows:1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 { 'choices': [ { 'finish_reason': 'length', 'index': 0, 'logprobs': null, 'text': '\n\n\'Let Your Sweet Tooth Run Wild at Our Creamy Ice Cream Shack' } ], 'created': 1683130927, 'id': 'cmpl-7C9Wxi9Du4j1lQjdjhxBlO22M61LD', 'model': 'gpt-3.5-turbo-instruct', 'object': 'text_completion', 'usage': { 'completion_tokens': 16, 'prompt_tokens': 10, 'total_tokens': 26 } In Python, the output can be extracted with response['choices'][0]['text'].The response format is similar to the response format of the Chat Completions API but also includes the optional field logprobs.


Chat Completions vs. Completions

The Chat Completions format can be made similar to the completions format by constructing a request using a single user message. For example, one can translate from English to French with the following completions prompt:Translate the following English text to French: '{text}'And an equivalent chat prompt would be:[{'role': 'user', 'content': 'Translate the following English text to French: '{text}''}]Likewise, the completions API can be used to simulate a chat between a user and an assistant by formatting the input accordingly.The difference between these APIs is the underlying models that are available in each. The chat completions API is the interface to our most capable model (gpt-4), and our most cost effective model (gpt-3.5-turbo).


Inserting text

The completions endpoint also supports inserting text by providing a suffix in addition to the standard prompt which is treated as a prefix. This need naturally arises when writing long-form text, transitioning between paragraphs, following an outline, or guiding the model towards an ending. This also works on code, and can be used to insert in the middle of a function or file.


Prompt engineering

An awareness of the best practices for working with OpenAI models can make a significant difference in application performance. The failure modes that each exhibit and the ways of working around or correcting those failure modes are not always intuitive. There is an entire field related to working with language models which has come to be known as 'prompt engineering', but as the field has progressed its scope has outgrown merely engineering the prompt into engineering systems that use model queries as components. To learn more, read our guide on prompt engineering which covers methods to improve model reasoning, reduce the likelihood of model hallucinations, and more. You can also find many useful resources including code samples in the OpenAI Cookbook.


Reproducible outputs Beta

Chat Completions are non-deterministic by default (which means model outputs may differ from request to request). That being said, we offer some control towards deterministic outputs by giving you access to the seed parameter and the system_fingerprint response field.To receive (mostly) deterministic outputs across API calls, you can:Set the seed parameter to any integer of your choice and use the same value across requests you'd like deterministic outputs for.Ensure all other parameters (like prompt or temperature) are the exact same across requests.Sometimes, determinism may be impacted due to necessary changes OpenAI makes to model configurations on our end. To help you keep track of these changes, we expose the system_fingerprint field. If this value is different, you may see different outputs due to changes we've made on our systems.Deterministic outputsExplore the new seed parameter in the OpenAI cookbook


How should I set the temperature parameter?

Lower values for temperature result in more consistent outputs, while higher values generate more diverse and creative results. Select a temperature value based on the desired trade-off between coherence and creativity for your specific application.


Is fine-tuning available for the latest models?

Yes, for some. Currently, you can only fine-tune gpt-3.5-turbo and our updated base models (babbage-002 and davinci-002). See the fine-tuning guide for more details on how to use fine-tuned models.


Do you store the data that is passed into the API?

As of March 1st, 2023, we retain your API data for 30 days but no longer use your data sent via the API to improve our models. Learn more in our data usage policy. Some endpoints offer zero retention.


Should I use ChatGPT or the API?

ChatGPT offers a chat interface to the models in the OpenAI API and a range of built-in features such as integrated browsing, code execution, plugins, and more. By contrast, using OpenAI's API provides more flexibility.


Which model should I use?

We generally recommend that you use either gpt-4 or gpt-3.5-turbo. Which of these you should use depends on the complexity of the tasks you are using the models for. gpt-4 generally performs better on a wide range of evaluations. In particular, gpt-4 is more capable at carefully following complex instructions. By contrast gpt-3.5-turbo is more likely to follow just one part of a complex multi-part instruction. gpt-4 is less likely than gpt-3.5-turbo to make up information, a behavior known as 'hallucination'. gpt-4 also has a larger context window with a maximum size of 8,192 tokens compared to 4,096 tokens for gpt-3.5-turbo. However, gpt-3.5-turbo returns outputs with lower latency and costs much less per token.We recommend experimenting in the playground to investigate which models provide the best price performance trade-off for your usage. A common design pattern is to use several distinct query types which are each dispatched to the model appropriate to handle them.


How can I make my application more safe?

If you want to add a moderation layer to the outputs of the Chat API, you can follow our moderation guide to prevent content that violates OpenAI's usage policies from being shown.


Token log probabilities

The completions API can provide a limited number of log probabilities associated with the most likely tokens for each output token. This feature is controlled by using the logprobs field. This can be useful in some cases to assess the confidence of the model in its output.


Safety in deployment

We monitor the use of our tools and update safety mitigations based on what we learn about model risks and capabilities, reflecting our leadership in commercial AI deployment.Learn about safetyNew AI classifier for indicating AI-written textJan 31, 2023January 31, 2023Reducing bias and improving safety in DALL·E 2Jul 18, 2022July 18, 2022New and improved content moderation toolingAug 10, 2022August 10, 2022DALL·E 2 pre-training mitigationsJun 28, 2022June 28, 2022


Our principles

Minimize harmWe will build safety into our AI tools where possible, and work hard to aggressively reduce harms posed by the misuse or abuse of our AI tools.Build trustAlongside our user and developer community, we'll share the responsibility of supporting safe, beneficial applications of our technology.Learn and iterateWe will observe and analyze how our models behave and are used and seek input on our approach to safety in order to improve our systems over time.Be a pioneer in trust and safetyWe will support research into the unique trust and safety challenges posed by generative AI, to help improve safety beyond our ecosystem.


Documents and policies

We've created and compiled resources about our safety practices. Here's how you can uphold trust and safety as you engage with our products.Usage policiesBy following our usage policies, you'll help us make sure that our technology is used for good.ModerationThe moderation endpoint is a tool you can use to check whether content complies with OpenAI's content policy.Safety best practicesRead about how to build with safety in mind.Educator considerations for ChatGPTLearn more about the capabilities, limitations, and considerations for using ChatGPT for teaching and learning.


Our commitment to secure AI

OpenAI's mission is to create artificial intelligence systems that benefit everyone. To that end, we invest heavily in research and engineering to ensure our AI systems are safe and secure. However, as with any complex technology, we understand that vulnerabilities and flaws can emerge.We believe that transparency and collaboration are crucial to addressing this reality. That's why we are inviting the global community of security researchers, ethical hackers, and technology enthusiasts to help us identify and address vulnerabilities in our systems. We are excited to build on our coordinated disclosure commitments by offering incentives for qualifying vulnerability information. Your expertise and vigilance will have a direct impact on keeping our systems and users secure.


Introducing the Bug Bounty Program

The OpenAI Bug Bounty Program is a way for us to recognize and reward the valuable insights of security researchers who contribute to keeping our technology and company secure. We invite you to report vulnerabilities, bugs, or security flaws you discover in our systems. By sharing your findings, you will play a crucial role in making our technology safer for everyone.We have partnered with Bugcrowd, a leading bug bounty platform, to manage the submission and reward process, which is designed to ensure a streamlined experience for all participants. Detailed guidelines and rules for participation can be found on our Bug Bounty Program page.


Staying secure together

At OpenAI, we recognize the critical importance of security and view it as a collaborative effort. We invite the security research community to participate in our Bug Bounty Program.Interested in contributing further? We're hiring—explore open security roles on our careers page. Join us in ensuring that the frontier of technology is secure.Participate in our Bug Bounty Program


Incentives and rewards

To incentivize testing and as a token of our appreciation, we will be offering cash rewards based on the severity and impact of the reported issues. Our rewards range from $200 for low-severity findings to up to $20,000 for exceptional discoveries. We recognize the importance of your contributions and are committed to acknowledging your efforts.