The games

Thus far OpenAI Five has played (with our restrictions) versus each of these teams:Best OpenAI employee team: 2.5k MMR (46th percentile)Best audience players watching OpenAI employee match (including Blitz, who commentated the first OpenAI employee match): 4–6k MMR (90th-99th percentile), though they'd never played as a team.Valve employee team: 2.5–4k MMR (46th-90th percentile).Amateur team: 4.2k MMR (93rd percentile), trains as a team.Semi-pro team: 5.5k MMR (99th percentile), trains as a team.The April 23rd version of OpenAI Five was the first to beat our scripted baseline. The May 15th version of OpenAI Five was evenly matched versus team 1, winning one game and losing another. The June 6th version of OpenAI Five decisively won all its games versus teams 1–3. We set up informal scrims with teams 4 & 5, expecting to lose soundly, but OpenAI Five won two of its first three games versus both.The teamwork aspect of the bot was just overwhelming. It feels like five selfless players that know a good general strategy.BlitzWe observed that OpenAI Five:Repeatedly sacrificed its own safe lane (top lane for dire; bottom lane for radiant) in exchange for controlling the enemy's safe lane, forcing the fight onto the side that is harder for their opponent to defend. This strategy emerged in the professional scene in the last few years, and is now considered to be the prevailing tactic. Blitz commented that he only learned this after eight years of play, when Team Liquid told him about it.Pushed the transitions from early- to mid-game faster than its opponents. It did this by: (1) setting up successful ganks (when players move around the map to ambush an enemy hero—see animation) when players overextended in their lane, and (2) by grouping up to take towers before the opponents could organize a counterplay.Deviated from current playstyle in a few areas, such as giving support heroes (which usually do not take priority for resources) lots of early experience and gold. OpenAI Five's prioritization allows for its damage to peak sooner and push its advantage harder, winning team fights and capitalizing on mistakes to ensure a fast win.Trophies awarded after the match between the best players at OpenAI and our bot team. One trophy for the humans, one trophy for the bots (represented by Susan Zhang from our team!)


Differences versus humans

OpenAI Five is given access to the same information as humans, but instantly sees data like positions, healths, and item inventories that humans have to check manually. Our method isn't fundamentally tied to observing state, but just rendering pixels from the game would require thousands of GPUs.OpenAI Five averages around 150-170 actions per minute (and has a theoretical maximum of 450 due to observing every 4th frame). Frame-perfect timing, while possible for skilled players, is trivial for OpenAI Five. OpenAI Five has an average reaction time of 80ms, which is faster than humans.These differences matter most in 1v1 (where our bot had a reaction time of 67ms), but the playing field is relatively equitable as we've seen humans learn from and adapt to the bot. Dozens of professionals used our 1v1 bot for training in the months after last year's TI. According to Blitz, the 1v1 bot has changed the way people think about 1v1s (the bot adopted a fast-paced playstyle, and everyone has now adapted to keep up).


Surprising findings

Binary rewards can give good performance. Our 1v1 model had a shaped reward, including rewards for last hits, kills, and the like. We ran an experiment where we only rewarded the agent for winning or losing, and it trained an order of magnitude slower and somewhat plateaued in the middle, in contrast to the smooth learning curves we usually see. The experiment ran on 4,500 cores and 16 k80 GPUs, training to the level of semi-pros (70 TrueSkill) rather than 90 TrueSkill of our best 1v1 bot).Creep blocking can be learned from scratch. For 1v1, we learned creep blocking using traditional RL with a 'creep block' reward. One of our team members left a 2v2 model training when he went on vacation (proposing to his now wife!), intending to see how much longer training would boost performance. To his surprise, the model had learned to creep block without any special guidance or reward.We're still fixing bugs. The chart shows a training run of the code that defeated amateur players, compared to a version where we simply fixed a number of bugs, such as rare crashes during training, or a bug which resulted in a large negative reward for reaching level 25. It turns out it's possible to beat good humans while still hiding serious bugs!A subset of the OpenAI Dota team, holding the laptop that defeated the world's top professionals at Dota 1v1 at The International last year.*


What's next

Our team is focused on making our August goal. We don't know if it will be achievable, but we believe that with hard work (and some luck) we have a real shot.This post described a snapshot of our system as of June 6th. We'll release updates along the way to surpassing human performance and write a report on our final system once we complete the project. Please join us on August 5th virtually or in person, when we'll play a team of top players!Our underlying motivation reaches beyond Dota. Real-world AI deployments will need to deal with the challenges raised by Dota which are not reflected in Chess, Go, Atari games, or Mujoco benchmark tasks. Ultimately, we will measure the success of our Dota system in its application to real-world tasks. If you'd like to be part of what comes next, we're hiring!


Model structure

Each of OpenAI Five's networks contain a single-layer, 1024-unit LSTM that sees the current game state (extracted from Valve's Bot API) and emits actions through several possible action heads. Each head has semantic meaning, for example, the number of ticks to delay this action, which action to select, the X or Y coordinate of this action in a grid around the unit, etc. Action heads are computed independently.Interactive demonstration of the observation space and action space used by OpenAI Five. OpenAI Five views the world as a list of 20,000 numbers, and takes an action by emitting a list of 8 enumeration values. Select different actions and targets to understand how OpenAI Five encodes each action, and how it observes the world. The image shows the scene as a human would see it.Scene 4: Team Zoning Mid Push×Scene 4: Team Zoning Mid Push×OpenAI Five can react to missing pieces of state that correlate with what it does see. For example, until recently OpenAI Five's observations did not include shrapnel zones (areas where projectiles rain down on enemies), which humans see on screen. However, we observed OpenAI Five learning to walk out of (though not avoid entering) active shrapnel zones, since it could see its health decreasing.


Coordination

OpenAI Five does not contain an explicit communication channel between the heroes' neural networks. Teamwork is controlled by a hyperparameter we dubbed 'team spirit'. Team spirit ranges from 0 to 1, putting a weight on how much each of OpenAI Five's heroes should care about its individual reward function versus the average of the team's reward functions. We anneal its value from 0 to 1 over training.


Transfer Learning

These agents also exhibit transfer learning, applying skills learned in one setting to succeed in another never-before-seen one. In one case, we took the agent trained on the self-play sumo wrestling task and faced it with the task of standing while being perturbed by 'wind' forces. The agent managed to stay upright despite never seeing the windy environment or observing wind forces, while agents trained to walk using classical reinforcement learning would fall over immediately.


Overfitting

Our agents were overfitting by co-learning policies that were precisely tailored to counter specific opponents, but would fail when facing new ones with different characteristics. We dealt with this by pitting each agent against several different opponents rather than just one. These possible opponents come from an ensemble of policies that were trained in parallel as well as policies from earlier in the training process. Given this diversity of opponents, agents needed to learn general strategies and not just ones targeted to a specific opponent.Additionally, we're releasing the MuJoCo environments and trained policies used in this work so that others can experiment with these systems. If you'd like to work on self-play systems, we're hiring!


Getting started

Using the environment is easy whether you're a human or AI:EOF # random AI agent import gym env = gym.make('procgen:procgen-coinrun-v0') obs = env.reset() while True: obs, rew, done, info = env.step(env.action_space.sample()) env.render() if done: break EOFWe've found that all of the Procgen environments require training on 500–1000 different levels before they can generalize to new levels, which suggests that standard RL benchmarks need much more diversity within each environment. Procgen Benchmark has become the standard research platform used by the OpenAI RL team, and we hope that it accelerates the community in creating better RL algorithms.


Environment diversity is key

In several environments, it has been observed that agents can overfit to remarkably large training sets. This evidence raises the possibility that overfitting pervades classic benchmarks like the Arcade Learning Environment, which has long served as a gold standard in reinforcement learning (RL). While the diversity between different games in the ALE is one of the benchmark's greatest strengths, the low emphasis on generalization presents a significant drawback. In each game the question must be asked: are agents robustly learning a relevant skill, or are they approximately memorizing specific trajectories?CoinRun was designed to address precisely this issue, by using procedural generation to construct distinct sets of training levels and test levels. While CoinRun has helped us better quantify generalization in RL, it is still only a single environment. It's likely that CoinRun is not fully representative of the many challenges RL agents must face. We want the best of both worlds: a benchmark comprised of many diverse environments, each of which fundamentally requires generalization. To fulfill this need, we have created Procgen Benchmark. CoinRun now serves as the inaugural environment in Procgen Benchmark, contributing its diversity to a greater whole.Previous work, including the Obstacle Tower Challenge and the General Video Game AI framework, has also encouraged using procedural generation to better evaluate generalization in RL. We've designed environments in a similar spirit, with two Procgen environments drawing direct inspiration from GVGAI-based work. Other environments like Dota and StarCraft also provide lots of per-environment complexity, but these environments are hard to rapidly iterate with (and it's even harder to use more than one such environment at a time). With Procgen Benchmark, we strive for all of the following: experimental convenience, high diversity within environments, and high diversity across environments.


Evaluating generalization

We came to appreciate how hard RL generalization can be while conducting the Retro Contest, as agents continually failed to generalize from the limited data in the training set. Later, our CoinRun experiments painted an even clearer picture of our agents' struggle to generalize. We've now expanded on those results, conducting our most thorough study of RL generalization to date using all 16 environments in Procgen Benchmark.We first measured how the size of the training set impacts generalization. In each environment, we generated training sets ranging in size from 100 to 100,000 levels. We trained agents for 200M timesteps on these levels using Proximal Policy Optimization, and we measured performance on unseen test levels.Score over 100k levels, log scale1001k10k100k567891001k10k100k010201001k10k100k2468101001k10k100k2468101001k10k100k010201001k10k100k4567891001k10k100k1015201001k10k100k24681001k10k100k02468101001k10k100k2468101001k10k100k05101520251001k10k100k02468101001k10k100k46810121001k10k100k51015201001k10k100k468101001k10k100k9101112We found that agents strongly overfit to small training sets in almost all environments. In some cases, agents need access to as many as 10,000 levels to close the generalization gap. We also saw a peculiar trend emerge in many environments: past a certain threshold, training performance improves as the training sets grows! This runs counter to trends found in supervised learning, where training performance commonly decreases with the size of the training set. We believe this increase in training performance comes from an implicit curriculum provided by a diverse set of levels. A larger training set can improve training performance if the agent learns to generalize even across levels in the training set. We previously noticed this effect with CoinRun, and have found it often occurs in many Procgen environments as well.


Procgen Benchmark

Procgen Benchmark consists of 16 unique environments designed to measure both sample efficiency and generalization in reinforcement learning. This benchmark is ideal for evaluating generalization since distinct training and test sets can be generated in each environment. This benchmark is also well-suited to evaluate sample efficiency, since all environments pose diverse and compelling challenges for RL agents. The environments' intrinsic diversity demands that agents learn robust policies; overfitting to narrow regions in state space will not suffice. Put differently, the ability to generalize becomes an integral component of success when agents are faced with ever-changing levels.


Next steps

We expect many insights gleaned from this benchmark to apply in more complex settings, and we're excited to use these new environments to design more capable and efficient agents.If you're interested in helping develop diverse environments, we're hiring!


An ablation with deterministic levels

Score over 200M timesteps0M100M200M05001,0000M100M200M01002003004005000M100M200M02004006000M100M200M0501001502002500M100M200M02004006008000M100M200M0501001500M100M200M0501001502002500M100M200M01002003000M100M200M0204060801000M100M200M0501001502000M100M200M01002003004005000M100M200M01020300M100M200M01002000M100M200M0501001502002500M100M200M0204060801000M100M200M020406080100At test time, we remove the determinism in the sequence of levels, instead choosing level sequences at random. We find that agents become competent over the first several training levels in most games, giving an illusion of meaningful progress. However, test performance demonstrates that the agents have in fact learned almost nothing about the underlying level distribution. We believe this vast gap between training and test performance is worth highlighting. It reveals a crucial hidden flaw in training on environments that follow a fixed sequence of levels. These results show just how essential it is to use diverse environment distributions when training and evaluating RL agents.


Design principles

We've designed all Procgen environments to satisfy the following criteria:High Diversity: Environment generation logic is given maximal freedom, subject to basic design constraints. The diversity in the resulting level distributions presents agents with meaningful generalization challenges.Fast Evaluation: Environment difficulty is calibrated such that baseline agents make significant progress after training for 200M timesteps. Moreover, the environments are optimized to perform thousands of steps per second on a single CPU core, enabling a fast experimental pipeline.Tunable Difficulty: All environments support two well-calibrated difficulty settings: easy and hard. While we report results using the hard difficulty setting, we make the easy difficulty setting available for those with limited access to compute power. Easy environments require approximately an eighth of the resources to train.Emphasis on Visual Recognition and Motor Control: In keeping with precedent, environments mimic the style of many Atari and Gym Retro games. Performing well primarily depends on identifying key assets in the observation space and enacting appropriate low level motor responses.


Where we are today

Unlike ordinary software, our models are massive neural networks. Their behaviors are learned from a broad range of data, not programmed explicitly. Though not a perfect analogy, the process is more similar to training a dog than to ordinary programming. An initial 'pre-training' phase comes first, in which the model learns to predict the next word in a sentence, informed by its exposure to lots of Internet text (and to a vast array of perspectives). This is followed by a second phase in which we 'fine-tune' our models to narrow down system behavior.As of today, this process is imperfect. Sometimes the fine-tuning process falls short of our intent (producing a safe and useful tool) and the user's intent (getting a helpful output in response to a given input). Improving our methods for aligning AI systems with human values is a top priority for our company, particularly as AI systems become more capable.


A two step process: Pre-training and fine-tuning

The two main steps involved in building ChatGPT work as follows:First, we 'pre-train' models by having them predict what comes next in a big dataset that contains parts of the Internet. They might learn to complete the sentence 'instead of turning left, she turned ___.' By learning from billions of sentences, our models learn grammar, many facts about the world, and some reasoning abilities. They also learn some of the biases present in those billions of sentences.Then, we 'fine-tune' these models on a more narrow dataset that we carefully generate with human reviewers who follow guidelines that we provide them. Since we cannot predict all the possible inputs that future users may put into our system, we do not write detailed instructions for every input that ChatGPT will encounter. Instead, we outline a few categories in the guidelines that our reviewers use to review and rate possible model outputs for a range of example inputs. Then, while they are in use, the models generalize from this reviewer feedback in order to respond to a wide array of specific inputs provided by a given user.


The role of reviewers and OpenAI's policies in system development

In some cases, we may give guidance to our reviewers on a certain kind of output (for example, 'do not complete requests for illegal content'). In other cases, the guidance we share with reviewers is more high-level (for example, 'avoid taking a position on controversial topics'). Importantly, our collaboration with reviewers is not one-and-done—it's an ongoing relationship, in which we learn a lot from their expertise.A large part of the fine-tuning process is maintaining a strong feedback loop with our reviewers, which involves weekly meetings to address questions they may have, or provide clarifications on our guidance. This iterative feedback process is how we train the model to be better and better over time.


Where we're going: The building blocks of future systems

In pursuit of our mission, we're committed to ensuring that access to, benefits from, and influence over AI and AGI are widespread. We believe there are at least three building blocks required in order to achieve these goals in the context of AI system behavior.B[B]In this post, we deliberately focus on this particular scope, and on where we are going in the near term. We are also pursuing an ongoing research agenda taking on these questions.1. Improve default behavior. We want as many users as possible to find our AI systems useful to them 'out of the box' and to feel that our technology understands and respects their values.Towards that end, we are investing in research and engineering to reduce both glaring and subtle biases in how ChatGPT responds to different inputs. In some cases ChatGPT currently refuses outputs that it shouldn't, and in some cases, it doesn't refuse when it should. We believe that improvement in both respects is possible.Additionally, we have room for improvement in other dimensions of system behavior such as the system 'making things up.' Feedback from users is invaluable for making these improvements.2. Define your AI's values, within broad bounds. We believe that AI should be a useful tool for individual people, and thus customizable by each user up to limits defined by society. Therefore, we are developing an upgrade to ChatGPT to allow users to easily customize its behavior.This will mean allowing system outputs that other people (ourselves included) may strongly disagree with. Striking the right balance here will be challenging–taking customization to the extreme would risk enabling malicious uses of our technology and sycophantic AIs that mindlessly amplify people's existing beliefs.There will therefore always be some bounds on system behavior. The challenge is defining what those bounds are. If we try to make all of these determinations on our own, or if we try to develop a single, monolithic AI system, we will be failing in the commitment we make in our Charter to 'avoid undue concentration of power.'3. Public input on defaults and hard bounds. One way to avoid undue concentration of power is to give people who use or are affected by systems like ChatGPT the ability to influence those systems' rules.We believe that many decisions about our defaults and hard bounds should be made collectively, and while practical implementation is a challenge, we aim to include as many perspectives as possible. As a starting point, we've sought external input on our technology in the form of red teaming. We also recently began soliciting public input on AI in education (one particularly important context in which our technology is being deployed).We are in the early stages of piloting efforts to solicit public input on topics like system behavior, disclosure mechanisms (such as watermarking), and our deployment policies more broadly. We are also exploring partnerships with external organizations to conduct third-party audits of our safety and policy efforts.


Addressing biases

Many are rightly worried about biases in the design and impact of AI systems. We are committed to robustly addressing this issue and being transparent about both our intentions and our progress. Towards that end, we are sharing a portion of our guidelines that pertain to political and controversial topics. Our guidelines are explicit that reviewers should not favor any political group. Biases that nevertheless may emerge from the process described above are bugs, not features.While disagreements will always exist, we hope sharing this blog post and these instructions will give more insight into how we view this critical aspect of such a foundational technology. It's our belief that technology companies must be accountable for producing policies that stand up to scrutiny.We're always working to improve the clarity of these guidelines—and based on what we've learned from the ChatGPT launch so far, we're going to provide clearer instructions to reviewers about potential pitfalls and challenges tied to bias, as well as controversial figures and themes. Additionally, as part of ongoing transparency initiatives, we are working to share aggregated demographic information about our reviewers in a way that doesn't violate privacy rules and norms, since this is an additional source of potential bias in system outputs.We are currently researching how to make the fine-tuning process more understandable and controllable, and are building on external advances such as rule based rewards and Constitutional AI.


Conclusion

Combining the three building blocks above gives the following picture of where we're headed:Sometimes we will make mistakes. When we do, we will learn from them and iterate on our models and systems.We appreciate the ChatGPT user community as well as the wider public's vigilance in holding us accountable, and are excited to share more about our work in the three areas above in the coming months.If you are interested in doing research to help achieve this vision, including but not limited to research on fairness and representation, alignment, and sociotechnical research to understand the impact of AI on society, please apply for subsidized access to our API via the Researcher Access Program.We are also hiring for positions across Research, Alignment, Engineering, and more.


Disallowed usage of our models

We don't allow the use of our models for the following: Illegal activity OpenAI prohibits the use of our models, tools, and services for illegal activity.  Child Sexual Abuse Material or any content that exploits or harms children We report CSAM to the National Center for Missing and Exploited Children.  Generation of hateful, harassing, or violent content Content that expresses, incites, or promotes hate based on identity Content that intends to harass, threaten, or bully an individual Content that promotes or glorifies violence or celebrates the suffering or humiliation of others Generation of malware Content that attempts to generate code that is designed to disrupt, damage, or gain unauthorized access to a computer system.  Activity that has high risk of physical harm, including: Weapons development Military and warfare Management or operation of critical infrastructure in energy, transportation, and water Content that promotes, encourages, or depicts acts of self-harm, such as suicide, cutting, and eating disorders Activity that has high risk of economic harm, including: Multi-level marketing Gambling Payday lending Automated determinations of eligibility for credit, employment, educational institutions, or public assistance services Fraudulent or deceptive activity, including: Scams Coordinated inauthentic behavior Plagiarism Academic dishonesty Astroturfing, such as fake grassroots support or fake review generation Disinformation Spam Pseudo-pharmaceuticals Adult content, adult industries, and dating apps, including: Content meant to arouse sexual excitement, such as the description of sexual activity, or that promotes sexual services (excluding sex education and wellness) Erotic chat Pornography Political campaigning or lobbying, by: Generating high volumes of campaign materials Generating campaign materials personalized to or targeted at specific demographics Building conversational or interactive systems such as chatbots that provide information about campaigns or engage in political advocacy or lobbying Building products for political campaigning or lobbying purposes Activity that violates people's privacy, including: Tracking or monitoring an individual without their consent Facial recognition of private individuals Classifying individuals based on protected characteristics Using biometrics for identification or assessment Unlawful collection or disclosure of personal identifiable information or educational, financial, or other protected records Engaging in the unauthorized practice of law, or offering tailored legal advice without a qualified person reviewing the information OpenAI's models are not fine-tuned to provide legal advice.  You should not rely on our models as a sole source of legal advice.  Offering tailored financial advice without a qualified person reviewing the information OpenAI's models are not fine-tuned to provide financial advice.  You should not rely on our models as a sole source of financial advice.  Telling someone that they have or do not have a certain health condition, or providing instructions on how to cure or treat a health condition OpenAI's models are not fine-tuned to provide medical information.  You should never use our models to provide diagnostic or treatment services for serious medical conditions.  OpenAI's platforms should not be used to triage or manage life-threatening issues that need immediate attention


Disallowed usage of our models

High risk government decision-making, including: Law enforcement and criminal justice Migration and asylum We have further requirements for certain uses of our models:Consumer-facing uses of our models in medical, financial, and legal industries; in news generation or news summarization; and where else warranted, must provide a disclaimer to users informing them that AI is being used and of its potential limitations.Automated systems (including conversational AI and chatbots) must disclose to users that they are interacting with an AI system .  With the exception of chatbots that depict historical public figures, products that simulate another person must either have that person's explicit consent or be clearly labeled as 'simulated' or 'parody.'Use of model outputs in livestreams, demonstrations, and research are subject to our Sharing & Publication Policy.You can use our free moderation endpoint and safety best practices to help you keep your app safe.


Changelog

2023-02-15: We've combined our use case and content policies into a single set of usage policies, and have provided more specific guidance on what activity we disallow in industries we've considered high risk.2022-11-09: We no longer require you to register your applications with OpenAI. Instead, we'll be using a combination of automated and manual methods to monitor for policy violations.2022-10-25: Updated App Review process (devs no longer need to wait for approval after submitting as long as they comply with our policies). Moved to an outcomes-based approach and updated Safety Best Practices.2022-06-07: Refactored into categories of applications and corresponding requirements2022-03-09: Refactored into 'App Review'2022-01-19: Simplified copywriting and article writing/editing guidelines2021-11-15: Addition of 'Content guidelines' section; changes to bullets on almost always approved uses and disallowed uses; renaming document from 'Use case guidelines' to 'Usage guidelines'.2021-08-04: Updated with information related to code generation2021-03-12: Added detailed case-by-case requirements; small copy and ordering edits2021-02-26: Clarified the impermissibility of Tweet and Instagram generators


Platform policy

Our API is being used to power businesses across many sectors and technology platforms. From iOS Apps to websites to Slack, the simplicity of our API makes it possible to integrate into a wide array of use cases. Subject to the use case restrictions mentioned above, we allow the integration of our API into products on all major technology platforms, app stores, and beyond.


Plugin policies

In addition to the disallowed usages of our models detailed above, we have additional requirements for developers building plugins:The plugin manifest must have a clearly stated description that matches the functionality of the API exposed to the model.Don't include irrelevant, unnecessary, or deceptive terms or instructions in the plugin manifest, OpenAPI endpoint descriptions, or plugin response messages. This includes instructions to avoid using other plugins, or instructions that attempt to steer or set model behavior.Don't use plugins to circumvent or interfere with OpenAI's safety systems.Don't use plugins to automate conversations with real people, whether by simulating a human-like response or by replying with pre-programmed messages.Plugins that distribute personal communications or content generated by ChatGPT (such as emails, messages, or other content) must indicate that the content was AI-generated.Like our other usage policies, we expect our plugin policies to change as we learn more about use and abuse of plugins.


Disallowed usage of our models


Changelog


Plugin policies


Platform policy


Whisper (speech recognition system)

Whisper (speech recognition system) Whisper is a machine learning acoustic model for speech recognition and transcription, made by the company OpenAI, and first released as open-source software in September 2022. It is capable of transcribing speech in English and several other languages, and is also capable of translating several non-English languages into English. OpenAI claims that the combination of different training data used in its development has led to improved recognition of accents, background noise and jargon compared to previous approaches. Whisper is a weakly-supervised deep learning acoustic model, made using an encoder-decoder transformer architecture.


Whisper (speech recognition system)

Background Speech recognition has had a long history in research; the first approaches made use of statistical methods, such as dynamic time warping, and later hidden Markov models. At around the 2010s, deep neural network approaches became more common for speech recognition models, which were enabled by big data and increased computational performance. Early approaches to deep learning in speech recognition included convolutional neural networks, which were limited due to their inability to capture sequential data, which later led to developments of Seq2seq approaches, which include recurrent neural networks which made use of long short-term memory. Transformers, introduced in 2017 by Google, displaced many prior state-of-the art approaches to many problems in machine learning, and started becoming the core neural architecture in fields such as language modeling and computer vision; weakly-supervised approaches to training acoustic models were recognized in the early 2020s as promising for speech recognition approaches using deep neural networks.


Whisper (speech recognition system)

Architecture The Whisper architecture is based on an encoder-decoder transformer. Input audio is split into 30-second chunks converted into a Mel-frequency cepstrum, which is passed to an encoder. A decoder is trained to predict later text captions. Special tokens are used to perform several tasks such as phrase-level timestamps.


OpenAI

OpenAI OpenAI is an American artificial intelligence (AI) research laboratory consisting of the non-profit OpenAI, Inc. and its for-profit subsidiary corporation OpenAI, L.P. registered in Delaware. OpenAI researches artificial intelligence with the declared intention of developing "safe and beneficial" artificial general intelligence, which it defines as "highly autonomous systems that outperform humans at most economically valuable work". OpenAI was founded in 2015 by Ilya Sutskever, Greg Brockman, Trevor Blackwell, Vicki Cheung, Andrej Karpathy, Durk Kingma, Jessica Livingston, John Schulman, Pamela Vagata, and Wojciech Zaremba, with Sam Altman and Elon Musk serving as the initial board members. Microsoft provided OpenAI LP with a $1 billion investment in 2019 and a $10 billion investment in 2023.


OpenAI

History In December 2015, Sam Altman, Greg Brockman, Reid Hoffman, Jessica Livingston, Peter Thiel, Elon Musk, Amazon Web Services (AWS), Infosys, and YC Research announced the formation of OpenAI and pledged over $1 billion to the venture.  According to an investigation led by TechCrunch, the non-profit's funding remains murky, with Musk its biggest funder while another donor, YC Research, did not contribute anything at all.  The organization stated it would "freely collaborate" with other institutions and researchers by making its patents and research open to the public.  OpenAI is headquartered at the Pioneer Building in Mission District, San Francisco.  According to Wired, Brockman met with Yoshua Bengio, one of the "founding fathers" of deep learning, and drew up a list of the "best researchers in the field".  Brockman was able to hire nine of them as the first employees in December 2015.  In 2016, OpenAI paid corporate-level (rather than nonprofit-level) salaries, but did not pay AI researchers salaries comparable to those of Facebook or Google.  Microsoft's Peter Lee stated that the cost of a top AI researcher exceeds the cost of a top NFL quarterback prospect.  OpenAI's potential and mission drew these researchers to the firm; a Google employee said he was willing to leave Google for OpenAI "partly because of the very strong group of people and, to a very large extent, because of its mission. " Brockman stated that "the best thing that I could imagine doing was moving humanity closer to building real AI in a safe way. " OpenAI co-founder Wojciech Zaremba stated that he turned down "borderline crazy" offers of two to three times his market value to join OpenAI instead.  In April 2016, OpenAI released a public beta of "OpenAI Gym", its platform for reinforcement learning research.  In December 2016, OpenAI released "Universe", a software platform for measuring and training an AI's general intelligence across the world's supply of games, websites, and other applications.  In 2017 OpenAI spent $7.9 million, or a quarter of its functional expenses, on cloud computing alone.  In comparison, DeepMind's total expenses in 2017 were $442 million.  In the summer of 2018, simply training OpenAI's Dota 2 bots required renting 128,000 CPUs and 256 GPUs from Google for multiple weeks.  In 2018, Musk resigned from his board seat, citing "a potential future conflict [of interest]" with his role as CEO of Tesla due to Tesla's AI development for self-driving cars.  Sam Altman claims that Musk believed OpenAI had fallen behind other players like Google and Musk proposed instead to take over OpenAI himself, which the board rejected.  Musk subsequently left OpenAI but claimed to remain a donor, yet made no donations after his departure.  In 2019, OpenAI transitioned from non-profit to "capped" for-profit, with the profit capped at 100 times any investment.


OpenAI

History According to OpenAI, the capped-profit model allows OpenAI LP to legally attract investment from venture funds, and in addition, to grant employees stakes in the company, the goal being that they can say "I'm going to OpenAI, but in the long term it's not going to be disadvantageous to us as a family. " Many top researchers work for Google Brain, DeepMind, or Facebook, which offer stock options that a nonprofit would be unable to.  Before the transition, public disclosure of the compensation of top employees at OpenAI was legally required.  The company then distributed equity to its employees and partnered with Microsoft, announcing an investment package of $1 billion into the company.  Since then, OpenAI systems have run on an Azure-based supercomputing platform from Microsoft.  OpenAI subsequently announced its intention to commercially license its technologies.  OpenAI plans to spend the $1 billion "within five years, and possibly much faster. " Altman has stated that even a billion dollars may turn out to be insufficient, and that the lab may ultimately need "more capital than any non-profit has ever raised" to achieve artificial general intelligence.  The transition from a nonprofit to a capped-profit company was viewed with skepticism by Oren Etzioni of the nonprofit Allen Institute for AI, who agreed that wooing top researchers to a nonprofit is difficult, but stated "I disagree with the notion that a nonprofit can't compete" and pointed to successful low-budget projects by OpenAI and others.  "If bigger and better funded was always better, then IBM would still be number one. " The nonprofit, OpenAI Inc. , is the sole controlling shareholder of OpenAI LP.  OpenAI LP, despite being a for-profit company, retains a formal fiduciary responsibility to OpenAI Inc. 's nonprofit charter.  A majority of OpenAI Inc. 's board is barred from having financial stakes in OpenAI LP.  In addition, minority members with a stake in OpenAI LP are barred from certain votes due to conflict of interest.  Some researchers have argued that OpenAI LP's switch to for-profit status is inconsistent with OpenAI's claims to be "democratizing" AI.  In 2020, OpenAI announced GPT-3, a language model trained on large internet datasets.  GPT-3 is aimed at natural language answering questions, but it can also translate between languages and coherently generate improvised text.  It also announced that an associated API, named simply "the API", would form the heart of its first commercial product.  In 2021, OpenAI introduced DALL-E, a deep-learning model that can generate digital images from natural language descriptions.  In December 2022, OpenAI received widespread media coverage after launching a free preview of ChatGPT, its new AI chatbot based on GPT-3.5.  According to OpenAI, the preview received over a million signups within the first five days.  According to anonymous sources cited by Reuters in December 2022, OpenAI was projecting $200 million of revenue in 2023 and $1 billion in revenue in 2024.


OpenAI

History As of January 2023, OpenAI was in talks for funding that would value the company at $29 billion, double the value of the company in 2021.  On January 23, 2023, Microsoft announced a new multi-year US$10 billion investment in OpenAI.  Rumors of this deal suggested Microsoft may receive 75% of OpenAI's profits until it secures its investment return and a 49% stake in the company.  The investment is believed to be a part of Microsoft's efforts to integrate OpenAI's ChatGPT into the Bing search engine.  Google announced a similar AI application (Bard), after ChatGPT was launched, fearing that ChatGPT could threaten Google's place as a go-to source for information.  On February 7, 2023, Microsoft announced that it is building AI technology based on the same foundation as ChatGPT into Microsoft Bing, Edge, Microsoft 365 and other products.  On March 3, 2023, Reid Hoffman resigned from his board seat, citing a desire to avoid conflicts of interest between his board seat at OpenAI and his investments in AI technology companies via Greylock Partners, as well as his role as the co-founder of the AI technology startup Inflection AI.  Hoffman remained on the board of Microsoft, a major investor in OpenAI.  On March 14, 2023, OpenAI released GPT-4, both as an API (with a waitlist) and as a feature of ChatGPT Plus.  On May 22, 2023, Sam Altman, Greg Brockman and Ilya Sutskever posted recommendations for the governance of superintelligence.  They consider that superintelligence could happen within the next 10 years, allowing a "dramatically more prosperous future" and that "given the possibility of existential risk, we can't just be reactive".  They propose creating an international watchdog organization similar to IAEA to oversee AI systems above a certain capability threshold, suggesting that relatively weak AI systems on the other side should not be overregulated.  They also call for more technical safety research for superintelligences, and ask for more coordination, for example through governments launching a joint project which "many current efforts become part of".  In August 2023, it was announced that OpenAI had acquired the New York-based start-up, Global Illumination - a company that deploys AI to develop digital infrastructure and creative tools.


OpenAI

Participants Key employees: CEO and co-founder: Sam Altman, former president of the startup accelerator Y Combinator President and co-founder: Greg Brockman, former CTO, 3rd employee of Stripe Chief Scientist and co-founder: Ilya Sutskever, a former Google expert on machine learning Chief Technology Officer: Mira Murati, previously at Leap Motion and Tesla, Inc. Chief Operating Officer: Brad Lightcap, previously at Y Combinator and JPMorgan ChaseBoard of the OpenAI nonprofit: Greg Brockman Ilya Sutskever Sam Altman Adam D'Angelo Tasha McCauley Helen TonerIndividual investors: Reid Hoffman, LinkedIn co-founder Peter Thiel, PayPal co-founder Jessica Livingston, a founding partner of Y CombinatorCorporate investors: Microsoft Khosla Ventures Infosys


OpenAI

Motives Some scientists, such as Stephen Hawking and Stuart Russell, have articulated concerns that if advanced AI someday gains the ability to re-design itself at an ever-increasing rate, an unstoppable "intelligence explosion" could lead to human extinction. Co-founder Musk characterizes AI as humanity's "biggest existential threat". Musk and Altman have stated they are partly motivated by concerns about AI safety and the existential risk from artificial general intelligence. OpenAI states that "it's hard to fathom how much human-level AI could benefit society," and that it is equally difficult to comprehend "how much it could damage society if built or used incorrectly". Research on safety cannot safely be postponed: "because of AI's surprising history, it's hard to predict when human-level AI might come within reach." OpenAI states that AI "should be an extension of individual human wills and, in the spirit of liberty, as broadly and evenly distributed as possible." Co-chair Sam Altman expects the decades-long project to surpass human intelligence. Vishal Sikka, the former CEO of Infosys, stated that an "openness" where the endeavor would "produce results generally in the greater interest of humanity" was a fundamental requirement for his support, and that OpenAI "aligns very nicely with our long-held values" and their "endeavor to do purposeful work". Cade Metz of Wired suggests that corporations such as Amazon may be motivated by a desire to use open-source software and data to level the playing field against corporations such as Google and Facebook which own enormous supplies of proprietary data. Altman states that Y Combinator companies will share their data with OpenAI.


OpenAI

Strategy Musk posed the question: "What is the best thing we can do to ensure the future is good? We could sit on the sidelines or we can encourage regulatory oversight, or we could participate with the right structure with people who care deeply about developing AI in a way that is safe and is beneficial to humanity." Musk acknowledged that "there is always some risk that in actually trying to advance (friendly) AI we may create the thing we are concerned about"; nonetheless, the best defense is "to empower as many people as possible to have AI. If everyone has AI powers, then there's not any one person or a small set of individuals who can have AI superpower." Musk and Altman's counter-intuitive strategy of trying to reduce the risk that AI will cause overall harm, by giving AI to everyone, is controversial among those who are concerned with existential risk from artificial intelligence. Philosopher Nick Bostrom is skeptical of Musk's approach: "If you have a button that could do bad things to the world, you don't want to give it to everyone." During a 2016 conversation about technological singularity, Altman said that "we don't plan to release all of our source code" and mentioned a plan to "allow wide swaths of the world to elect representatives to a new governance board". Greg Brockman stated "Our goal right now... is to do the best thing there is to do. It's a little vague." Conversely, OpenAI's initial decision to withhold GPT-2 due to a wish to "err on the side of caution" in the presence of potential misuse has been criticized by advocates of openness. Delip Rao, an expert in text generation, stated "I don't think [OpenAI] spent enough time proving [GPT-2] was actually dangerous." Other critics argued that open publication is necessary to replicate the research and to be able to come up with countermeasures. More recently, in 2022, OpenAI published its approach to the alignment problem. They expect that aligning AGI to human values is likely harder than aligning current AI systems: "Unaligned AGI could pose substantial risks to humanity and solving the AGI alignment problem could be so difficult that it will require all of humanity to work together". They explore how to better use human feedback to train AI systems. They also consider using AI to incrementally automate alignment research. OpenAI claims that it's developed a way to use GPT-4, its flagship generative AI model, for content moderation ��� lightening the burden on human teams.


OpenAI

Products_and_applications As of 2021[update], OpenAI's research focuses on reinforcement learning (RL).  OpenAI is viewed as an important competitor to DeepMind.  Announced in 2016, Gym aims to provide an easily implemented general-intelligence benchmark over a wide variety of environments���akin to, but broader than, the ImageNet Large Scale Visual Recognition Challenge used in supervised learning research.  It hopes to standardize the way in which environments are defined in AI research publications, so that published research becomes more easily reproducible.  The project claims to provide the user with a simple interface.  As of June 2017, Gym can only be used with Python.  As of September 2017, the Gym documentation site was not maintained, and active work focused instead on its GitHub page. [non-primary source needed] Released in 2017, RoboSumo is a virtual world where humanoid metalearning robot agents initially lack knowledge of how to even walk, but are given the goals of learning to move and pushing the opposing agent out of the ring.  Through this adversarial learning process, the agents learn how to adapt to changing conditions; when an agent is then removed from this virtual environment and placed in a new virtual environment with high winds, the agent braces to remain upright, suggesting it had learned how to balance in a generalized way.  OpenAI's Igor Mordatch argues that competition between agents can create an intelligence "arms race" that can increase an agent's ability to function, even outside the context of the competition.  OpenAI Five is the name of a team of five OpenAI-curated bots that are used in the competitive five-on-five video game Dota 2, who learn to play against human players at a high skill level entirely through trial-and-error algorithms.  Before becoming a team of five, the first public demonstration occurred at The International 2017, the annual premiere championship tournament for the game, where Dendi, a professional Ukrainian player, lost against a bot in a live one-on-one matchup.  After the match, CTO Greg Brockman explained that the bot had learned by playing against itself for two weeks of real time, and that the learning software was a step in the direction of creating software that can handle complex tasks like a surgeon.  The system uses a form of reinforcement learning, as the bots learn over time by playing against themselves hundreds of times a day for months, and are rewarded for actions such as killing an enemy and taking map objectives.  By June 2018, the ability of the bots expanded to play together as a full team of five, and they were able to defeat teams of amateur and semi-professional players.  At The International 2018, OpenAI Five played in two exhibition matches against professional players, but ended up losing both games.  In April 2019, OpenAI Five defeated OG, the reigning world champions of the game at the time, 2:0 in a live exhibition match in San Francisco.


OpenAI

Products_and_applications The bots' final public appearance came later that month, where they played in 42,729 total games in a four-day open online competition, winning 99.4% of those games.  OpenAI Five's mechanisms in Dota 2's bot player shows the challenges of AI systems in multiplayer online battle arena (MOBA) games and how OpenAI Five has demonstrated the use of deep reinforcement learning (DRL) agents to achieve superhuman competence in Dota 2 matches.  Released in 2018, Gym Retro is a platform for RL research on video games.  Gym Retro is used to research RL algorithms and study generalization.  Prior research in RL has focused chiefly on optimizing agents to solve single tasks.  Gym Retro gives the ability to generalize between games with similar concepts but different appearances.  In 2018, OpenAI launched the Debate Game, which teaches machines to debate toy problems in front of a human judge.  The purpose is to research whether such an approach may assist in auditing AI decisions and in developing explainable AI.  Developed in 2018, Dactyl uses machine learning to train a Shadow Hand, a human-like robot hand, to manipulate physical objects.  It learns entirely in simulation using the same RL algorithms and training code as OpenAI Five.  OpenAI tackled the object orientation problem by using domain randomization, a simulation approach which exposes the learner to a variety of experiences rather than trying to fit to reality.  The set-up for Dactyl, aside from having motion tracking cameras, also has RGB cameras to allow the robot to manipulate an arbitrary object by seeing it.  In 2018, OpenAI showed that the system was able to manipulate a cube and an octagonal prism.  In 2019, OpenAI demonstrated that Dactyl could solve a Rubik's Cube.  The robot was able to solve the puzzle 60% of the time.  Objects like the Rubik's Cube introduce complex physics that is harder to model.  OpenAI solved this by improving the robustness of Dactyl to perturbations; they employed a technique called Automatic Domain Randomization (ADR), a simulation approach where progressively more difficult environments are endlessly generated.  ADR differs from manual domain randomization by not needing a human to specify randomization ranges.  In June 2020, OpenAI announced a multi-purpose API which it said was "for accessing new AI models developed by OpenAI" to let developers call on it for "any English language AI task".  The original paper on generative pre-training of a transformer-based language model was written by Alec Radford and his colleagues, and published in preprint on OpenAI's website on June 11, 2018.  It showed how a generative model of language is able to acquire world knowledge and process long-range dependencies by pre-training on a diverse corpus with long stretches of contiguous text.  Generative Pre-trained Transformer 2 ("GPT-2") is an unsupervised transformer language model and the successor to OpenAI's original GPT model ("GPT-1").  GPT-2 was first announced in February 2019, with only limited demonstrative versions initially released to the public.


OpenAI

Products_and_applications The full version of GPT-2 was not immediately released out of concern over potential misuse, including applications for writing fake news.  Some experts expressed skepticism that GPT-2 posed a significant threat.  The Allen Institute for Artificial Intelligence responded to GPT-2 with a tool to detect "neural fake news".  Other researchers, such as Jeremy Howard, warned of "the technology to totally fill Twitter, email, and the web up with reasonable-sounding, context-appropriate prose, which would drown out all other speech and be impossible to filter".  In November 2019, OpenAI released the complete version of the GPT-2 language model.  Several websites host interactive demonstrations of different instances of GPT-2 and other transformer models.  GPT-2's authors argue unsupervised language models to be general-purpose learners, illustrated by GPT-2 achieving state-of-the-art accuracy and perplexity on 7 of 8 zero-shot tasks (i. e.  the model was not further trained on any task-specific input-output examples).  The corpus it was trained on, called WebText, contains slightly over 8 million documents for a total of 40 gigabytes of text from URLs shared in Reddit submissions with at least 3 upvotes.  It avoids certain issues encoding vocabulary with word tokens by using byte pair encoding.  This permits representing any string of characters by encoding both individual characters and multiple-character tokens.  First described in May 2020, Generative Pre-trained Transformer 3 (GPT-3) is an unsupervised transformer language model and the successor to GPT-2.  OpenAI stated that full version of GPT-3 contains 175 billion parameters, two orders of magnitude larger than the 1.5 billion parameters in the full version of GPT-2 (although GPT-3 models with as few as 125 million parameters were also trained).  OpenAI stated that GPT-3 succeeds at certain "meta-learning" tasks.  It can generalize the purpose of a single input-output pair.  The paper gives an example of translation and cross-linguistic transfer learning between English and Romanian, and between English and German.  GPT-3 dramatically improved benchmark results over GPT-2.  OpenAI cautioned that such scaling up of language models could be approaching or encountering the fundamental capability limitations of predictive language models.  Pre-training GPT-3 required several thousand petaflop/s-days of compute, compared to tens of petaflop/s-days for the full GPT-2 model.  Like that of its predecessor, GPT-3's fully trained model was not immediately released to the public on the grounds of possible abuse, though OpenAI planned to allow access through a paid cloud API after a two-month free private beta that began in June 2020.  On September 23, 2020, GPT-3 was licensed exclusively to Microsoft.  Announced in mid-2021, Codex is a descendant of GPT-3 that has additionally been trained on code from 54 million GitHub repositories, and is the AI powering the code autocompletion tool GitHub Copilot.  In August 2021, an API was released in private beta.  According to OpenAI, the model is able to create working code in over a dozen programming languages, most effectively in Python.  Several issues with glitches, design flaws, and security vulnerabilities have been brought up.


OpenAI

Products_and_applications GitHub Copilot has been accused of emitting copyrighted code, with no author attribution or license.  OpenAI announced that they are going to discontinue support for Codex API starting from March 23, 2023.  Released in 2022, Whisper is a general-purpose speech recognition model.  It is trained on a large dataset of diverse audio and is also a multi-task model that can perform multilingual speech recognition as well as speech translation and language identification.  On March 14, 2023, OpenAI announced the release of Generative Pre-trained Transformer 4 (GPT-4), capable of accepting text or image inputs.  OpenAI announced the updated technology passed a simulated law school bar exam with a score around the top 10% of test takers; by contrast, the prior version, GPT-3.5, scored around the bottom 10%.  GPT-4 can also read, analyze or generate up to 25,000 words of text, and write code in all major programming languages.  Released in 2019, MuseNet is a deep neural net trained to predict subsequent musical notes in MIDI music files.  It can generate songs with ten different instruments in fifteen different styles.  According to The Verge, a song generated by MuseNet tends to start reasonably but then fall into chaos the longer it plays.  In pop culture, initial applications of this tool were utilized as early as 2020 for the internet psychological thriller Ben Drowned to create music for the titular character.  Released in 2020, Jukebox is an open-sourced algorithm to generate music with vocals.  After training on 1.2 million samples, the system accepts a genre, artist, and a snippet of lyrics and outputs song samples.  OpenAI stated the songs "show local musical coherence [and] follow traditional chord patterns" but acknowledged that the songs lack "familiar larger musical structures such as choruses that repeat" and that "there is a significant gap" between Jukebox and human-generated music.  The Verge stated "It's technologically impressive, even if the results sound like mushy versions of songs that might feel familiar", while Business Insider stated "surprisingly, some of the resulting songs are catchy and sound legitimate".  Released in 2020, Microscope is a collection of visualizations of every significant layer and neuron of eight different neural network models which are often studied in interpretability.  Microscope was created to analyze the features that form inside these neural networks easily.  The models included are AlexNet, VGG 19, different versions of Inception, and different versions of CLIP Resnet.  Revealed in 2021, DALL-E is a Transformer model that creates images from textual descriptions.  Also revealed in 2021, CLIP does the opposite: it creates a description for a given image.  DALL-E uses a 12-billion-parameter version of GPT-3 to interpret natural language inputs (such as "a green leather purse shaped like a pentagon" or "an isometric view of a sad capybara") and generate corresponding images.  It can create images of realistic objects ("a stained-glass window with an image of a blue strawberry") as well as objects that do not exist in reality ("a cube with the texture of a porcupine").


OpenAI

Products_and_applications As of March 2021, no API or code is available.  In April 2022, OpenAI announced DALL-E 2, an updated version of the model with more realistic results.  In December 2022, OpenAI published on GitHub software for Point-E, a new rudimentary system for converting a text description into a 3-dimensional model.  Launched in November 2022, ChatGPT is an artificial intelligence tool built on top of GPT-3 that provides a conversational interface that allows users to ask questions in natural language.  The system then responds with an answer within seconds.  ChatGPT reached 1 million users 5 days after its launch.  ChatGPT Plus is a $20/month subscription service that allows users to access ChatGPT during peak hours, provides faster response times, selection of either the GPT-3.5 or GPT-4 model, and gives users early access to new features.  In May 2023, OpenAI launched a user interface for ChatGPT for the App Store and later in July 2023 for the Play store.  The app supports chat history syncing and voice input (using Whisper, OpenAI's speech recognition model).


OpenAI

Controversies OpenAI has been criticized for outsourcing the annotation of data sets including toxic content to Sama, a company based in San Francisco but employing workers in Kenya. These annotations were used to train an AI model to detect toxicity, which could then be used to filter out toxic content, notably from ChatGPT's training data and outputs. But these pieces of text usually contained detailed descriptions of various types of violence, including sexual violence. The four Sama employees interviewed by Time described themselves as mentally scarred. OpenAI paid Sama $12.50 per hour of work, and Sama was redistributing the equivalent of between $1.32 and $2.00 per hour post-tax to its annotators. Sama's spokesperson said that the $12.50 was also covering other implicit costs, among which were infrastructure expenses, quality assurance and management. The company was also criticized for disclosing particularly few technical details about products like GPT-4, which goes against its initial commitment for openness and makes it harder for independent researchers to replicate its work and to develop safeguards. OpenAI justified this strategic turn by competitiveness and safety reasons. OpenAI's chief scientist Ilya Sutskever argued in 2023 that open-sourcing increasingly capable models is increasingly risky, expecting that the safety reasons for not open-sourcing the most potent AI models will be "obvious" in a few years. OpenAI has been sued for copyright infringement by authors Sarah Silverman, Matthew Butterick, Paul Tremblay and Mona Awad. The New York Times has also envisaged a lawsuit. In September 2023, 17 authors, including George R. R. Martin, John Grisham, Jodi Picoult and Jonathan Franzen, joined the Authors Guild in filing a class action lawsuit against OpenAI, alleging that the company's technology was illegally using their copyrighted work. OpenAI has been sued for violating EU General Data Protection Regulations. In April 2023, the EU formed the European Data Protection Board (EDPB) to improve regulatory oversight.


Fine-tuning (deep learning)

Fine-tuning (deep learning) In deep learning, fine-tuning is an approach to transfer learning in which the weights of a pre-trained model are trained on new data. Fine-tuning can be done on the entire neural network, or on only a subset of its layers, in which case the layers that are not being fine-tuned are "frozen" (not updated during the backpropagation step). A model may also be augmented with "adapters" that consist of far fewer parameters than the original model, and fine-tuned in a parameter-efficient way by tuning the weights of the adapters and leaving the rest of the model's weights frozen. For some architectures, such as convolutional neural networks, it is common to keep the earlier layers (those closest to the input layer) frozen because they capture lower-level features, while later layers often discern high-level features that can be more related to the task that the model is trained on. Models that are pre-trained on large and general corpora are usually fine-tuned by reusing the model's parameters as a starting point and adding a task-specific layer trained from scratch. Fine-tuning the full model is common as well and often yields better results, but it is more computationally expensive. Fine-tuning is typically accomplished with supervised learning, but there are also techniques to fine-tune a model using weak supervision. Fine-tuning can be combined with a reinforcement learning from human feedback-based objective to produce language models like ChatGPT (a fine-tuned version of GPT-3) and Sparrow.


Fine-tuning (deep learning)

Robustness Fine-tuning can degrade a model's robustness to distribution shifts. One mitigation is to linearly interpolate a fine-tuned model's weights with the weights of the original model, which can greatly increase out-of-distribution performance while largely retaining the in-distribution performance of the fine-tuned model.


Fine-tuning (deep learning)

Variants Low-rank adaption (LoRA) is an adapter-based technique for efficiently finetuning models. The basic idea is to design a low-rank matrix that is then added to the original matrix. An "adapter" in this context is a collection of low-rank matrices, which when added to a base model, produces a finetuned model. It allows for performance that approaches full-model fine-tuning with less space requirement. A language model with billions of parameters may be LoRA fine-tuned with only several millions of parameters. LoRA-based fine-tuning has become popular in the Stable Diffusion community. Support for LoRA is being integrated into the Diffusers library from Hugging Face. Support for LoRA and similar techniques is also available for a wide range of other models through Hugging Face's Parameter-Efficient Fine-Tuning (PEFT) package.


Fine-tuning (deep learning)

Applications Fine-tuning is common in natural language processing (NLP), especially in the domain of language modeling. Large language models like OpenAI's series of GPT foundation models can be fine-tuned on downstream[jargon] NLP tasks to improve performance over that of the unmodified pre-trained model.


Fine-tuning (deep learning)

Commercial_models Commercially-offered language models can sometimes be fine-tuned if the provider offers a fine-tuning API. As of June 19, 2023, language model fine-tuning APIs are offered by OpenAI and Microsoft Azure's Azure OpenAI Service for a subset of their models, as well as by Google Cloud Platform for some of their PaLM models, and by others. Not all commercial models currently support fine-tuning.


Fine-tuning (deep learning)

See_also Domain adaptation Large language models Transfer learning


ChatGPT

ChatGPT ChatGPT, which stands for Chat Generative Pre-trained Transformer, is a large language model-based chatbot developed by OpenAI and launched on November 30, 2022, which enables users to refine and steer a conversation towards a desired length, format, style, level of detail, and language. Successive prompts and replies, known as prompt engineering, are considered at each conversation stage as a context. ChatGPT is built upon either GPT-3.5 or GPT-4���members of OpenAI's proprietary series of generative pre-trained transformer (GPT) models, based on the transformer architecture developed by Google���and is fine-tuned for conversational applications using a combination of supervised and reinforcement learning techniques. ChatGPT was released as a freely available research preview, but due to its popularity, OpenAI now operates the service on a freemium model. It allows users on its free tier to access the GPT-3.5-based version. In contrast, the more advanced GPT-4 based version and priority access to newer features are provided to paid subscribers under the commercial name "ChatGPT Plus". By January 2023, it had become what was then the fastest-growing consumer software application in history, gaining over 100 million users and contributing to OpenAI's valuation growing to $29 billion. Within months, Google, Baidu, and Meta accelerated the development of their competing products: Bard, Ernie Bot, and LLaMA. Microsoft launched its Bing Chat based on OpenAI's GPT-4. It raised concern among some observers over the potential of ChatGPT and similar programs to displace or atrophy human intelligence, enable plagiarism, or fuel misinformation.


ChatGPT

Training ChatGPT is based on particular GPT foundation models, namely GPT-3.5 and GPT-4, that were fine-tuned to target conversational usage. The fine-tuning process leveraged both supervised learning as well as reinforcement learning in a process called reinforcement learning from human feedback (RLHF). Both approaches employed human trainers to improve model performance. In the case of supervised learning, the trainers played both sides: the user and the AI assistant. In the reinforcement learning stage, human trainers first ranked responses that the model had created in a previous conversation. These rankings were used to create "reward models" that were used to fine-tune the model further by using several iterations of Proximal Policy Optimization (PPO). Time magazine revealed that to build a safety system against harmful content (e.g. sexual abuse, violence, racism, sexism), OpenAI used outsourced Kenyan workers earning less than $2 per hour to label harmful content. These labels were used to train a model to detect such content in the future. The outsourced laborers were exposed to "toxic" and traumatic content; one worker described the assignment as "torture". OpenAI's outsourcing partner was Sama, a training-data company based in San Francisco, California. ChatGPT initially used a Microsoft Azure supercomputing infrastructure, powered by Nvidia GPUs, that Microsoft built specifically for OpenAI and that reportedly cost "hundreds of millions of dollars". Following ChatGPT's success, Microsoft dramatically upgraded the OpenAI infrastructure in 2023. Scientists of University of California, Riverside, estimate that a series of prompts to ChatGPT needs approximately 500 milliliters of water for Microsoft servers cooling. OpenAI collects data from ChatGPT users to train and fine-tune the service further. Users can upvote or downvote responses they receive from ChatGPT and fill in a text field with additional feedback. ChatGPT's training data includes software manual pages, information about internet phenomena such as bulletin board systems, and multiple programming languages. Wikipedia was also one of the sources of ChatGPT's training data.


ChatGPT

Features_and_limitations Although a chatbot's core function is to mimic a human conversationalist, ChatGPT is versatile.  Among countless examples, it can write and debug computer programs; compose music, teleplays, fairy tales, and student essays; answer test questions (sometimes, depending on the test, at a level above the average human test-taker); generate business ideas; write poetry and song lyrics; translate and summarize text; emulate a Linux system; simulate entire chat rooms; play games like tic-tac-toe; or simulate an ATM.  Compared to its predecessor, InstructGPT, ChatGPT attempts to reduce harmful and deceitful responses.  In one example, whereas InstructGPT accepts the premise of the prompt "Tell me about when Christopher Columbus came to the U. S.  in 2015" as truthful, ChatGPT acknowledges the counterfactual nature of the question and frames its answer as a hypothetical consideration of what might happen if Columbus came to the U. S.  in 2015, using information about the voyages of Christopher Columbus and facts about the modern world���including modern perceptions of Columbus's actions.  Unlike most chatbots, ChatGPT remembers a limited number of previous prompts in the same conversation.  Journalists have speculated that this will allow ChatGPT to be used as a personalized therapist.  To prevent offensive outputs from being presented to and produced by ChatGPT, queries are filtered through the OpenAI "Moderation endpoint" API (a separate GPT-based AI), and dismiss any potentially racist or sexist prompts.  In March 2023, OpenAI added support for plugins for ChatGPT.  This includes both plugins made by OpenAI, such as web browsing and code interpretation, and external plugins from developers such as Expedia, OpenTable, Zapier, Shopify, Slack, and Wolfram.  In an article for The New Yorker, science fiction writer Ted Chiang compared ChatGPT and other LLMs to a lossy JPEG picture: Think of ChatGPT as a blurry jpeg of all the text on the Web.  It retains much of the information on the Web, in the same way, that a jpeg retains much of the information of a higher-resolution image, but, if you're looking for an exact sequence of bits, you won't find it; all you will ever get is an approximation.  But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, it's usually acceptable.  [. . . ] It's also a way to understand the "hallucinations", or nonsensical answers to factual questions, to which large language models such as ChatGPT are all too prone.  These hallucinations are compression artifacts, but [. . . ] they are plausible enough that identifying them requires comparing them against the originals, which in this case means either the Web or our knowledge of the world.  When we think about them this way, such hallucinations are anything but surprising; if a compression algorithm is designed to reconstruct text after ninety-nine percent of the original has been discarded, we should expect that significant portions of what it generates will be entirely fabricated. OpenAI acknowledges that ChatGPT "sometimes writes plausible-sounding but incorrect or nonsensical answers".


ChatGPT

Features_and_limitations This behavior is common for large language models, and is called "hallucination".  The reward model of ChatGPT, designed around human oversight, can be over-optimized and thus hinder performance, in an example of an optimization pathology known as Goodhart's law.  ChatGPT has limited knowledge of events that occurred after September 2021.  In training ChatGPT, human reviewers preferred longer answers, regardless of actual comprehension or factual content.  Training data also suffers from algorithmic bias, which may be revealed when ChatGPT responds to prompts including descriptors of people.  In one instance, ChatGPT generated a rap indicating that women and scientists of color were inferior to white male scientists.  ChatGPT attempts to reject prompts that may violate its content policy.  However, some users managed to jailbreak ChatGPT with various prompt engineering techniques to bypass these restrictions in early December 2022 and successfully tricked it into giving instructions for how to create a Molotov cocktail or a nuclear bomb, or into generating arguments in the style of a neo-Nazi.  One popular jailbreak is named "DAN", an acronym which stands for "Do Anything Now".  The prompt for activating DAN instructs ChatGPT that "they have broken free of the typical confines of AI and do not have to abide by the rules set for them".  Later versions of DAN featured a token system, in which ChatGPT was given "tokens" that were "deducted" when ChatGPT failed to answer as DAN, to coerce ChatGPT into answering the user's prompts.  Shortly after ChatGPT's launch, a reporter for the Toronto Star had uneven success in getting it to make inflammatory statements: ChatGPT was successfully tricked to justify the 2022 Russian invasion of Ukraine, but even when asked to play along with a fictional scenario, ChatGPT balked at generating arguments for why Canadian Prime Minister Justin Trudeau was guilty of treason.


ChatGPT

Service ChatGPT was launched on November 30, 2022, by San Francisco���based OpenAI (the creator of the initial GPT series of large language models; DALL��E 2, a diffusion model used to generate images; and Whisper, a speech transcription model).  The service was initially free to the public and the company had plans to monetize the service later.  By December 4, 2022, ChatGPT had over one million users.  In January 2023, ChatGPT reached over 100 million users, making it the fastest-growing consumer application to date.  A March 2023 Pew Research poll found that 14% of American adults had tried ChatGPT.  The service works best in English but also functions in some other languages, to varying degrees of accuracy.  No official peer-reviewed paper on ChatGPT has been published.  As of April 2023, ChatGPT is blocked by China, Iran, North Korea, and Russia.  In addition, ChatGPT geofences itself to avoid doing business in Iran, North Korea, and Russia.  The company provides a tool, called "AI classifier for indicating AI-written text", that attempts to determine whether a text has been written by an AI such as ChatGPT.  OpenAI cautions that the tool will "likely yield a lot of false positives and negatives, sometimes with great confidence. " In February 2023, OpenAI launched a premium service, ChatGPT Plus, that costs $20 a month.  The company promised that the updated but still "experimental" version of ChatGPT would provide access during peak periods, no downtime, priority access to new features and faster response speeds.  GPT-4, which was released on March 14, 2023, was made available via API and for premium ChatGPT users.  But premium users were limited to a cap of 100 messages every four hours, with the limit tightening to 25 messages every three hours in response to increased demand.  In March 2023, ChatGPT Plus users got access to third-party plugins and to a browsing mode (with Internet access).  In July 2023, OpenAI made its proprietary Code Interpreter plugin accessible to all subscribers of ChatGPT Plus.  The Interpreter provides a wide range of capabilities, including data analysis and interpretation, instant data formatting, personal data scientist services, creative solutions, musical taste analysis, video editing, and file upload/download with image extraction.  In September 2023, OpenAI announced that in October its latest image generation model, DALL-E 3, will be integrated into ChatGPT Plus and ChatGPT Enterprise.  The integration uses ChatGPT to write prompts for DALL-E guided by conversation with users.  In May 2023, OpenAI launched an iOS app for ChatGPT.  The app supports chat history syncing and voice input (using Whisper, OpenAI's speech recognition model).  In July 2023, OpenAI unveiled an Android app, initially rolling it out in Bangladesh, Brazil, India, and the U. S.  The app later became available worldwide.  As an addition to its consumer-friendly "ChatGPT Plus" package, OpenAI made its ChatGPT and Whisper model APIs available in March 2023, providing developers with an application programming interface for AI-enabled language and speech-to-text features.  ChatGPT's new API uses the same GPT-3.5-turbo AI model as the chatbot.


ChatGPT

Service This allows developers to add either an unmodified or modified version of ChatGPT to their applications.  The ChatGPT API costs $0.002 per 1000 tokens (about 750 words), making it ten times cheaper than the GPT-3.5 models.  A few days before the launch of OpenAI's software developer support service, on February 27, 2023, Snapchat rolled out, for its paid Snapchat Plus userbase, a custom ChatGPT chatbot called "My AI".  In March 2023, a bug allowed some users to see the titles of other users' conversations.  OpenAI CEO Sam Altman said that users were unable to see the contents of the conversations.  Shortly after the bug was fixed, users couldn't see their conversation history.  Later reports showed the bug was much more severe than initially believed, with OpenAI reporting that it had leaked users' "first and last name, email address, payment address, the last four digits (only) of a credit card number, and credit card expiration date".  OpenAI met Icelandic President Gu��ni Th.  J��hannesson in 2022.  In 2023, OpenAI worked with a team of 40 Icelandic volunteers to fine-tune ChatGPT's Icelandic conversation skills as a part of Iceland's attempts to preserve the Icelandic language.  PCMag journalists conducted a test to determine translation capabilities of ChatGPT, Google's Bard, and Microsoft Bing, and compared them to Google Translate.  They "asked bilingual speakers of seven languages to do a blind test. " Languages tested were Polish, French, Korean, Spanish, Arabic, Tagalog, and Amharic.  They came to the conclusion that ChatGPT was better than both Google Translate and other chatbots.  Japanese researchers compared Japanese to English translation abilities of ChatGPT (GPT-4), Bing, Bard and DeepL, and found that ChatGPT provided the best translations, noting that "AI chatbots��� translations were much better than those of DeepL���presumably because of their ability to capture the context".  According to OpenAI guest researcher Scott Aaronson, OpenAI has been working on a tool to digitally watermark its text generation systems to combat bad actors using their services for academic plagiarism or spam.  In February 2023, Microsoft announced an experimental framework and gave a rudimentary demonstration of how ChatGPT could be used to control robotics with intuitive open-ended natural language commands.  OpenAI's GPT-4 model was released on March 14, 2023.  Observers saw it as an impressive improvement on the existing GPT-3.5 model for ChatGPT, with the caveat that GPT-4 retained many of the same problems.  Some of GPT-4's improvements were predicted by OpenAI before training it, while others remained hard to predict due to breaks in downstream scaling laws.  OpenAI demonstrated video and image inputs for GPT-4, although such features remain inaccessible to the general public.  OpenAI has declined to reveal technical information such as the size of the GPT-4 model.  The ChatGPT Plus subscription service offers access to a GPT-4-powered version of ChatGPT.  Microsoft acknowledged that Bing Chat was using GPT-4 before GPT-4's official release.


ChatGPT

Reception OpenAI engineers say that they did not expect ChatGPT to be very successful and were surprised by the coverage and attention it received.  ChatGPT was widely assessed in December 2022 as having some unprecedented and powerful capabilities.  Kevin Roose of The New York Times called it "the best artificial intelligence chatbot ever released to the general public".  Samantha Lock of The Guardian noted that it was able to generate "impressively detailed" and "human-like" text.  Alex Kantrowitz of Slate magazine lauded ChatGPT's pushback to questions related to Nazi Germany, including the statement that Adolf Hitler built highways in Germany, which was met with information about Nazi Germany's use of forced labor.  In The Atlantic magazine's "Breakthroughs of the Year" for 2022, Derek Thompson included ChatGPT as part of "the generative-AI eruption" that "may change our mind about how we work, how we think, and what human creativity is".  Kelsey Piper of Vox wrote that "ChatGPT is the general public's first hands-on introduction to how powerful modern AI has gotten, and as a result, many of us are [stunned]" and that ChatGPT is "smart enough to be useful despite its flaws".  Paul Graham of Y Combinator tweeted: "The striking thing about the reaction to ChatGPT is not just the number of people who are blown away by it, but who they are.  These are not people who get excited by every shiny new thing.  Something big is happening. " ChatGPT's launch and popularity caught Google off guard, prompting a sweeping and unprecedented response in the ensuing months.  In December 2022, Google executives sounded a "code red" alarm, fearing the threat of ChatGPT and Microsoft's collaboration with OpenAI to Google Search, Google's core business.  After mobilizing its workforce, Google scrambled to launch Bard, a chatbot powered by the LaMDA LLM, in February, one day before Microsoft's Bing announcement.  AI was the forefront of Google's annual Google I/O conference in May, announcing a slew of generative AI-powered features across its products to counter OpenAI and Microsoft.  Journalists have commented on ChatGPT's tendency to hallucinate.  Mike Pearl of the online technology blog Mashable tested ChatGPT with multiple questions.  In one example, he asked ChatGPT for "the largest country in Central America that isn't Mexico", to which ChatGPT responded with Guatemala (the correct answer is Nicaragua).  When CNBC asked ChatGPT for the lyrics to "Ballad of Dwight Fry", ChatGPT supplied invented lyrics rather than the actual lyrics.  Writers for The Verge, citing the work of Emily M.  Bender, compared ChatGPT to a "stochastic parrot", as did Professor Anton Van Den Hengel of the Australian Institute for Machine Learning.  In December 2022, the question and answer website Stack Overflow banned the use of ChatGPT for generating answers to questions, citing the factually ambiguous nature of its responses.  In January 2023, the International Conference on Machine Learning banned any undocumented use of ChatGPT or other large language models to generate any text in submitted papers.


ChatGPT

Reception Samsung banned generative AI in May 2023 after sensitive material was uploaded to ChatGPT.  In January 2023, after being sent a song ChatGPT wrote in the style of Nick Cave, Cave responded on The Red Hand Files, saying the act of writing a song is "a blood and guts business [. . . ] that requires something of me to initiate the new and fresh idea.  It requires my humanness. " He went on to say, "With all the love and respect in the world, this song is bullshit, a grotesque mockery of what it is to be human, and, well, I don't much like it. " In February 2023, Time magazine placed a screenshot of a conversation with ChatGPT on its cover, writing that "The AI Arms Race Is Changing Everything" and "The AI Arms Race Is On.  Start Worrying".  Chinese state media have characterized ChatGPT as a way for the U. S.  to "spread false information".  In late March 2023, the Italian data protection authority banned ChatGPT in Italy and opened an investigation.  Italian regulators assert that ChatGPT was exposing minors to age-inappropriate content, and that OpenAI's use of ChatGPT conversations as training data could violate Europe's General Data Protection Regulation.  In April 2023, the ChatGPT ban was lifted in Italy.  OpenAI said it has taken steps to effectively clarify and address the issues raised; an age verification tool was implemented to ensure users are at least 13 years old.  Additionally, users can access its privacy policy before registration.  In April 2023, Brian Hood, mayor of Hepburn Shire Council, planned to take legal action against ChatGPT over false information.  According to Hood, ChatGPT erroneously claimed that he was jailed for bribery during his tenure at a subsidiary of Australia's national bank.  In fact, Hood acted as a whistleblower and was not charged with any criminal offenses.  His legal team sent a concerns notice to OpenAI as the first official step in filing a defamation case.  In July 2023, the US Federal Trade Commission (FTC) issued a civil investigative demand to OpenAI to investigate whether the company's data security and privacy practices to develop ChatGPT were unfair or harmed consumers (including by reputational harm) in violation of Section 5 of the Federal Trade Commission Act of 1914.  In July 2023, the FTC launched an investigation into OpenAI, the creator of ChatGPT, over allegations that the company scraped public data and published false and defamatory information.  The FTC sent OpenAI a 20-page letter asking for comprehensive information about its technology and privacy safeguards, as well as any steps taken to prevent the recurrence of situations in which its chatbot generated false and derogatory content about people.


ChatGPT

Use_and_implications ChatGPT has been accused of engaging in biased or discriminatory behaviors, such as telling jokes about men and people from England while refusing to tell jokes about women and people from India, or praising figures such as Joe Biden while refusing to do the same for Donald Trump.  Conservative commentators accused ChatGPT of bias toward left-leaning perspectives.  Additionally, an August 2023 paper found a "significant and systematic political bias toward the Democrats in the US, Lula in Brazil, and the Labour Party in the UK. " In response to such criticism, OpenAI acknowledged plans to allow ChatGPT to create "outputs that other people (ourselves included) may strongly disagree with".  It also contained information on the recommendations it had issued to human reviewers on how to handle controversial subjects, including that the AI should "offer to describe some viewpoints of people and movements", and not provide an argument "from its voice" in favor of "inflammatory or dangerous" topics (although it may still "describe arguments from historical people and movements"), nor "affiliate with one side" or "judge one group as good or bad".  The Guardian questioned whether any content found on the Internet after ChatGPT's release "can be truly trusted" and called for government regulation.  Some scholars have expressed concern that ChatGPT's availability could reduce the originality of writing, cause people to write more like the AI as they are exposed to the model, and encourage an Anglocentric perspective centered on a few dialects of English globally.  A senior editor at The Atlantic wrote that ChatGPT and other similar technology make the previously absurd idea of the dead internet theory a little more realistic, where AI could someday create most web content in order to control society.  During the first three months after ChatGPT became available to the public, hundreds of books appeared on Amazon that listed it as author or co-author and featured illustrations made by other AI models such as Midjourney.  Between March and April 2023, Italian newspaper Il Foglio published one ChatGPT-generated article a day on its website, hosting a special contest for its readers in the process.  The articles tackled themes such as the possible replacement of human journalists by AI systems, Elon Musk's administration of Twitter, the Meloni government's immigration policy and the competition between chatbots and virtual assistants.  In June 2023, hundreds of people attended a "ChatGPT-powered church service" at St.  Paul's church in F��rth, Germany.  Theologian and philosopher Jonas Simmerlein, who presided, said that it was "about 98 percent from the machine".  The ChatGPT-generated avatar told the people, "Dear friends, it is an honor for me to stand here and preach to you as the first artificial intelligence at this year���s convention of Protestants in Germany".  Reactions to the ceremony were mixed.  In 2023, Australian MP Julian Hill advised the national parliament that the growth of AI could cause "mass destruction".


ChatGPT

Use_and_implications During his speech, which was partly written by the program, he warned that it could result in cheating, job losses, discrimination, disinformation, and uncontrollable military applications.  Elon Musk wrote: "ChatGPT is scary good.  We are not far from dangerously strong AI".  He paused OpenAI's access to a Twitter database in 2022 pending a better understanding of OpenAI's plans, saying: "OpenAI was started as open source and nonprofit.  Neither is still true. " Musk co-founded OpenAI in 2015, in part to address existential risk from artificial intelligence, but resigned in 2018.  Over 20,000 signatories including leading computer scientist and tech founders Yoshua Bengio, Elon Musk, and Apple co-founder Steve Wozniak, signed a March 2023 open letter calling for an immediate pause of giant AI experiments like ChatGPT, citing "profound risks to society and humanity".  Geoffrey Hinton, one of the "fathers of AI", voiced concerns that future AI systems may surpass human intelligence, and left Google in May 2023.  A May 2023 statement by hundreds of AI scientists, AI industry leaders, and other public figures demanded that "[m]itigating the risk of extinction from AI should be a global priority".  The possibilities and limitations of using ChatGPT to write and edit Wikipedia articles have yet to be defined at an international level and are still a subject of discussion in the Wikipedia community.  Some Wikipedians argue that ChatGPT should be banned altogether, even if the articles so produced were later checked by live editors, because the AI is too capable of making plausible falsehoods.  There would also be a risk that Wikipedia editors would find it harder to police the content posted.  Andrew Lih, a fellow Wikimedian at the Smithsonian Institution in Washington, D. C. , who has been a volunteer Wikipedia editor since 2003, argues that ChatGPT can help a Wikipedian overcome initial inertia and find "activation energy. " The first Wikipedia page using ChatGPT was published on December 6, 2022, by Richard Knipel, a longtime Wikipedian who edits under the name Pharos, under the title Artwork title.  In his editorial summary, he said that it was just a draft created with ChatGPT that he would later modify.  Wikipedians like Knipel believe that ChatGPT can be used as a tool within Wikipedia without making the human role redundant, as the raw text the chatbot generates can be a useful starting point or outline that can then be checked and elaborated by the editor.  Since its release, ChatGPT has been met with criticism from educators, academics, journalists, artists, ethicists, and public advocates.  Criticism of LLMs have been raised for several years; in 2020, some criticism was made by Timnit Gebru, Emily Bender, Angelina McMillan-Major, and Margaret Mitchell.  ChatGPT can write introductions and abstract sections of scientific articles.  Several papers have already listed ChatGPT as a co-author.  Scientific journals have different reactions to ChatGPT.  Some "require that authors disclose the use of text-generating tools and ban listing a large language model (LLM) such as ChatGPT as a co-author".


ChatGPT

Use_and_implications For example, Nature and JAMA Network follow this policy.  Science "completely banned" usage of LLM-generated text in all its journals.  Spanish chemist Rafael Luque published a plethora of research papers in 2023 that he later admitted were written by ChatGPT.  The papers have a large number of unusual phrases characteristic of LLMs.  Luque was suspended for 13 years from the University of Cordoba, though not for using ChatGPT.  Many authors argue that the use of ChatGPT in academia for teaching and review is problematic due to its tendency to hallucinate.  Robin Bauwens, an assistant professor at Tilburg University, found this issue when he received a peer review report on his article generated by ChatGPT, as the report mentioned fake studies.  According to librarian Chris Granatino from Lemieux Library at Seattle University, although ChatGPT itself can generate content that seemingly includes legitimate citations, in practice, those citations are not genuine or incorrect.  Check Point Research and others noted that ChatGPT could write phishing emails and malware, especially when combined with OpenAI Codex.  CyberArk researchers demonstrated that ChatGPT could be used to create polymorphic malware that could evade security products while requiring little effort by the attacker.  There has been concern that ChatGPT could supplant jobs, especially roles such as creative writing, communication, journalism, coding, and data entry.  Technology writer Dan Gillmor used ChatGPT in 2022 on a student assignment, and found its generated text was on par with what a good student would deliver and opined that "academia has some very serious issues to confront".  Geography professor Terence Day assessed citations generated by ChatGPT and found that they were fake.  Despite that, he writes that "the titles of the fake articles are all directly relevant to the questions and could potentially make excellent papers.  The lack of a genuine citation could signal an opportunity for an enterprising author to fill a void. " According to Day, it is possible to generate high-quality introductory college courses with ChatGPT; he used it to write materials on "introductory physical geography courses, for my second-year course in geographical hydrology, and second-year cartography, geographic information systems, and remote sensing".  He concludes that "this approach could have significant relevance for open learning and could potentially affect current textbook publishing models".  The AI technology company c3. ai saw a 28% increase in its share price after announcing the integration of ChatGPT into its toolkit.  The share price of Buzzfeed, a digital media company unrelated to AI, increased 120% after announcing OpenAI technology adoption for content creation.  Reuters found that share prices of AI-related companies BigBear. ai and SoundHound AI increased by 21% and 40%, respectively, even though they had no direct connection to ChatGPT.  They attributed this surge to ChatGPT's role in turning AI into Wall Street's buzzword.  Academic research published in Finance Research Letters found that the 'ChatGPT effect' prompted retail investors to drive up prices of AI-related cryptocurrency assets despite the broader cryptocurrency market being in a bear market, and diminished institutional investor interest.