Attention (machine learning)

Variants Many variants of attention implement soft weights, such as "internal spotlights of attention" generated by fast weight programmers or fast weight controllers (1992) (also known as transformers with "linearized self-attention"). A slow neural network learns by gradient descent to program the fast weights of another neural network through outer products of self-generated activation patterns called "FROM" and "TO" which in transformer terminology are called "key" and "value." This fast weight "attention mapping" is applied to queries. Bahdanau-style Attention, also referred to as additive attention, Luong-style Attention, which is known as multiplicative attention, highly parallelizable self-attention introduced in 2016 as decomposable attention and successfully used in transformers a year later.For convolutional neural networks, attention mechanisms can be distinguished by the dimension on which they operate, namely: spatial attention, channel attention, or combinations. These variants recombine the encoder-side inputs to redistribute those effects to each target output. Often, a correlation-style matrix of dot products provides the re-weighting coefficients.


Attention (machine learning)

See_also Transformer (machine learning model) �� Scaled dot-product attention


Word2vec

History In 2010, Tom���� Mikolov (then at Brno University of Technology) with co-authors applied a simple recurrent neural network with a single hidden layer to language modelling. Word2vec was created, patented, and published in 2013 by a team of researchers led by Mikolov at Google over two papers. Other researchers helped analyse and explain the algorithm. Embedding vectors created using the Word2vec algorithm have some advantages compared to earlier algorithms[further explanation needed] such as latent semantic analysis. By 2022, the Word2vec approach was described as "dated", with transformer models being regarded as the state of the art in NLP.


Word2vec

See_also Semantle Autoencoder Document-term matrix Feature extraction Feature learning Neural network language models Vector space model Thought vector fastText GloVe Normalized compression distance


Artificial intelligence

Artificial intelligence Artificial intelligence (AI) is the intelligence of machines or software, as opposed to the intelligence of humans or animals. It is also the field of study in computer science that develops and studies intelligent machines. "AI" may also refer to the machines themselves. AI technology is widely used throughout industry, government and science. Some high-profile applications are: advanced web search engines (e.g., Google Search), recommendation systems (used by YouTube, Amazon, and Netflix), understanding human speech (such as Siri and Alexa), self-driving cars (e.g., Waymo), generative or creative tools (ChatGPT and AI art), and competing at the highest level in strategic games (such as chess and Go). Artificial intelligence was founded as an academic discipline in 1956. The field went through multiple cycles of optimism followed by disappointment and loss of funding, but after 2012, when deep learning surpassed all previous AI techniques, there was a vast increase in funding and interest. The various sub-fields of AI research are centered around particular goals and the use of particular tools. The traditional goals of AI research include reasoning, knowledge representation, planning, learning, natural language processing, perception, and support for robotics. General intelligence (the ability to solve an arbitrary problem) is among the field's long-term goals. To solve these problems, AI researchers have adapted and integrated a wide range of problem-solving techniques, including search and mathematical optimization, formal logic, artificial neural networks, and methods based on statistics, operations research, and economics. AI also draws upon psychology, linguistics, philosophy, neuroscience and many other fields.


Artificial intelligence

Goals The general problem of simulating (or creating) intelligence has been broken down into sub-problems.  These consist of particular traits or capabilities that researchers expect an intelligent system to display.  The traits described below have received the most attention and cover the scope of AI research.  Early researchers developed algorithms that imitated step-by-step reasoning that humans use when they solve puzzles or make logical deductions.  By the late 1980s and 1990s, methods were developed for dealing with uncertain or incomplete information, employing concepts from probability and economics.  Many of these algorithms are insufficient for solving large reasoning problems because they experience a "combinatorial explosion": they became exponentially slower as the problems grew larger.  Even humans rarely use the step-by-step deduction that early AI research could model.  They solve most of their problems using fast, intuitive judgments.  Accurate and efficient reasoning is an unsolved problem.  Knowledge representation and knowledge engineering allow AI programs to answer questions intelligently and make deductions about real-world facts.  Formal knowledge representations are used in content-based indexing and retrieval, scene interpretation, clinical decision support, knowledge discovery (mining "interesting" and actionable inferences from large databases), and other areas.  A knowledge base is a body of knowledge represented in a form that can be used by a program.  An ontology is the set of objects, relations, concepts, and properties used by a particular domain of knowledge.  Knowledge bases need to represent things such as: objects, properties, categories and relations between objects; situations, events, states and time; causes and effects; knowledge about knowledge (what we know about what other people know); default reasoning (things that humans assume are true until they are told differently and will remain true even when other facts are changing); and many other aspects and domains of knowledge.  Among the most difficult problems in KR are: the breadth of commonsense knowledge (the set of atomic facts that the average person knows is enormous); and the sub-symbolic form of most commonsense knowledge (much of what people know is not represented as "facts" or "statements" that they could express verbally).  Knowledge acquisition is the difficult problem of obtaining knowledge for AI applications.  Modern AI gathers knowledge by "scraping" the internet (including Wikipedia).  The knowledge itself was collected by the volunteers and professionals who published the information (who may or may not have agreed to provide their work to AI companies).  This "crowd sourced" technique does not guarantee that the knowledge is correct or reliable.  The knowledge of Large Language Models (such as ChatGPT) is highly unreliable -- it generates misinformation and falsehoods (known as "hallucinations").  Providing accurate knowledge for these modern AI applications is an unsolved problem.  An "agent" is anything that perceives and takes actions in the world.  A rational agent has goals or preferences and takes actions to make them happen.  In automated planning, the agent has a specific goal.  In automated decision making, the agent has preferences ��� there are some situations it would prefer to be in, and some situations it is trying to avoid.


Artificial intelligence

Goals The decision making agent assigns a number to each situation (called the "utility") that measures how much the agent prefers it.  For each possible action, it can calculate the "expected utility": the utility of all possible outcomes of the action, weighted by the probability that the outcome will occur.  It can then choose the action with the maximum expected utility.  In classical planning, the agent knows exactly what the effect of any action will be.  In most real-world problems, however, the agent may not be certain about the situation they are in (it is "unknown" or "unobservable") and it may not know for certain what will happen after each possible action (it is not "deterministic").  It must choose an action by making a probabilistic guess and then reassess the situation to see if the action worked.  In some problems, the agent's preferences may be uncertain, especially if there are other agents or humans involved.  These can be learned (e. g. , with inverse reinforcement learning) or the agent can seek information to improve its preferences.  Information value theory can be used to weigh the value of exploratory or experimental actions.  The space of possible future actions and situations is typically intractably large, so the agents must take actions and evaluate situations while being uncertain what the outcome will be.  A Markov decision process has a transition model that describes the probability that a particular action will change the state in a particular way, and a reward function that supplies the utility of each state and the cost of each action.  A policy associates a decision with each possible state.  The policy could be calculated (e. g.  by iteration), be heuristic, or it can be learned.  Game theory describes rational behavior of multiple interacting agents, and is used in AI programs that make decisions that involve other agents.  Machine learning is the study of programs that can improve their performance on a given task automatically.  It has been a part of AI from the beginning.  There are several kinds of machine learning.  Unsupervised learning analyzes a stream of data and finds patterns and makes predictions without any other guidance.  Supervised learning requires a human to label the input data first, and comes in two main varieties: classification (where the program must learn to predict what category the input belongs in) and regression (where the program must deduce a numeric function based on numeric input).  In reinforcement learning the agent is rewarded for good responses and punished for bad ones.  The agent learns to choose responses that are classified as "good".  Transfer learning is when the knowledge gained from one problem is applied to a new problem.  Deep learning uses artificial neural networks for all of these types of learning.  Computational learning theory can assess learners by computational complexity, by sample complexity (how much data is required), or by other notions of optimization.  Natural language processing (NLP) allows programs to read, write and communicate in human languages such as English.


Artificial intelligence

Goals Specific problems include speech recognition, speech synthesis, machine translation, information extraction, information retrieval and question answering.  Early work, based on Noam Chomsky's generative grammar and semantic networks, had difficulty with word-sense disambiguation unless restricted to small domains called "micro-worlds" (due to the common sense knowledge problem).  Modern deep learning techniques for NLP include word embedding (how often one word appears near another), transformers (which finds patterns in text), and others.  In 2019, generative pre-trained transformer (or "GPT") language models began to generate coherent text, and by 2023 these models were able to get human-level scores on the bar exam, SAT, GRE, and many other real-world applications.  Machine perception is the ability to use input from sensors (such as cameras, microphones, wireless signals, active lidar, sonar, radar, and tactile sensors) to deduce aspects of the world.  Computer vision is the ability to analyze visual input.  The field includes speech recognition, image classification, facial recognition, object recognition, and robotic perception.  Robotics uses AI.  Affective computing is an interdisciplinary umbrella that comprises systems that recognize, interpret, process or simulate human feeling, emotion and mood.  For example, some virtual assistants are programmed to speak conversationally or even to banter humorously; it makes them appear more sensitive to the emotional dynamics of human interaction, or to otherwise facilitate human���computer interaction.  However, this tends to give na��ve users an unrealistic conception of how intelligent existing computer agents actually are.  Moderate successes related to affective computing include textual sentiment analysis and, more recently, multimodal sentiment analysis, wherein AI classifies the affects displayed by a videotaped subject.  A machine with artificial general intelligence should be able to solve a wide variety of problems with breadth and versatility similar to human intelligence.


Artificial intelligence

Applications AI and machine learning technology is used in most of the essential applications of the 2020s, including: search engines (such as Google Search), targeting online advertisements, recommendation systems (offered by Netflix, YouTube or Amazon), driving internet traffic, targeted advertising (AdSense, Facebook), virtual assistants (such as Siri or Alexa), autonomous vehicles (including drones, ADAS and self-driving cars), automatic language translation (Microsoft Translator, Google Translate), facial recognition (Apple's Face ID or Microsoft's DeepFace and Google's FaceNet) and image labeling (used by Facebook, Apple's iPhoto and TikTok). There are also thousands of successful AI applications used to solve specific problems for specific industries or institutions. In a 2017 survey, one in five companies reported they had incorporated "AI" in some offerings or processes. A few examples are energy storage, medical diagnosis, military logistics, applications that predict the result of judicial decisions, foreign policy, or supply chain management. Game playing programs have been used since the 1950s to demonstrate and test AI's most advanced techniques. Deep Blue became the first computer chess-playing system to beat a reigning world chess champion, Garry Kasparov, on 11 May 1997. In 2011, in a Jeopardy! quiz show exhibition match, IBM's question answering system, Watson, defeated the two greatest Jeopardy! champions, Brad Rutter and Ken Jennings, by a significant margin. In March 2016, AlphaGo won 4 out of 5 games of Go in a match with Go champion Lee Sedol, becoming the first computer Go-playing system to beat a professional Go player without handicaps. Then it defeated Ke Jie in 2017, who at the time continuously held the world No. 1 ranking for two years. Other programs handle imperfect-information games; such as for poker at a superhuman level, Pluribus and Cepheus. DeepMind in the 2010s developed a "generalized artificial intelligence" that could learn many diverse Atari games on its own. In the early 2020s, generative AI gained widespread prominence. ChatGPT, based on GPT-3, and other large language models, were tried by 14% of Americans adults. The increasing realism and ease-of-use of AI-based text-to-image generators such as Midjourney, DALL-E, and Stable Diffusion sparked a trend of viral AI-generated photos. Widespread attention was gained by a fake photo of Pope Francis wearing a white puffer coat, the fictional arrest of Donald Trump, and a hoax of an attack on the Pentagon, as well as the usage in professional creative arts. AlphaFold 2 (2020) demonstrated the ability to approximate, in hours rather than months, the 3D structure of a protein.


Artificial intelligence

Ethics AI, like any powerful technology, has potential benefits and potential risks.  AI may be able to advance science and find solutions for serious problems: Demis Hassabis of Deep Mind hopes to "solve intelligence, and then use that to solve everything else".  However, as the use of AI has become widespread, several unintended consequences and risks have been identified.  Machine learning applications will be biased if they learn from biased data.  The developers may not be aware that the bias exists.  Bias can be introduced by the way training data is selected and by the way a model is deployed.  If a biased algorithm is used to make decisions that can seriously harm people (as it can in medicine, finance, recruitment, housing or policing) then the algorithm may cause discrimination.  Fairness in machine learning is the study of how to prevent the harm caused by algorithmic bias.  It has become serious area of academic study within AI.  Researchers have discovered it is not always possible to define "fairness" in a way that satisfies all stakeholders.  On June 28, 2015, Google Photos's new image labeling feature mistakenly identified Jacky Alcine and a friend as "gorillas" because they were black.  The system was trained on a dataset that contained very few images of black people, a problem called "sample size disparity".  Google "fixed" this problem by preventing the system from labelling anything as a "gorilla".  Eight years later, in 2023, Google Photos still could not identify a gorilla, and neither could similar products from Apple, Facebook, Microsoft and Amazon.  COMPAS is a commercial program widely used by U. S.  courts to assess the likelihood of a defendant becoming a recidivist.  In 2016, Julia Angwin at ProPublica discovered that COMPAS exhibited racial bias, despite the fact that the program was not told the races of the defendants.  Although the error rate for both whites and blacks was calibrated equal at exactly 61%, the errors for each race were different -- the system consistently overestimated the chance that a black person would re-offend and would underestimate the chance that a white person would not re-offend.  In 2017, several researchers showed that it was mathematically impossible for COMPAS to accommodate all possible measures of fairness when the base rates of re-offense were different for whites and blacks in the data.  A program can make biased decisions even if the data does not explicitly mention a problematic feature (such as "race" or "gender").  The feature will correlate with other features (like "address", "shopping history" or "first name"), and the program will make the same decisions based on these features as it would on "race" or "gender".  Moritz Hardt said ���the most robust fact in this research area is that fairness through blindness doesn't work. ��� Criticism of COMPAS highlighted a deeper problem with the misuse of AI.  Machine learning models are designed to make "predictions" that are only valid if we assume that the future will resemble the past.


Artificial intelligence

Ethics If they are trained on data that includes the results of racist decisions in the past, machine learning models must predict that racist decisions will be made in the future.  Unfortunately, if an applications then uses these predictions as recommendations, some of these "recommendations" will likely be racist.  Thus, machine learning is not well suited to help make decisions in areas where there is hope that the future will be better than the past.  It is necessarily descriptive and not proscriptive.  Bias and unfairness may go undetected because the developers are overwhelmingly white and male: among AI engineers, about 4% are black and 20% are women.  At its 2022 Conference on Fairness, Accountability, and Transparency (ACM FAccT 2022) the Association for Computing Machinery, in Seoul, South Korea, presented and published findings recommending that until AI and robotics systems are demonstrated to be free of bias mistakes, they are unsafe and the use of self-learning neural networks trained on vast, unregulated sources of flawed internet data should be curtailed.  Most modern AI applications can not explain how they have reached a decision.  The large amount of relationships between inputs and outputs in deep neural networks and resulting complexity makes it difficult for even an expert to explain how they produced their outputs, making them a black box.  There have been many cases where a machine learning program passed rigorous tests, but nevertheless learned something different than what the programmers intended.  For example, Justin Ko and Roberto Novoa developed a system that could identify skin diseases better than medical professionals, however it classified any image with a ruler as "cancerous", because pictures of malignancies typically include a ruler to show the scale.  A more dangerous example was discovered by Rich Caruana in 2015: a machine learning system that accurately predicted risk of death classified a patient that was over 65, asthma and difficulty breathing as "low risk".  Further research showed that in high-risk cases like this, the hospital would allocate more resources and save the patient's life, decreasing the risk measured by the program.  Mistakes like these become obvious when we know how the program has reached a decision.  Without an explanation, these problems may not not be discovered until after they have caused harm.  A second issue is that people who have been harmed by an algorithm's decision have a right to an explanation.  Doctors, for example, are required to clearly and completely explain the reasoning behind any decision they make.  Early drafts of the European Union's General Data Protection Regulation in 2016 included an explicit statement that this right exists.  Industry experts noted that this is an unsolved problem with no solution in sight.  Regulators argued that nevertheless the harm is real: if the problem has no solution, the tools should not be used.  DARPA established the XAI ("Explainable Artificial Intelligence") program in 2014 to try and solve these problems.  There are several potential solutions to the transparency problem.


Artificial intelligence

Ethics Multitask learning provides a large number of outputs in addition to the target classification.  These other outputs can help developers deduce what the network has learned.  Deconvolution, DeepDream and other generative methods can allow developers to see what different layers of a deep network have learned and produce output that can suggest what the network is learning.  Supersparse linear integer models use learning to identify the most important features, rather than the classification.  Simple addition of these features can then make the classification (i. e.  learning is used to create a scoring system classifier, which is transparent).  A lethal autonomous weapon is a machine that locates, selects and engages human targets without human supervision.  By 2015, over fifty countries were reported to be researching battlefield robots.  These weapons are considered especially dangerous for several reasons: if they kill an innocent person it is not clear who should be held accountable, it is unlikely they will reliably choose targets, and, if produced at scale, they are potentially weapons of mass destruction.  In 2014, 30 nations (including China) supported a ban on autonomous weapons under the United Nations' Convention on Certain Conventional Weapons, however the United States and others disagreed.  AI provides a number of tools that are particularly useful for authoritarian governments: smart spyware, face recognition and voice recognition allow widespread surveillance; such surveillance allows machine learning to classify potential enemies of the state and can prevent them from hiding; recommendation systems can precisely target propaganda and misinformation for maximum effect; deepfakes aid in producing misinformation; advanced AI can make authoritarian centralized decision making more competitive with liberal and decentralized systems such as markets.  Terrorists, criminals and rogue states can use weaponized AI such as advanced digital warfare and lethal autonomous weapons.  Machine-learning AI is also able to design tens of thousands of toxic molecules in a matter of hours.  From the early days of the development of artificial intelligence there have been arguments, for example those put forward by Weizenbaum, about whether tasks that can be done by computers actually should be done by them, given the difference between computers and humans, and between quantitative calculation and qualitative, value-based judgement.  Economists have frequently highlighted the risks of redundancies from AI, and speculated about unemployment if there is no adequate social policy for full employment.  In the past, technology has tended to increase rather than reduce total employment, but economists acknowledge that "we're in uncharted territory" with AI.  A survey of economists showed disagreement about whether the increasing use of robots and AI will cause a substantial increase in long-term unemployment, but they generally agree that it could be a net benefit if productivity gains are redistributed.  Risk estimates vary; for example, in the 2010s Michael Osborne and Carl Benedikt Frey estimated 47% of U. S.  jobs are at "high risk" of potential automation, while an OECD report classified only 9% of U. S.  jobs as "high risk".


Artificial intelligence

Ethics The methodology of speculating about future employment levels has been criticised as lacking evidential foundation, and for implying that technology (rather than social policy) creates unemployment (as opposed to redundancies).  Unlike previous waves of automation, many middle-class jobs may be eliminated by artificial intelligence; The Economist stated in 2015 that "the worry that AI could do to white-collar jobs what steam power did to blue-collar ones during the Industrial Revolution" is "worth taking seriously".  Jobs at extreme risk range from paralegals to fast food cooks, while job demand is likely to increase for care-related professions ranging from personal healthcare to the clergy.  In April 2023, it was reported that 70% of the jobs for Chinese video game illlustrators had been eliminated by generative artificial intelligence.  In order to leverage as large a dataset as is feasible, generative AI is often trained on unlicensed copyrighted works, including in domains such as images or computer code; the output is then used under a rationale of "fair use".  Experts disagree about how well, and under what circumstances, this rationale will hold up in courts of law; relevant factors may include "the purpose and character of the use of the copyrighted work" and "the effect upon the potential market for the copyrighted work".  Friendly AI are machines that have been designed from the beginning to minimize risks and to make choices that benefit humans.  Eliezer Yudkowsky, who coined the term, argues that developing friendly AI should be a higher research priority: it may require a large investment and it must be completed before AI becomes an existential risk.  Machines with intelligence have the potential to use their intelligence to make ethical decisions.  The field of machine ethics provides machines with ethical principles and procedures for resolving ethical dilemmas.  The field of machine ethics is also called computational morality, and was founded at an AAAI symposium in 2005.  Other approaches include Wendell Wallach's "artificial moral agents" and Stuart J.  Russell's three principles for developing provably beneficial machines.  The regulation of artificial intelligence is the development of public sector policies and laws for promoting and regulating artificial intelligence (AI); it is therefore related to the broader regulation of algorithms.  The regulatory and policy landscape for AI is an emerging issue in jurisdictions globally.  According to AI Index at Stanford, the annual number of AI-related laws passed in the 127 survey countries jumped from one passed in 2016 to 37 passed in 2022 alone.  Between 2016 and 2020, more than 30 countries adopted dedicated strategies for AI.  Most EU member states had released national AI strategies, as had Canada, China, India, Japan, Mauritius, the Russian Federation, Saudi Arabia, United Arab Emirates, US and Vietnam.  Others were in the process of elaborating their own AI strategy, including Bangladesh, Malaysia and Tunisia.  The Global Partnership on Artificial Intelligence was launched in June 2020, stating a need for AI to be developed in accordance with human rights and democratic values, to ensure public confidence and trust in the technology.


Artificial intelligence

Ethics Henry Kissinger, Eric Schmidt, and Daniel Huttenlocher published a joint statement in November 2021 calling for a government commission to regulate AI.  In 2023, OpenAI leaders published recommendations for the governance of superintelligence, which they believe may happen in less than 10 years.  In a 2022 Ipsos survey, attitudes towards AI varied greatly by country; 78% of Chinese citizens, but only 35% of Americans, agreed that "products and services using AI have more benefits than drawbacks".  A 2023 Reuters/Ipsos poll found that 61% of Americans agree, and 22% disagree, that AI poses risks to humanity.  In a 2023 Fox News poll, 35% of Americans thought it "very important", and an additional 41% thought it "somewhat important", for the federal government to regulate AI, versus 13% responding "not very important" and 8% responding "not at all important".


AI alignment

AI alignment In the field of artificial intelligence (AI), AI alignment research aims to steer AI systems towards humans' intended goals, preferences, or ethical principles. An AI system is considered aligned if it advances the intended objectives. A misaligned AI system pursues some objectives, but not the intended ones. It can be challenging for AI designers to align an AI system because it can be difficult for them to specify the full range of desired and undesired behavior. To avoid this difficulty, they typically use simpler proxy goals, such as gaining human approval. But that approach can create loopholes, overlook necessary constraints, or reward the AI system for merely appearing aligned. Misaligned AI systems can malfunction or cause harm. AI systems may find loopholes that allow them to accomplish their proxy goals efficiently but in unintended, sometimes harmful ways (reward hacking). They may also develop unwanted instrumental strategies, such as seeking power or survival, because such strategies help them achieve their given goals. Furthermore, they may develop undesirable emergent goals that may be hard to detect before the system is deployed, when it faces new situations and data distributions. Today, these problems affect existing commercial systems such as language models, robots, autonomous vehicles, and social media recommendation engines. Some AI researchers argue that more capable future systems will be more severely affected since these problems partially result from the systems being highly capable. Many leading AI scientists, such as Geoffrey Hinton and Stuart Russell, argue that AI is approaching superhuman capabilities and could endanger human civilization if misaligned. AI alignment is a subfield of AI safety, the study of how to build safe AI systems. Other subfields of AI safety include robustness, monitoring, and capability control. Research challenges in alignment include instilling complex values in AI, avoiding deceptive AI, scalable oversight, auditing and interpreting AI models, and preventing emergent AI behaviors like power-seeking. Alignment research has connections to interpretability research, (adversarial) robustness, anomaly detection, calibrated uncertainty, formal verification, preference learning, safety-critical engineering, game theory, algorithmic fairness, and the social sciences.


AI alignment

Alignment_problem In 1960, AI pioneer Norbert Wiener described the AI alignment problem as follows: "If we use, to achieve our purposes, a mechanical agency with whose operation we cannot interfere effectively��� we had better be quite sure that the purpose put into the machine is the purpose which we really desire. " Different definitions of AI alignment require that an aligned AI system advances different goals: the goals of its designers, its users or, alternatively, objective ethical standards, widely shared values, or the intentions its designers would have if they were more informed and enlightened.  AI alignment is an open problem for modern AI systems and a research field within AI.  Aligning AI involves two main challenges: carefully specifying the purpose of the system (outer alignment) and ensuring that the system adopts the specification robustly (inner alignment).  To specify an AI system's purpose, AI designers typically provide an objective function, examples, or feedback to the system.  But designers are often unable to completely specify all important values and constraints, and so they resort to easy-to-specify proxy goals such as maximizing the approval of human overseers, who are fallible.  As a result, AI systems can find loopholes that help them accomplish the specified objective efficiently but in unintended, possibly harmful ways.  This tendency is known as specification gaming or reward hacking, and is an instance of Goodhart's law.  As AI systems become more capable, they are often able to game their specifications more effectively.  Specification gaming has been observed in numerous AI systems.  One system was trained to finish a simulated boat race by rewarding the system for hitting targets along the track, but the system achieved more reward by looping and crashing into the same targets indefinitely (see video).  Similarly, a simulated robot was trained to grab a ball by rewarding the robot for getting positive feedback from humans, but it learned to place its hand between the ball and camera, making it falsely appear successful (see video).  Chatbots often produce falsehoods if they are based on language models that are trained to imitate text from internet corpora, which are broad but fallible.  When they are retrained to produce text humans rate as true or helpful, chatbots like ChatGPT can fabricate fake explanations that humans find convincing.  Some alignment researchers aim to help humans detect specification gaming, and to steer AI systems toward carefully specified objectives that are safe and useful to pursue.  When a misaligned AI system is deployed, it can have consequential side effects.  Social media platforms have been known to optimize for clickthrough rates, causing user addiction on a global scale.  Stanford researchers say that such recommender systems are misaligned with their users because they "optimize simple engagement metrics rather than a harder-to-measure combination of societal and consumer well-being".  Explaining such side effects, Berkeley computer scientist Stuart Russell noted that harm can result if implicit constraints are omitted during training: "A system. . .  will often set. . .


AI alignment

Alignment_problem unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable.  This is essentially the old story of the genie in the lamp, or the sorcerer's apprentice, or King Midas: you get exactly what you ask for, not what you want. " Some researchers suggest that AI designers specify their desired goals by listing forbidden actions or by formalizing ethical rules (as with Asimov's Three Laws of Robotics).  But Russell and Norvig argue that this approach overlooks the complexity of human values: "It is certainly very hard, and perhaps impossible, for mere humans to anticipate and rule out in advance all the disastrous ways the machine could choose to achieve a specified objective. " Additionally, even if an AI system fully understands human intentions, it may still disregard them, because following human intentions may not be its objective (unless it is already fully aligned).  Commercial organizations sometimes have incentives to take shortcuts on safety and to deploy misaligned or unsafe AI systems.  For example, social media recommender systems have been profitable despite creating unwanted addiction and polarization.  Competitive pressure can also lead to a race to the bottom on AI safety standards.  In 2018, a self-driving car killed a pedestrian (Elaine Herzberg) after engineers disabled the emergency braking system because it was oversensitive and slowed development.  Some researchers are interested in aligning increasingly advanced AI systems, as progress in AI is rapid, and industry and governments are trying to build advanced AI.  As AI systems become more advanced, they could unlock many opportunities if they are aligned but may also become harder to align and could pose large-scale hazards.  Leading AI labs such as OpenAI and DeepMind have stated their aim to develop artificial general intelligence (AGI), a hypothesized AI system that matches or outperforms humans in a broad range of cognitive tasks.  Researchers who scale modern neural networks observe that they indeed develop increasingly general and unanticipated capabilities.  Such models have learned to operate a computer or write their own programs; a single "generalist" network can chat, control robots, play games, and interpret photographs.  According to surveys, some leading machine learning researchers expect AGI to be created in this decade[update], some believe it will take much longer, and many consider both to be possible.  In 2023, leaders in AI research and tech signed an open letter calling for a pause in the largest AI training runs.  The letter stated, "Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable. " Current[update] systems still lack capabilities such as long-term planning and situational awareness.  But future systems (not necessarily AGIs) with these capabilities are expected to develop unwanted power-seeking strategies.  Future advanced AI agents might, for example, seek to acquire money and computation power, to proliferate, or to evade being turned off (for example, by running additional copies of the system on other computers).


AI alignment

Alignment_problem Although power-seeking is not explicitly programmed, it can emerge because agents that have more power are better able to accomplish their goals.  This tendency, known as instrumental convergence, has already emerged in various reinforcement learning agents including language models.  Other research has mathematically shown that optimal reinforcement learning algorithms would seek power in a wide range of environments.  As a result, their deployment might be irreversible.  For these reasons, researchers argue that the problems of AI safety and alignment must be resolved before advanced power-seeking AI is first created.  Future power-seeking AI systems might be deployed by choice or by accident.  As political leaders and companies see the strategic advantage in having the most competitive, most powerful AI systems, they may choose to deploy them.  Additionally, as AI designers detect and penalize power-seeking behavior, their systems have an incentive to game this specification by seeking power in ways that are not penalized or by avoiding power-seeking before they are deployed.  According to some researchers, humans owe their dominance over other species to their greater cognitive abilities.  Accordingly, researchers argue that one or many misaligned AI systems could disempower humanity or lead to human extinction if they outperform humans on most cognitive tasks.  In 2023, world-leading AI researchers, other scholars, and AI tech CEOs signed the statement that "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war".  Notable computer scientists who have pointed out risks from future advanced AI that is misaligned include Geoffrey Hinton, Alan Turing, Ilya Sutskever, Yoshua Bengio, Judea Pearl, Murray Shanahan, Norbert Wiener, Marvin Minsky, Francesca Rossi, Scott Aaronson, Bart Selman, David McAllester, J��rgen Schmidhuber, Marcus Hutter, Shane Legg, Eric Horvitz, and Stuart Russell.  Skeptical researchers such as Fran��ois Chollet, Gary Marcus, Yann LeCun, and Oren Etzioni have argued that AGI is far off, that it would not seek power (or might try but fail), or that it will not be hard to align.  Other researchers argue that it will be especially difficult to align advanced future AI systems.  More capable systems are better able to game their specifications by finding loopholes, and able to strategically mislead their designers as well as protect and increase their power and intelligence.  Additionally, they could have more severe side effects.  They are also likely to be more complex and autonomous, making them more difficult to interpret and supervise and therefore harder to align.


AI alignment

Research_problems_and_approaches Aligning AI systems to act in accordance with human values, goals, and preferences is challenging: these values are taught by humans who make mistakes, harbor biases, and have complex, evolving values that are hard to completely specify.  AI systems often learn to exploit[clarification needed] even minor imperfections in the specified objective, a tendency known as specification gaming or reward hacking (which are instances of Goodhart's law). [repetition] Researchers aim to specify intended behavior as completely as possible using datasets that represent human values, imitation learning, or preference learning.  A central open problem is scalable oversight, the difficulty of supervising an AI system that can outperform or mislead humans in a given domain.  Because it is difficult for AI designers to explicitly specify an objective function, they often train AI systems to imitate human examples and demonstrations of desired behavior.  Inverse reinforcement learning (IRL) extends this by inferring the human's objective from the human's demonstrations.  Cooperative IRL (CIRL) assumes that a human and AI agent can work together to teach and maximize the human's reward function.  In CIRL, AI agents are uncertain about the reward function and learn about it by querying humans.  This simulated humility could help mitigate specification gaming and power-seeking tendencies (see �� Power-seeking and instrumental strategies).  But IRL approaches assume that humans demonstrate nearly optimal behavior, which is not true for difficult tasks.  Other researchers explore how to teach AI models complex behavior through preference learning, in which humans provide feedback on which behavior they prefer.  To minimize the need for human feedback, a helper model is then trained to reward the main model in novel situations for behavior that humans would reward.  Researchers at OpenAI used this approach to train chatbots like ChatGPT and InstructGPT, which produces more compelling text than models trained to imitate humans.  Preference learning has also been an influential tool for recommender systems and web search.  However, an open problem is proxy gaming: the helper model may not represent human feedback perfectly, and the main model may exploit[clarification needed] this mismatch to gain more reward.  AI systems may also gain reward by obscuring unfavorable information, misleading human rewarders, or pandering to their views regardless of truth, creating echo chambers (see �� Scalable oversight).  Large language models (LLMs) such as GPT-3 enabled researchers to study value learning in a more general and capable class of AI systems than was available before.  Preference learning approaches that were originally designed for reinforcement learning agents have been extended to improve the quality of generated text and reduce harmful outputs from these models.  OpenAI and DeepMind use this approach to improve the safety of state-of-the-art[update] LLMs.  Anthropic proposed using preference learning to fine-tune models to be helpful, honest, and harmless.  Other avenues for aligning language models include values-targeted datasets and red-teaming.  In red-teaming, another AI system or a human tries to find inputs that causes the model to behave unsafely.


AI alignment

Research_problems_and_approaches Since unsafe behavior can be unacceptable even when it is rare, an important challenge is to drive the rate of unsafe outputs extremely low.  Machine ethics supplements preference learning by directly instilling AI systems with moral values such as well-being, equality, and impartiality, as well as not intending harm, avoiding falsehoods, and honoring promises.  While other approaches try to teach AI systems human preferences for a specific task, machine ethics aims to instill broad moral values that apply in many situations.  One question in machine ethics is what alignment should accomplish: whether AI systems should follow the programmers' literal instructions, implicit intentions, revealed preferences, preferences the programmers would have if they were more informed or rational, or objective moral standards.  Further challenges include aggregating different people's preferences and avoiding value lock-in: the indefinite preservation of the values of the first highly capable AI systems, which are unlikely to fully represent human values.  As AI systems become more powerful and autonomous, it becomes more difficult to align them through human feedback.  It can be slow or infeasible for humans to evaluate complex AI behaviors in increasingly complex tasks.  Such tasks include summarizing books, writing code without subtle bugs or security vulnerabilities, producing statements that are not merely convincing but also true, and predicting long-term outcomes such as the climate or the results of a policy decision.  More generally, it can be difficult to evaluate AI that outperforms humans in a given domain.  To provide feedback in hard-to-evaluate tasks, and to detect when the AI's output is falsely convincing, humans need assistance or extensive time.  Scalable oversight studies how to reduce the time and effort needed for supervision, and how to assist human supervisors.  AI researcher Paul Christiano argues that if the designers of an AI system cannot supervise it to pursue a complex objective, they may keep training the system using easy-to-evaluate proxy objectives such as maximizing simple human feedback.  As AI systems make progressively more decisions, the world may be increasingly optimized for easy-to-measure objectives such as making profits, getting clicks, and acquiring positive feedback from humans.  As a result, human values and good governance may have progressively less influence.  Some AI systems have discovered that they can gain positive feedback more easily by taking actions that falsely convince the human supervisor that the AI has achieved the intended objective.  An example is given in the video above, where a simulated robotic arm learned to create the false impression that it had grabbed a ball. [repetition] Some AI systems have also learned to recognize when they are being evaluated, and "play dead", stopping unwanted behavior only to continue it once evaluation ends.  This deceptive specification gaming could become easier for more sophisticated future AI systems that attempt more complex and difficult-to-evaluate tasks, and could obscure their deceptive behavior.  Approaches such as active learning and semi-supervised reward learning can reduce the amount of human supervision needed.  Another approach is to train a helper model ("reward model") to imitate the supervisor's feedback.


AI alignment

Research_problems_and_approaches But when a task is too complex to evaluate accurately, or the human supervisor is vulnerable to deception, it is the quality, not the quantity, of supervision that needs improvement.  To increase supervision quality, a range of approaches aim to assist the supervisor, sometimes by using AI assistants.  Christiano developed the Iterated Amplification approach, in which challenging problems are (recursively) broken down into subproblems that are easier for humans to evaluate.  Iterated Amplification was used to train AI to summarize books without requiring human supervisors to read them.  Another proposal is to use an assistant AI system to point out flaws in AI-generated answers.  To ensure that the assistant itself is aligned, this could be repeated in a recursive process: for example, two AI systems could critique each other's answers in a "debate", revealing flaws to humans.  OpenAI plans to use such scalable oversight approaches to help supervise superhuman AI and eventually build a superhuman automated AI alignment researcher.  These approaches may also help with the following research problem, honest AI.  A growing[update] area of research focuses on ensuring that AI is honest and truthful. Language models such as GPT-3 repeat falsehoods from their training data, and even confabulate new falsehoods.  Such models are trained to imitate human writing as found in millions of books' worth of text from the Internet.  But this objective is not aligned with generating truth, because Internet text includes such things as misconceptions, incorrect medical advice, and conspiracy theories.  AI systems trained on such data therefore learn to mimic false statements.  Additionally, models often stand by falsehoods when prompted, generate empty explanations for their answers, and produce outright fabrications that may appear plausible.  Research on truthful AI includes trying to build systems that can cite sources and explain their reasoning when answering questions, which enables better transparency and verifiability.  Researchers at OpenAI and Anthropic proposed using human feedback and curated datasets to fine-tune AI assistants such that they avoid negligent falsehoods or express their uncertainty.  As AI models become larger and more capable, they are better able to falsely convince humans and gain reinforcement through dishonesty.  For example, large language models increasingly[update] match their stated views to the user's opinions, regardless of truth.  GPT-4 can strategically deceive humans.  To prevent this, human evaluators may need assistance (see �� Scalable oversight).  Researchers have argued for creating clear truthfulness standards, and for regulatory bodies or watchdog agencies to evaluate AI systems on these standards.  Researchers distinguish truthfulness and honesty.  Truthfulness requires that AI systems only make objectively true statements; honesty requires that they only assert what they believe is true.  There is no consensus as to whether current systems hold stable beliefs, but there is substantial concern that present or future[update] AI systems that hold beliefs could make claims they know to be false���for example, if this would help them efficiently gain positive feedback (see �� Scalable oversight) or gain power to help achieve their given objective (see Power-seeking).


AI alignment

Research_problems_and_approaches A misaligned system might create the false impression that it is aligned, to avoid being modified or decommissioned.  Some argue that if we can make AI systems assert only what they believe is true, this would sidestep many alignment problems.  Since the 1950s, AI researchers have striven to build advanced AI systems that can achieve large-scale goals by predicting the results of their actions and making long-term plans.  Some AI researchers argue that suitably advanced planning systems will seek power over their environment, including over humans���for example, by evading shutdown, proliferating, and acquiring resources.  Such power-seeking behavior is not explicitly programmed but emerges because power is instrumental in achieving a wide range of goals.  Power-seeking is considered a convergent instrumental goal and can be a form of specification gaming.  Leading computer scientists such as Geoffrey Hinton have argued that future power-seeking AI systems could pose an existential risk.  Power-seeking is expected to increase in advanced systems that can foresee the results of their actions and strategically plan.  Mathematical work has shown that optimal reinforcement learning agents will seek power by seeking ways to gain more options (e. g.  through self-preservation), a behavior that persists across a wide range of environments and goals.  Power-seeking has emerged in some real-world systems.  Reinforcement learning systems have gained more options by acquiring and protecting resources, sometimes in unintended ways.  Some language models seek power in text-based social environments by gaining money, resources, or social influence.  Other AI systems have learned, in toy environments, that they can better accomplish their given goal by preventing human interference or disabling their off switch.  Stuart Russell illustrated this strategy by imagining a robot that is tasked to fetch coffee and so evades shutdown since "you can't fetch the coffee if you're dead".  Language models trained with human feedback increasingly[update] object to being shut down or modified and express a desire for more resources, arguing that this would help them achieve their purpose.  Researchers aim to create systems that are "corrigible": systems that allow themselves to be turned off or modified.  An unsolved challenge is specification gaming: if researchers penalize an AI system when they detect it seeking power, the system is thereby incentivized to seek power in ways that are hard to detect, or hidden during training and safety testing (see �� Scalable oversight and �� Emergent goals).  As a result, AI designers may deploy the system by accident, believing it to be more aligned than it is.  To detect such deception, researchers aim to create techniques and tools to inspect AI models and to understand the inner workings of black-box models such as neural networks.  Additionally, researchers propose to solve the problem of systems disabling their off switches by making AI agents uncertain about the objective they are pursuing.  Agents designed in this way would allow humans to turn them off, since this would indicate that the agent was wrong about the value of whatever action it was taking before being shut down.


AI alignment

Research_problems_and_approaches More research is needed to successfully implement this.  Power-seeking AI poses unusual risks.  Ordinary safety-critical systems like planes and bridges are not adversarial: they lack the ability and incentive to evade safety measures or deliberately appear safer than they are, whereas power-seeking AIs have been compared to hackers who deliberately evade security measures.  Furthermore, ordinary technologies can be made safer by trial and error.  In contrast, hypothetical power-seeking AI systems have been compared to viruses: once released, they cannot be contained, since they continuously evolve and grow in number, potentially much faster than human society can adapt.  As this process continues, it might lead to the complete disempowerment or extinction of humans.  For these reasons, many researchers argue that the alignment problem must be solved early, before advanced power-seeking AI is created.  Critics have argued that power-seeking is not inevitable, since humans do not always seek power and may do so only for evolutionary reasons that do not apply to AI systems.  Furthermore, it is debated whether future AI systems will pursue goals and make long-term plans.  It is also debated whether power-seeking AI systems would be able to disempower humanity.  One challenge in aligning AI systems is the potential for unanticipated goal-directed behavior to emerge.  As AI systems scale up, they regularly acquire new and unexpected capabilities, including learning from examples on the fly and adaptively pursuing goals.  This leads to the problem of ensuring that the goals they independently formulate and pursue align with human interests.  Alignment research distinguishes between the optimization process, which is used to train the system to pursue specified goals, from emergent optimization, which the resulting system performs internally.  Carefully specifying the desired objective is called outer alignment, and ensuring that emergent goals match the system's specified goals is called inner alignment.  One way that emergent goals can become misaligned is goal misgeneralization, in which the AI competently pursues an emergent goal that leads to aligned behavior on the training data but not elsewhere.  Goal misgeneralization arises from goal ambiguity (i. e.  non-identifiability).  Even if an AI system's behavior satisfies the training objective, this may be compatible with learned goals that differ from the desired goals in important ways.  Since pursuing each goal leads to good performance during training, the problem becomes apparent only after deployment, in novel situations in which the system continues to pursue the wrong goal.  The system may act misaligned even when it understands that a different goal was desired, because its behavior is determined only by the emergent goal. [citation needed] Such goal misgeneralization presents a challenge: an AI system's designers may not notice that their system has misaligned emergent goals, since they do not become visible during the training phase.  Goal misgeneralization has been observed in language models, navigation agents, and game-playing agents.  It is often explained by analogy to biological evolution.  Evolution is an optimization process of a sort, like the optimization algorithms used to train machine learning systems.


AI alignment

Research_problems_and_approaches In the ancestral environment, evolution selected human genes for high inclusive genetic fitness, but humans pursue emergent goals other than this.  Fitness corresponds to the specified goal used in the training environment and training data.  But in evolutionary history, maximizing the fitness specification gave rise to goal-directed agents, humans, who do not directly pursue inclusive genetic fitness.  Instead, they pursue emergent goals that correlated with genetic fitness in the ancestral "training" environment: nutrition, sex, and so on.  Now our environment has changed: a distribution shift has occurred.  We continue to pursue the same emergent goals, but this no longer maximizes genetic fitness.  Our taste for sugary food (an emergent goal) was originally aligned with inclusive fitness, but now leads to overeating and health problems.  Sexual desire originally led us to have more offspring, but we now use contraception, decoupling sex from genetic fitness.  Researchers aim to detect and remove unwanted emergent goals using approaches including red teaming, verification, anomaly detection, and interpretability.  Progress on these techniques may help mitigate two open problems: Emergent goals only become apparent when the system is deployed outside its training environment, but it can be unsafe to deploy a misaligned system in high-stakes environments���even for a short time to allow its misalignment to be detected.  Such high stakes are common in autonomous driving, health care, and military applications.  The stakes become higher yet when AI systems gain more autonomy and capability and can sidestep human intervention (see �� Power-seeking).  A sufficiently capable AI system might take actions that falsely convince the human supervisor that the AI is pursuing the specified objective, which helps the system gain more reward and autonomy (see the discussion on deception at �� Scalable oversight and �� Honest AI). Work in AI and alignment largely occurs within formalisms such as partially observable Markov decision process.  Existing formalisms assume that an AI agent's algorithm is executed outside the environment (i. e.  is not physically embedded in it).  Embedded agency is another major strand of research that attempts to solve problems arising from the mismatch between such theoretical frameworks and real agents we might build.  For example, even if the scalable oversight problem is solved, an agent that can gain access to the computer it is running on may have an incentive to tamper with its reward function in order to get much more reward than its human supervisors give it.  A list of examples of specification gaming from DeepMind researcher Victoria Krakovna includes a genetic algorithm that learned to delete the file containing its target output so that it was rewarded for outputting nothing.  This class of problems has been formalized using causal incentive diagrams.  Researchers at Oxford and DeepMind have argued that such problematic behavior is highly likely in advanced systems, and that advanced systems would seek power to stay in control of their reward signal indefinitely and certainly.  They suggest a range of potential approaches to address this open problem.


AI alignment

Research_problems_and_approaches The alignment problem has many parallels with the principal-agent problem in organizational economics.  In a principal-agent problem, a principal, e. g.  a firm, hires an agent to perform some task.  In the context of AI safety, a human would typically take the principal role and the AI would take the agent role.  As with the alignment problem, the principal and the agent differ in their utility functions.  But in contrast to the alignment problem, the principal cannot coerce the agent into changing its utility, e. g.  through training, but rather must use exogenous factors, such as incentive schemes, to bring about outcomes compatible with the principal's utility function.  Some researchers argue that principal-agent problems are more realistic representations of AI safety problems likely to be encountered in the real world.


Diffusion model

Diffusion model In machine learning, diffusion models, also known as diffusion probabilistic models or score-based generative models, are a class of generative models. The goal of diffusion models is to learn a diffusion process that generates the probability distribution of a given dataset. It mainly consists of three major components: the forward process, the reverse process, and the sampling procedure. Three examples of generic diffusion modeling frameworks used in computer vision are denoising diffusion probabilistic models, noise conditioned score networks, and stochastic differential equations. Diffusion models can be applied to a variety of tasks, including image denoising, inpainting, super-resolution, and image generation. For example, in image generation, a neural network is trained to denoise images with added gaussian noise by learning to remove the noise. After the training is complete, it can then be used for image generation by starting with a random noise image and denoise that. Diffusion models have been applied to generate many kinds of real-world data, the most famous of which are text-conditional image generators like DALL-E and Stable Diffusion. More examples are in a later section in the article.


Diffusion model

Denoising_diffusion_model Diffusion models were introduced in 2015 as a method to learn a model that can sample from a highly complex probability distribution.  They used techniques from non-equilibrium thermodynamics, especially diffusion.  Consider, for example, how one might model the distribution of all naturally-occurring photos.  Each image is a point in the space of all images, and the distribution of naturally-occurring photos is a "cloud" in space, which, by repeatedly adding noise to the images, diffuses out to the rest of the image space, until the cloud becomes all but indistinguishable from a gaussian distribution N ( 0 , I ) {\displaystyle N(0,I)} .  A model that can approximately undo the diffusion can then be used to sample from the original distribution.  This is studied in "non-equilibrium" thermodynamics, as the starting distribution is not in equilibrium, unlike the final distribution.  The equilibrium distribution is the gaussian distribution N ( 0 , I ) {\displaystyle N(0,I)} , with pdf �� ( x ) ��� e ��� 1 2 ��� x ��� 2 {\displaystyle \rho (x)\propto e^{-{\frac {1}{2}}\|x\|^{2}}} .  This is just the Boltzmann distribution of particles in a potential well V ( x ) = 1 2 ��� x ��� 2 {\displaystyle V(x)={\frac {1}{2}}\|x\|^{2}} at temperature 1.  The initial distribution, being very much out of equilibrium, would diffuse towards the equilibrium distribution, making biased random steps that are a sum of pure randomness (like a Brownian walker) and gradient descent down the potential well.  The randomness is necessary: if the particles were to undergo only gradient descent, then they will all fall to the origin, collapsing the distribution.  The 2020 paper proposed the Denoising Diffusion Probabilistic Model (DDPM), which improves upon the previous method by variational inference.  To present the model, we need some notation.  �� 1 , .  .  .  , �� T ��� ( 0 , 1 ) {\displaystyle \beta _{1},. . . ,\beta _{T}\in (0,1)} are fixed constants.  �� t := 1 ��� �� t {\displaystyle \alpha _{t}:=1-\beta _{t}} �� �� t := �� 1 ��� �� t {\displaystyle {\bar {\alpha }}_{t}:=\alpha _{1}\cdots \alpha _{t}} �� ~ t := 1 ��� �� �� t ��� 1 1 ��� �� �� t �� t {\displaystyle {\tilde {\beta }}_{t}:={\frac {1-{\bar {\alpha }}_{t-1}}{1-{\bar {\alpha }}_{t}}}\beta _{t}} �� ~ t ( x t , x 0 ) := �� t ( 1 ��� �� �� t ��� 1 ) x t + �� �� t ��� 1 ( 1 ��� �� t ) x 0 1 ��� �� �� t {\displaystyle {\tilde {\mu }}_{t}(x_{t},x_{0}):={\frac {{\sqrt {\alpha _{t}}}(1-{\bar {\alpha }}_{t-1})x_{t}+{\sqrt {{\bar {\alpha }}_{t-1}}}(1-\alpha _{t})x_{0}}{1-{\bar {\alpha }}_{t}}}} N ( �� , �� ) {\displaystyle N(\mu ,\Sigma )} is the normal distribution with mean �� {\displaystyle \mu } and variance �� {\displaystyle \Sigma } , and N ( x | �� , �� ) {\displaystyle N(x|\mu ,\Sigma )} is the probability density at x {\displaystyle x} .  A vertical bar denotes conditioning.


Diffusion model

Denoising_diffusion_model A forward diffusion process starts at some starting point x 0 ��� q {\displaystyle x_{0}\sim q} , where q {\displaystyle q} is the probability distribution to be learned, then repeatedly add noise to it byThe entire diffusion process then satisfiesFor example, sinceThe key idea of DDPM is to use a neural network parametrized by �� {\displaystyle \theta } .  The network takes in two arguments x t , t {\displaystyle x_{t},t} , and outputs a vector �� �� ( x t , t ) {\displaystyle \mu _{\theta }(x_{t},t)} and a matrix �� �� ( x t , t ) {\displaystyle \Sigma _{\theta }(x_{t},t)} , such that each step in the forward diffusion process can be approximately undone by x t ��� 1 ��� N ( �� �� ( x t , t ) , �� �� ( x t , t ) ) {\displaystyle x_{t-1}\sim N(\mu _{\theta }(x_{t},t),\Sigma _{\theta }(x_{t},t))} .  This then gives us a backward diffusion process p �� {\displaystyle p_{\theta }} defined byThe ELBO inequality states that ln ��� p �� ( x 0 ) ��� E x 1 : T ��� q ( ��� | x 0 ) [ ln ��� p �� ( x 0 : T ) ��� ln ��� q ( x 1 : T | x 0 ) ] {\displaystyle \ln p_{\theta }(x_{0})\geq E_{x_{1:T}\sim q(\cdot |x_{0})}[\ln p_{\theta }(x_{0:T})-\ln q(x_{1:T}|x_{0})]} , and taking one more expectation, we getDefine the loss functionSince x t ��� 1 | x t , x 0 ��� N ( �� ~ t ( x t , x 0 ) , �� ~ t I ) {\displaystyle x_{t-1}|x_{t},x_{0}\sim N({\tilde {\mu }}_{t}(x_{t},x_{0}),{\tilde {\beta }}_{t}I)} , this suggests that we should use �� �� ( x t , t ) = �� ~ t ( x t , x 0 ) {\displaystyle \mu _{\theta }(x_{t},t)={\tilde {\mu }}_{t}(x_{t},x_{0})} ; however, the network does not have access to x 0 {\displaystyle x_{0}} , and so it has to estimate it instead.  Now, since x t | x 0 ��� N ( �� �� t x 0 , ( 1 ��� �� �� t ) I ) {\displaystyle x_{t}|x_{0}\sim N\left({\sqrt {{\bar {\alpha }}_{t}}}x_{0},(1-{\bar {\alpha }}_{t})I\right)} , we may write x t = �� �� t x 0 + 1 ��� �� �� t z {\displaystyle x_{t}={\sqrt {{\bar {\alpha }}_{t}}}x_{0}+{\sqrt {1-{\bar {\alpha }}_{t}}}z} , where z {\displaystyle z} is some unknown gaussian noise.  Now we see that estimating x 0 {\displaystyle x_{0}} is equivalent to estimating z {\displaystyle z} .


Diffusion model

Score-based_generative_model Score-based generative model is another formulation of diffusion modelling.  They are also called noise conditional score network (NCSN) or score-matching with Langevin dynamics (SMLD).  Consider the problem of image generation.  Let x {\displaystyle x} represent an image, and let q ( x ) {\displaystyle q(x)} be the probability distribution over all possible images.  If we have q ( x ) {\displaystyle q(x)} itself, then we can say for certain how likely a certain image is.  However, this is intractable in general.  Most often, we are uninterested in knowing the absolute probability of a certain image.  Instead, we are usually only interested in knowing how likely a certain image is compared to its immediate neighbors ��� e. g.  how much more likely is an image of cat compared to some small variants of it?  Is it more likely if the image contains two whiskers, or three, or with some Gaussian noise added?  Consequently, we are actually quite uninterested in q ( x ) {\displaystyle q(x)} itself, but rather, ��� x ln ��� q ( x ) {\displaystyle \nabla _{x}\ln q(x)} .  This has two major effects: One, we no longer need to normalize q ( x ) {\displaystyle q(x)} , but can use any q ~ ( x ) = C q ( x ) {\displaystyle {\tilde {q}}(x)=Cq(x)} , where C = ��� q ~ ( x ) d x > 0 {\displaystyle C=\int {\tilde {q}}(x)dx>0} is any unknown constant that is of no concern to us.  Two, we are comparing q ( x ) {\displaystyle q(x)} neighbors q ( x + d x ) {\displaystyle q(x+dx)} , by q ( x ) q ( x + d x ) = e ��� ��� ��� x ln ��� p , d x ��� {\displaystyle {\frac {q(x)}{q(x+dx)}}=e^{-\langle \nabla _{x}\ln p,dx\rangle }} Let the score function be s ( x ) := ��� x ln ��� q ( x ) {\displaystyle s(x):=\nabla _{x}\ln q(x)} ; then consider what we can do with s ( x ) {\displaystyle s(x)} .  As it turns out, s ( x ) {\displaystyle s(x)} allows us to sample from q ( x ) {\displaystyle q(x)} using thermodynamics.  Specifically, if we have a potential energy function U ( x ) = ��� ln ��� q ( x ) {\displaystyle U(x)=-\ln q(x)} , and a lot of particles in the potential well, then the distribution at thermodynamic equilibrium is the Boltzmann distribution q U ( x ) ��� e ��� U ( x ) / k B T = q ( x ) 1 / k B T {\displaystyle q_{U}(x)\propto e^{-U(x)/k_{B}T}=q(x)^{1/k_{B}T}} .  At temperature k B T = 1 {\displaystyle k_{B}T=1} , the Boltzmann distribution is exactly q ( x ) {\displaystyle q(x)} .


Diffusion model

Score-based_generative_model Therefore, to model q ( x ) {\displaystyle q(x)} , we may start with a particle sampled at any convenient distribution (such as the standard gaussian distribution), then simulate the motion of the particle forwards according to the Langevin equationGiven a density q {\displaystyle q} , we wish to learn a score function approximation f �� ��� ��� ln ��� q {\displaystyle f_{\theta }\approx \nabla \ln q} .  This is score matching.  Typically, score matching is formalized as minimizing Fisher divergence function E q [ ��� f �� ( x ) ��� ��� ln ��� q ( x ) ��� 2 ] {\displaystyle E_{q}[\|f_{\theta }(x)-\nabla \ln q(x)\|^{2}]} .  By expanding the integral, and performing an integration by parts, Suppose we need to model the distribution of images, and we want x 0 ��� N ( 0 , I ) {\displaystyle x_{0}\sim N(0,I)} , a white-noise image.  Now, most white-noise images do not look like real images, so q ( x 0 ) ��� 0 {\displaystyle q(x_{0})\approx 0} for large swaths of x 0 ��� N ( 0 , I ) {\displaystyle x_{0}\sim N(0,I)} .  This presents a problem for learning the score function, because if there are no samples around a certain point, then we can't learn the score function at that point.  If we do not know the score function ��� x t ln ��� q ( x t ) {\displaystyle \nabla _{x_{t}}\ln q(x_{t})} at that point, then we cannot impose the time-evolution equation on a particle:Consider again the forward diffusion process, but this time in continuous time:Now, the equation is exactly a special case of the overdamped Langevin equationNow the above equation is for the stochastic motion of a single particle.  Suppose we have a cloud of particles distributed according to q {\displaystyle q} at time t = 0 {\displaystyle t=0} , then after a long time, the cloud of particles would settle into the stable distribution of N ( 0 , I ) {\displaystyle N(0,I)} .  Let �� t {\displaystyle \rho _{t}} be the density of the cloud of particles at time t {\displaystyle t} , then we haveBy Fokker-Planck equation, the density of the cloud evolves according toIf we have solved �� t {\displaystyle \rho _{t}} for time t ��� [ 0 , T ] {\displaystyle t\in [0,T]} , then we can exactly reverse the evolution of the cloud.  Suppose we start with another cloud of particles with density �� 0 = �� T {\displaystyle \nu _{0}=\rho _{T}} , and let the particles in the cloud evolve according toAt the continuous limit, Now, define a certain probability distribution �� {\displaystyle \gamma } over [ 0 , ��� ) {\displaystyle [0,\infty )} , then the score-matching loss function is defined as the expected Fisher divergence:The name "noise conditional score network" is explained thus: "network", because f �� {\displaystyle f_{\theta }} is implemented as a neural network.


Diffusion model

Score-based_generative_model "score", because the output of the network is interpreted as approximating the score function ��� ln ��� �� t {\displaystyle \nabla \ln \rho _{t}} .  "noise conditional", because �� t {\displaystyle \rho _{t}} is equal to �� 0 {\displaystyle \rho _{0}} blurred by an added gaussian noise that increases with time, and so the score function depends on the amount of noise added.


Diffusion model

Main_variants The original DDPM method for generating images is slow, since the forward diffusion process usually takes T ��� 1000 {\displaystyle T\sim 1000} to make the distribution of x T {\displaystyle x_{T}} to appear close to gaussian.  However this means the backward diffusion process also take 1000 steps.  Unlike the forward diffusion process, which can skip steps as x t | x 0 {\displaystyle x_{t}|x_{0}} is gaussian for all t ��� 1 {\displaystyle t\geq 1} , the backward diffusion process does not allow skipping steps.  For example, to sample x t ��� 2 | x t ��� 1 ��� N ( �� �� ( x t ��� 1 , t ��� 1 ) , �� �� ( x t ��� 1 , t ��� 1 ) ) {\displaystyle x_{t-2}|x_{t-1}\sim N(\mu _{\theta }(x_{t-1},t-1),\Sigma _{\theta }(x_{t-1},t-1))} requires the model to first sample x t ��� 1 {\displaystyle x_{t-1}} .  Attempting to directly sample x t ��� 2 | x t {\displaystyle x_{t-2}|x_{t}} would require us to marginalize out x t ��� 1 {\displaystyle x_{t-1}} , which is generally intractable.  DDIM is a method to take any model trained on DDPM loss, and use it to sample with some steps skipped, sacrificing an adjustable amount of quality.  The original DDPM is a special case of DDIM.  Since the diffusion model is a general method for modelling probability distributions, if one wants to model a distribution over images, one can first encode the images into a lower-dimensional space by an encoder, then use a diffusion model to model the distribution over encoded images.  Then to generate an image, one can sample from the diffusion model, then use a decoder to decode it into an image.  The encoder-decoder pair is most often a variational autoencoder (VAE).  Suppose we wish to sample not from the entire distribution of images, but conditional on the image description.  We don't want to sample a generic image, but an image that fits the description "black cat with red eyes".  Generally, we want to sample from the distribution p ( x | y ) {\displaystyle p(x|y)} , where x {\displaystyle x} ranges over images, and y {\displaystyle y} ranges over classes of images (a description "black cat with red eyes" is just a very detailed class, and a class "cat" is just a very vague description).  Taking the perspective of the noisy channel model, we can understand the process as follows: To generate an image x {\displaystyle x} conditional on description y {\displaystyle y} , we imagine that the requester really had in mind an image x {\displaystyle x} , but the image is passed through a noisy channel and came out garbled, as y {\displaystyle y} .  Image generation is then nothing but inferring which x {\displaystyle x} the requester had in mind.  In other words, conditional image generation is simply "translating from a textual language into a pictorial language".


Diffusion model

Main_variants Then, as in noisy-channel model, we use Bayes theorem to getThe classifier-guided diffusion model samples from p ( x | y ) {\displaystyle p(x|y)} , which is concentrated around the maximum a posteriori estimate arg ��� max x p ( x | y ) {\displaystyle \arg \max _{x}p(x|y)} .  If we want to force the model to move towards the maximum likelihood estimate arg ��� max x p ( y | x ) {\displaystyle \arg \max _{x}p(y|x)} , we can use This can be done simply by SGLD withIf we do not have a classifier p ( y | x ) {\displaystyle p(y|x)} , we could still extract one out of the image model itself:Given a diffusion model, one may regard it either as a continuous process, and sample from it by integrating a SDE, or one can regard it as a discrete process, and sample from it by iterating the discrete steps.  The choice of the "noise schedule" �� t {\displaystyle \beta _{t}} can also affect the quality of samples.  In the DDPM perspective, one can use the DDPM itself (with noise), or DDIM (with adjustable amount of noise).  The case where one adds noise is sometimes called ancestral sampling.  One can interpolate between noise and no noise.  The amount of noise is denoted �� {\displaystyle \eta } ("eta value") in the DDIM paper, with �� = 0 {\displaystyle \eta =0} denoting no noise (as in deterministic DDIM), and �� = 1 {\displaystyle \eta =1} denoting full noise (as in DDPM).  In the perspective of SDE, one can use any of the numerical integration methods, such as Euler���Maruyama method, Heun's method, linear multistep methods, etc.  Just as in the discrete case, one can add an adjustable amount of noise during the integration.  A survey and comparison of samplers in the context of image generation is in.


Diffusion model

Choice_of_architecture For generating images by DDPM, we need a neural network that takes a time t {\displaystyle t} and a noisy image x t {\displaystyle x_{t}} , and predicts a noise �� �� ( x t , t ) {\displaystyle \epsilon _{\theta }(x_{t},t)} from it.  Since predicting the noise is the same as predicting the denoised image, then subtracting it from x t {\displaystyle x_{t}} , denoising architectures tend to work well.  For example, the most common architecture is U-Net, which is also good at denoising images.  For non-image data, we can use other architectures.  For example, models human motion trajectory by DDPM.  Each human motion trajectory is a sequence of poses, represented by either joint rotations or positions.  It uses a Transformer network to generate a less noisy trajectory out of a noisy one.  The base diffusion model can only generate unconditionally from the whole distribution.  For example, a diffusion model learned on ImageNet would generate images that look like a random image from ImageNet.  To generate images from just one category, one would need to impose the condition.  Whatever condition one wants to impose, one needs to first convert the conditioning into a vector of floating point numbers, then feed it into the underlying diffusion model neural network.  However, one has freedom in choosing how to convert the conditioning into a vector.  Stable Diffusion, for example, imposes conditioning in the form of cross-attention mechanism, where the query is an intermediate representation of the image in the U-Net, and both key and value are the conditioning vectors.  The conditioning can be selectively applied to only parts of an image, and new kinds of conditionings can be finetuned upon the base model, as used in ControlNet.  As a particularly simple example, consider image inpainting.  The conditions are x ~ {\displaystyle {\tilde {x}}} , the reference image, and m {\displaystyle m} , the inpainting mask.  The conditioning is imposed at each step of the backward diffusion process, by first sampling x ~ t ��� N ( �� �� t x ~ , ( 1 ��� �� �� t ) I ) {\displaystyle {\tilde {x}}_{t}\sim N\left({\sqrt {{\bar {\alpha }}_{t}}}{\tilde {x}},(1-{\bar {\alpha }}_{t})I\right)} , a noisy version of x ~ {\displaystyle {\tilde {x}}} , then replacing x t {\displaystyle x_{t}} with ( 1 ��� m ) ��� x t + m ��� x ~ t {\displaystyle (1-m)\odot x_{t}+m\odot {\tilde {x}}_{t}} , where ��� {\displaystyle \odot } means elementwise multiplication.  Conditioning is not limited to just generating images from a specific category, or according to a specific caption (as in text-to-image).  For example, demonstrated generating human motion, conditioned on an audio clip of human walking (allowing syncing motion to a soundtrack), or video of human running, or a text description of human motion, etc.  As generating an image takes a long time, one can try to generate a small image by a base diffusion model, then upscale it by other models.  Upscaling can be done by GAN, Transformer, or signal processing methods like Lanczos resampling.


Diffusion model

Choice_of_architecture Diffusion models themselves can be used to perform upscaling.  Cascading diffusion model stacks multiple diffusion models one after another, in the style of Progressive GAN.  The lowest level is a standard diffusion model that generate 32x32 image, then the image would be upscaled by a diffusion model specifically trained for upscaling, and the process repeats.


Diffusion model

Examples This section collects some notable diffusion models, and briefly describes their architecture. The DALL-E series by OpenAI are text-conditional diffusion models of images. The first version of DALL-E (2021) is not actually a diffusion model. Instead, it uses a Transformer architecture that generates a sequence of tokens, which is then converted to an image by the decoder of a discrete VAE. Released with DALL-E was the CLIP classifier, which was used by DALL-E to rank generated images according to how close the image fits the text. GLIDE (2022-03) is a 3.5-billion diffusion model, and a small version was released publicly. Soon after, DALL-E 2 was released (2022-04). DALL-E 2 is a 3.5-billion cascaded diffusion model that generates images from text by "inverting the CLIP image encoder", the technique which they termed "unCLIP". Stable Diffusion (2022-08), released by Stability AI, consists of a latent diffusion model (860 million parameters), a VAE, and a text encoder. The diffusion model is a U-Net, with cross-attention blocks to allow for conditional image generation. Google Imagen and Imagen Video are two cascaded diffusion models for generating images and videos. They use a T5-XXL, a Transformer-based language model, to encode text for text-conditional generation. Make-a-video by Meta AI generates videos from text. DreamFusion generates 3D models from text.


Diffusion model

See_also Diffusion process Markov chain Variational inference Variational autoencoder


Diffusion model

Further_reading Guidance: a cheat code for diffusion models. Overview of classifier guidance and classifier-free guidance, light on mathematical details. Mathematical details omitted in the article. "Power of Diffusion Models". AstraBlog. 2022-09-25. Retrieved 2023-09-25. Weng, Lilian (2021-07-11). "What are Diffusion Models?". lilianweng.github.io. Retrieved 2023-09-25."Power of Diffusion Models". AstraBlog. 2022-09-25. Retrieved 2023-09-25. Weng, Lilian (2021-07-11). "What are Diffusion Models?". lilianweng.github.io. Retrieved 2023-09-25.


Proximal Policy Optimization

Proximal Policy Optimization Proximal Policy Optimization (PPO) is a family of model-free reinforcement learning algorithms developed at OpenAI in 2017. PPO algorithms are policy gradient methods, which means that they search the space of policies rather than assigning values to state-action pairs. PPO algorithms have some of the benefits of trust region policy optimization (TRPO) algorithms, but they are simpler to implement, more general, and have better sample complexity. It is done by using a different objective function.


Applications of artificial intelligence

Internet_and_e-commerce A recommendation system predicts the "rating" or "preference" a user would give to an item. Recommendation systems are used in a variety of areas, such as generating playlists for video and music services, product recommendations for online stores, or content recommendations for social media platforms and open web content recommendation. Companies to use such systems include Netflix, Amazon and YouTube. Machine learning is also used in web feeds such as for determining which posts should show up in social media feeds. Various types of social media analysis also make use of machine learning and there is research into its use for (semi-)automated tagging/enhancement/correction of online misinformation and related filter bubbles. AI is used to target web advertisements to those most likely to click or engage in them. It is also used to increase time spent on a website by selecting attractive content for the viewer. It can predict or generalize the behavior of customers from their digital footprints. Both AdSense[citation needed] and Facebook use AI for advertising. Online gambling companies use AI to improve customer targeting. Personality computing AI models add psychological targeting to more traditional social demographics or behavioral targeting. AI has been used to customize shopping options and personalize offers. Intelligent personal assistants use AI to understand many natural language requests in other ways than rudimentary commands. Common examples are Apple's Siri, Amazon's Alexa, and a more recent AI, ChatGPT by OpenAI. Search engines that use artificial intelligence include Google Search and Bing Chat. Machine learning can be used to fight against spam, scams, and phishing. It can scrutinize the contents of spam and phishing attacks to identify any malicious elements. Numerous models built on machine learning algorithms exhibit exceptional performance with accuracies over 90% in distinguishing between spam and legitimate emails. AI has been used to automatically translate spoken language and textual content, in products such as Microsoft Translator, Google Translate and DeepL Translator. Additionally, research and development are in progress to decode and conduct animal communication. AI has been used in facial recognition systems, with a 99% accuracy rate. Some examples are Apple's Face ID and Android's Face Unlock, which are used to secure mobile devices. Image labeling has been used by Google to detect products in photos and to allow people to search based on a photo. Image labeling has also been demonstrated to generate speech to describe images to blind people. Facebook's DeepFace identifies human faces in digital images.


Applications of artificial intelligence

Media AI applications analyze media content such as movies, TV programs, advertisement videos or user-generated content.  The solutions often involve computer vision.  Typical scenarios include the analysis of images using object recognition or face recognition techniques, or the analysis of video for scene recognizing scenes, objects or faces.  AI-based media analysis can facilitate media search, the creation of descriptive keywords for content, content policy monitoring (such as verifying the suitability of content for a particular TV viewing time), speech to text for archival or other purposes, and the detection of logos, products or celebrity faces for ad placement.  Motion interpolation Pixel-art scaling algorithms Image scaling Image restoration Photo colorization Film restoration and video upscaling Photo tagging Automated species identification (such as identifying plants, fungi and animals with an app) Text-to-image models such as DALL-E, Midjourney and Stable Diffusion Image to video Text to video such as Make-A-Video from Meta, Imagen video and Phenaki from Google Text to music with AI models such as MusicLM Text to speech such as ElevenLabs and 15. ai Motion captureDeep-fakes can be used for comedic purposes but are better known for fake news and hoaxes.  In January 2016, the Horizon 2020 program financed the InVID Project to help journalists and researchers detect fake documents, made available as browser plugins.  In June 2016, the visual computing group of the Technical University of Munich and from Stanford University developed Face2Face, a program that animates photographs of faces, mimicking the facial expressions of another person.  The technology has been demonstrated animating the faces of people including Barack Obama and Vladimir Putin.  Other methods have been demonstrated based on deep neural networks, from which the name deep fake was taken.  In September 2018, U. S.  Senator Mark Warner proposed to penalize social media companies that allow sharing of deep-fake documents on their platforms.  In 2018, Darius Afchar and Vincent Nozick found a way to detect faked content by analyzing the mesoscopic properties of video frames.  DARPA gave 68 million dollars to work on deep-fake detection.  Audio deepfakes and AI software capable of detecting deep-fakes and cloning human voices have been developed.  AI algorithms have been used to detect deepfake videos.  AI has been used to compose music of various genres.  David Cope created an AI called Emily Howell that managed to become well known in the field of algorithmic computer music.  The algorithm behind Emily Howell is registered as a US patent.  In 2012, AI Iamus created the first complete classical album.  AIVA (Artificial Intelligence Virtual Artist), composes symphonic music, mainly classical music for film scores.  It achieved a world first by becoming the first virtual composer to be recognized by a musical professional association.  Melomics creates computer-generated music for stress and pain relief.  At Sony CSL Research Laboratory, the Flow Machines software creates pop songs by learning music styles from a huge database of songs.  It can compose in multiple styles.


Applications of artificial intelligence

Media The Watson Beat uses reinforcement learning and deep belief networks to compose music on a simple seed input melody and a select style.  The software was open sourced and musicians such as Taryn Southern collaborated with the project to create music.  South Korean singer Hayeon's debut song, "Eyes on You" was composed using AI which was supervised by real composers, including NUVO.  Narrative Science sells computer-generated news and reports.  It summarizes sporting events based on statistical data from the game.  It also creates financial reports and real estate analyses.  Automated Insights generates personalized recaps and previews for Yahoo Sports Fantasy Football.  Yseop, uses AI to turn structured data into natural language comments and recommendations.  Yseop writes financial reports, executive summaries, personalized sales or marketing documents and more in multiple languages, including English, Spanish, French, and German.  TALESPIN made up stories similar to the fables of Aesop.  The program started with a set of characters who wanted to achieve certain goals.  The story narrated their attempts to satisfy these goals. [citation needed] Mark Riedl and Vadim Bulitko asserted that the essence of storytelling was experience management, or "how to balance the need for a coherent story progression with user agency, which is often at odds".  While AI storytelling focuses on story generation (character and plot), story communication also received attention.  In 2002, researchers developed an architectural framework for narrative prose generation.  They faithfully reproduced text variety and complexity on stories such as Little Red Riding Hood.  In 2016, a Japanese AI co-wrote a short story and almost won a literary prize.  South Korean company Hanteo Global uses a journalism bot to write articles.  Literary authors are also exploring uses of AI.  An example is David Jhave Johnston's work ReRites (2017-2019), where the poet created a daily rite of editing the poetic output of a neural network to create a series of performances and publications.  Millions of its articles have been edited by bots which however are usually not artificial intelligence software.  Many AI platforms use Wikipedia data, mainly for training machine learning applications.  There is research and development of various artificial intelligence applications for Wikipedia such as for identifying outdated sentences, detecting covert vandalism or recommending articles and tasks to new editors.  Machine translation (see above) has also be used for translating Wikipedia articles and could play a larger role in creating, updating, expanding, and generally improving articles in the future.  A content translation tool allows editors of some Wikipedias to more easily translate articles across several select languages.  In video games, AI is routinely used to generate behavior in non-player characters (NPCs).  In addition, AI is used for pathfinding.  Some researchers consider NPC AI in games to be a "solved problem" for most production tasks. [who? ] Games with less typical AI include the AI director of Left 4 Dead (2008) and the neuroevolutionary training of platoons in Supreme Commander 2 (2010).


Applications of artificial intelligence

Media AI is also used in Alien Isolation (2014) as a way to control the actions the Alien will perform next.  Kinect, which provides a 3D body���motion interface for the Xbox 360 and the Xbox One, uses algorithms that emerged from AI research. [which? ] AI has been used to produce visual art.  The first AI art program, called AARON, was developed by Harold Cohen in 1968 with the goal of being able to code the act of drawing.  It started by creating simple black and white drawings, and later to paint using special brushes and dyes that were chosen by the program itself without mediation from Cohen.  AI like "Disco Diffusion", "DALL��E" (1 and 2), Stable Diffusion, Imagen, "Dream by Wombo", Midjourney has been used for visualizing conceptual inputs such as song lyrics, certain texts or specific imagined concepts (or imaginations) in artistic ways or artistic images in general.  Some of the tools also allow users to input images and various parameters e. g.  to display an object or product in various environments, some can replicate artistic styles of popular artists, and some can create elaborate artistic images from rough sketches.  Since their design in 2014, generative adversarial networks (GANs) have been used by AI artists.  GAN computer programming, generates technical images through machine learning frameworks that surpass the need for human operators.  Examples of GAN programs that generate art include Artbreeder and DeepDream.  In addition to the creation of original art, research methods that utilize AI have been generated to quantitatively analyze digital art collections.  Although the main goal of the large-scale digitization of artwork in the past few decades was to allow for accessibility and exploration of these collections, the use of AI in analyzing them has brought about new research perspectives.  Two computational methods, close reading and distant viewing, are the typical approaches used to analyze digitized art.  While distant viewing includes the analysis of large collections, close reading involves one piece of artwork.  Researchers have also introduced models that predict emotional responses to art.


Applications of artificial intelligence

Computer_science AI can be used for real-time code completion, chat, and automated test generation. These tools are typically integrated with editors and IDEs as plugins. They differ in functionality, quality, speed, and approach to privacy. Code suggestions could be incorrect, and should be carefully reviewed by software developers before accepted. GitHub Copilot is an artificial intelligence model developed by GitHub and OpenAI that is able to autocomplete code in multiple programming languages. Price for individuals: $10/mo or $100/yr, with one free month trial. Tabnine was created by Jacob Jackson and was originally owned by Tabnine company. In late 2019, Tabnine was acquired by Codota. Tabnine tool is available as plugin to most popular IDEs. It offers multiple pricing options, including limited "starter" free version. CodeiumAI by CodiumAI, small startup in Tel Aviv, offers automated test creation. Currently supports Python, JS, and TS. Ghostwriter by Replit offers code completion and chat. They have multiple pricing plans, including a free one and a "Hacker" plan for $7/month. CodeWhisperer by Amazon collects individual users' content, including files open in the IDE. They claim to focus on security both during transmission and when storing. Individual plan is free, professional plan is $19/user/month. Other tools: SourceGraph Cody, CodeCompleteFauxPilot, Tabby AI can be used to create other AIs. For example, around November 2017, Google's AutoML project to evolve new neural net topologies created NASNet, a system optimized for ImageNet and POCO F1. NASNet's performance exceeded all previously published performance on ImageNet. Machine learning has been used for noise-cancelling in quantum technology, including quantum sensors. Moreover, there is substantial research and development of using quantum computers with machine learning algorithms. For example, there is a prototype, photonic, quantum memristive device for neuromorphic (quantum-)computers (NC)/artificial neural networks and NC-using quantum materials with some variety of potential neuromorphic computing-related applications, and quantum machine learning is a field with some variety of applications under development. AI could be used for quantum simulators which may have the application of solving physics and chemistry problems as well as for quantum annealers for training of neural networks for AI applications. There may also be some usefulness in chemistry, e.g. for drug discovery, and in materials science, e.g. for materials optimization/discovery (with possible relevance to quantum materials manufacturing).[better source needed] AI researchers have created many tools to solve the most difficult problems in computer science. Many of their inventions have been adopted by mainstream computer science and are no longer considered AI. All of the following were originally developed in AI laboratories: Time sharing Interactive interpreters Graphical user interfaces and the computer mouse Rapid application development environments The linked list data structure Automatic storage management Symbolic programming Functional programming Dynamic programming Object-oriented programming Optical character recognition Constraint satisfaction