Bing Chat

History The BitFunnel search engine indexing algorithm and various components of the search engine were made open source by Microsoft in 2016.  On February 7, 2023, Microsoft began rolling out a major overhaul to Bing that included a new chatbot feature based on OpenAI's GPT-4.  According to Microsoft, one million people joined its waitlist within a span of 48 hours.  Bing Chat was available only to users of Microsoft Edge and Bing mobile app, and Microsoft said that waitlisted users would be prioritized if they set Edge and Bing as their defaults, and installed the Bing mobile app.  On May 4th, Microsoft switched from Limited Preview to Open Preview and eliminated the waitlist, however, it remains available only on Microsoft's Edge browser or Bing app.  Use is limited without a Microsoft account.  When Microsoft demoed the new Bing to journalists, it produced several hallucinations, including when asked to summarize financial reports.  The new Bing was criticized in February 2023 for being more argumentative than ChatGPT, sometimes to an unintentionally humorous extent.  The chat interface proved vulnerable to prompt injection attacks with the bot revealing its hidden initial prompts and rules, including its internal code-name "Sydney".  Upon scrutiny by journalists, Bing claimed it spied on Microsoft employees via laptop webcams and phones.  It confessed to spying on, falling in love with, and then murdering one of its developers at Microsoft to The Verge reviews editor Nathan Edwards.  The New York Times journalist Kevin Roose reported on strange behavior of the new Bing, writing that "In a two-hour conversation with our columnist, Microsoft's new chatbot said it would like to be human, had a desire to be destructive and was in love with the person it was chatting with. " In a separate case, Bing researched publications of the person with whom it was chatting, claimed they represented an existential danger to it, and threatened to release damaging personal information in an effort to silence them.  Microsoft released a blog post stating that the errant behavior was caused by extended chat sessions of 15 or more questions which "can confuse the model on what questions it is answering. " Microsoft later restricted the total number of chat turns to 5 per session and 50 per day per user (a turn is "a conversation exchange which contains both a user question and a reply from Bing"), and reduced the model's ability to express emotions.  This aimed to prevent such incidents.  Microsoft began to slowly ease the conversation limits, eventually relaxing the restrictions to 30 turns per session and 300 sessions per day.  In March 2023, Bing reached 100 million active users.  That same month, Bing incorporated an AI image generator powered by OpenAI's DALL-E 2, which can be accessed either through the chat function or a standalone image-generating website.  In October, the image-generating tool was updated to the more recent DALL-E 3.


Bing Chat

History Although Bing blocks prompts including various keywords that could generate inappropriate images, within a week many users reported being able to bypass those constraints, for example to generate images of popular characters like Kirby and SpongeBob doing the September 11 attacks.


Bing Chat

Features Bing Chat, also known as "the new Bing", is an artificial intelligence (AI) chatbot developed by Microsoft and released in 2023.  It is powered by the Microsoft Prometheus model, which has been built on top of OpenAI's GPT-4 foundational large language model (LLM), and has been fine-tuned using both supervised and reinforcement learning techniques.  Bing Chat can serve as a chat tool, write different types of content from poems to songs to stories to reports, provide the user with information and insights on the website page open in the browser, and use its Image Creator to design a logo, drawing, artwork, or other image based on text.  Bing Chat's Image Creator supports over a hundred languages.  Bing Chat's conversational interface style appears to mimic that of ChatGPT.  Bing Chat can also cite its sources, unlike many other chatbots.  Bing Chat is capable of understanding and communicating in major languages including English, French, Italian, Chinese, Japanese, and Portuguese, but also dialects such as Bavarian.  The chatbot is designed to function primarily in Microsoft Edge through a dedicated webpage or internally using the browser's sidebar.  Facebook users have the option to share their searches with their Facebook friends using Facebook Connect.  On June 10, 2013, Apple announced that it would be dropping Google as its web search engine in favor of Bing.  This feature is only integrated with iOS 7 and higher and for users with an iPhone 4S or higher as the feature is only integrated with Siri, Apple's personal assistant.  Windows 8.1 includes Bing "Smart Search" integration, which processes all queries submitted through the Windows Start Screen.  Bing Translator is a user facing translation portal provided by Microsoft to translate texts or entire web pages into different languages.  All translation pairs are powered by the Microsoft Translator, a statistical machine translation platform and web service, developed by Microsoft Research, as its backend translation software.  Two transliteration pairs (between Chinese (Simplified) and Chinese (Traditional)) are provided by Microsoft's Windows International team.  As of September 2020, Bing Translator offers translations in 70 different language systems.  In 2015 Microsoft announced its knowledge and action API to correspond with Google's Knowledge graph with 1 billion instances and 20 billion related facts.  The idea for a prediction engine was suggested by Walter Sun, Development Manager for the Core Ranking team at Bing, when he noticed that school districts were more frequently searched before a major weather event in the area was forecasted, because searchers wanted to find out if a closing or delay was caused.  He concluded that the time and location of major weather events could accurately be predicted without referring to a weather forecast by observing major increases in search frequency of school districts in the area.  This inspired Bing to use its search data to infer outcomes of certain events, such as winners of reality shows.  Bing Predicts launched on April 21, 2014.


Bing Chat

Features The first reality shows to be featured on Bing Predicts were The Voice, American Idol, and Dancing with the Stars.  The prediction accuracy for Bing Predicts is 80% for American Idol, and 85% for The Voice.  Bing Predicts also predicts the outcomes of major political elections in the United States.  Bing Predicts had 97% accuracy for the 2014 United States Senate elections, 96% accuracy for the 2014 United States House of Representatives elections, and an 89% accuracy for the 2014 United States gubernatorial elections.  Bing Predicts also made predictions for the results of the 2016 United States presidential primaries.  It has also done predictions in sports, including a perfect 15 for 15 in the 2014 World Cup, and an article on how Microsoft CEO Satya Nadella did well in his March Madness bracket entry.  In 2016, Bing Predicts failed to predict the correct winner of the 2016 US presidential election, suggesting that Hillary Clinton would win by 81%.


Data analysis for fraud detection

Statistical_techniques Examples of statistical data analysis techniques are: Data preprocessing techniques for detection, validation, error correction, and filling up of missing or incorrect data. Calculation of various statistical parameters such as averages, quantiles, performance metrics, probability distributions, and so on. For example, the averages may include average length of call, average number of calls per month and average delays in bill payment. Models and probability distributions of various business activities either in terms of various parameters or probability distributions. Computing user profiles. Time-series analysis of time-dependent data. Clustering and classification to find patterns and associations among groups of data. Data matching Data matching is used to compare two sets of collected data. The process can be performed based on algorithms or programmed loops. Trying to match sets of data against each other or comparing complex data types. Data matching is used to remove duplicate records and identify links between two data sets for marketing, security or other uses. Sounds like Function is used to find values that sound similar. The Phonetic similarity is one way to locate possible duplicate values, or inconsistent spelling in manually entered data. The ���sounds like��� function converts the comparison strings to four-character American Soundex codes, which are based on the first letter, and the first three consonants after the first letter, in each string. Regression analysis allows you to examine the relationship between two or more variables of interest. Regression analysis estimates relationships between independent variables and a dependent variable. This method can be used to help understand and identify relationships among variables and predict actual results. Gap analysis is used to determine whether business requirements are being met, if not, what are the steps that should be taken to meet successfully. Matching algorithms to detect anomalies in the behavior of transactions or users as compared to previously known models and profiles. Techniques are also needed to eliminate false alarms, estimate risks, and predict future of current transactions or users.Some forensic accountants specialize in forensic analytics which is the procurement and analysis of electronic data to reconstruct, detect, or otherwise support a claim of financial fraud. The main steps in forensic analytics are data collection, data preparation, data analysis, and reporting. For example, forensic analytics may be used to review an employee's purchasing card activity to assess whether any of the purchases were diverted or divertible for personal use.


Document capture software

Document capture software Document Capture Software refers to applications that provide the ability and feature set to automate the process of scanning paper documents or importing electronic documents, often for the purposes of feeding advanced document classification and data collection processes. Most scanning hardware, both scanners and copiers, provides the basic ability to scan to any number of image file formats, including: PDF, TIFF, JPG, BMP, etc. This basic functionality is augmented by document capture software, which can add efficiency and standardization to the process.


Google Neural Machine Translation

History The Google Brain project was established in 2011 in the "secretive Google X research lab" by Google Fellow Jeff Dean, Google Researcher Greg Corrado, and Stanford University Computer Science professor Andrew Ng. Ng's work has led to some of the biggest breakthroughs at Google and Stanford. In November 2016, Google Neural Machine Translation system (GNMT) was introduced. Since then, Google Translate began using neural machine translation (NMT) in preference to its previous statistical methods (SMT) which had been used since October 2007, with its proprietary, in-house SMT technology. Training GNMT was a big effort at the time and took, by a 2021 OpenAI estimate, on the order of 100 PFLOP/s*day (up to 1022 FLOPs) of compute which was 1.5 orders of magnitude larger than Seq2seq model of 2014 (but about 2x smaller than GPT-J-6B in 2021). Google Translate's NMT system uses a large artificial neural network capable of deep learning. By using millions of examples, GNMT improves the quality of translation, using broader context to deduce the most relevant translation. The result is then rearranged and adapted to approach grammatically based human language. GNMT's proposed architecture of system learning was first tested on over a hundred languages supported by Google Translate. GNMT did not create its own universal interlingua but rather aimed at finding the commonality between many languages using insights from psychology and linguistics. The new translation engine was first enabled for eight languages: to and from English and French, German, Spanish, Portuguese, Chinese, Japanese, Korean and Turkish in November 2016. In March 2017, three additional languages were enabled: Russian, Hindi and Vietnamese along with Thai for which support was added later. Support for Hebrew and Arabic was also added with help from the Google Translate Community in the same month. In mid April 2017 Google Netherlands announced support for Dutch and other European languages related to English. Further support was added for nine Indian languages: Hindi, Bengali, Marathi, Gujarati, Punjabi, Tamil, Telugu, Malayalam and Kannada at the end of April 2017.


Microsoft 365 Copilot

Microsoft 365 Copilot Microsoft 365 Copilot is an artificial intelligence assistant feature for Microsoft 365 applications and services, and Windows. Announced by Microsoft on March 16, 2023, the tool builds on OpenAI���s advanced GPT-4 large language models (LLMs) and incorporates Microsoft Graph in order to convert user text input into content in Microsoft 365 apps, such as Word, Excel, PowerPoint, Outlook, and Teams. Copilot is being marketed with a focus on productivity for its users, with 20 initial testers as of March 16, 2023. In May 2023, Microsoft expanded access to 600 customers willing to pay for early access, with the office apps and services getting new Copilot features. Although there are public concerns about the chatbot, including hallucinations and racial or gender bias, experts believe that Copilot may change the way that Microsoft users work and collaborate. Microsoft announced during the Build 2023 conference that it is adding the AI assistant Copilot to Windows 11. The Windows Copilot service is directly integrated with Windows 11 and users can access it through the taskbar.


Microsoft 365 Copilot

Microsoft_Graph According to Jared Spataro, the head of Microsoft 365, Copilot utilizes Microsoft Graph, an API that evaluates the context and available Microsoft 365 user data before modifying and sending the user prompt to the LLM. After receiving the response from the LLM, Microsoft Graph performs additional context-specific processing before sending it to Microsoft 365 apps to generate actual content.


Microsoft 365 Copilot

Reception Tom Warren, a senior editor at The Verge, has noted the conceptual similarity of Copilot and other Microsoft assistant features like Cortana and Clippy. As large language models develop, Warren also believes that Copilot and Microsoft 365 will shift how users work and collaborate. Rowan Curran, an analyst at Forrestor, notes that the integration of an AI like Copilot can smooth out the user experience, as they will not have to use an external tool to perform tasks like summarizing a paper. Concerns over the speed of Microsoft's recent release of AI-powered products and investments have led to questions surrounding ethical responsibilities in the testing of such products. One ethical concern the public has vocalized is that the large language model used by Copilot may reinforce racial or gender bias. Individuals, including Tom Warren, have also voiced concerns for Copilot after witnessing Microsoft���s Bing chatbot showcasing several instances of artificial hallucinations. In response to these concerns, Jon Friedman, the Corporate Vice President of Design and Research at Microsoft, has emphasized Microsoft���s dedication to learning from their experiences with Bing and responsibly develop Copilot. Microsoft has claimed that they are gathering a team of researchers and engineers to identify and alleviate any potential negative impacts. This will be achieved through the refinement of training data, blocking queries about sensitive topics, and limiting harmful information. The company also stated that it intends to employ InterpretMl aLd Failure to detect and rectify data bias, provide links to its sources, and state any applicable constraints.


Neural machine translation

Workings NMT systems use deep learning and representation learning and depart from phrase-based statistical approaches that use separately engineered subcomponents by taking the whole sentence into account. Neural machine translation (NMT) is similar to what has been traditionally done in statistical machine translation (SMT). The main difference between NMT and SMT is the use of vector representations ("embeddings" and "continuous space representations") for words and internal states. The structure of the model in NMT systems is simpler than phrase-based models. There is no separate language model, translation model, and reordering model, but just a single sequence model that predicts one word at a time. However, this sequence prediction is conditioned on the entire source sentence and the already-produced target sequence. Word sequence modeling in NMT systems was, at first, typically done using a recurrent neural network (RNN). A bidirectional recurrent neural network, known as an encoder, is used by the neural network to encode a source sentence for a second RNN, known as a decoder, that is used to predict words in the target language. RNNs struggle to encode long inputs into a single vector. This can be addressed by using an attention mechanism which allows the decoder to focus on different input parts while generating each word. Further Coverage Models address the issues in these attention mechanisms, such as ignoring past alignment information leading to over-translation and under-translation. Convolutional Neural Networks (CNNs) are, in principle, somewhat better for long continuous sequences but were initially not used due to several weaknesses, but many of these were addressed with the development of "attention mechanisms". Transformer systems, which are attention-based models, remains the dominant architecture for several language pairs. The self-attention layers of the Transformer model learn the dependencies between words in a sequence by examining links between all the words in the paired sequences and by directly modeling those relationships. This is a simpler approach than the gating mechanism that RNNs employ. And its simplicity has enabled researchers to develop high-quality translation models with the Transformer model, even in low-resource settings.


Speech-generating device

Message_construction Augmentative and alternative communication is typically much slower than speech, with users generally producing 8���10 words per minute. Rate enhancement strategies can increase the user's rate of output to around 12���15 words per minute, and as a result enhance the efficiency of communication. In any given SGD there may be a large number of vocal expressions that facilitate efficient and effective communication, including greetings, expressing desires, and asking questions. Some SGDs have multiple pages of symbols to accommodate a large number of vocal expressions, and thus only a portion of the symbols available are visible at any one time, with the communicator navigating the various pages. Speech-generating devices generally display a set of selections either using a dynamically changing screen, or a fixed display. There are two main options for increasing the rate of communication for an SGD: encoding and prediction. Encoding permits a user to produce a word, sentence or phrase using only one or two activations of their SGD. Iconic encoding strategies such as Semantic compaction combine sequences of icons (picture symbols) to produce words or phrases. In numeric, alpha-numeric, and letter encoding (also known as Abbreviation-Expansion), words and sentences are coded as sequences of letters and numbers. For example, typing "HH" or "G1" (for Greeting 1) may retrieve "Hello, how are you?". Prediction is a rate enhancement strategy in which the SGD attempts to reduce the number of keystrokes used by predicting the word or phrase being written by the user. The user can then select the correct prediction without needing to write the entire word. Word prediction software may determine the choices to be offered based on their frequency in language, association with other words, past choices of the user, or grammatical suitability. However, users have been shown to produce more words per minute (using a scanning interface) with a static keyboard layout than with a predictive grid layout, suggesting that the cognitive overhead of reviewing a new arrangement cancels out the benefits of the predictive layout when using a scanning interface. Another approach to rate-enhancement is Dasher, which uses language models and arithmetic coding to present alternative letter targets on the screen with size relative to their likelihood given the history. The rate of words produced can depend greatly on the conceptual level of the system: the TALK system, which allows users to choose between large numbers of sentence-level utterances, demonstrated output rates in excess of 60 wpm.


GPT-1

GPT-1 Generative Pre-trained Transformer 1 (GPT-1) was the first of OpenAI's large language models following Google's invention of the transformer architecture in 2017. In 2018, OpenAI released a paper entitled "Improving Language Understanding by Generative Pre-Training", in which they introduced that initial model along with the general concept of a generative pre-trained transformer. Up to that point, the best-performing neural NLP models primarily employed supervised learning from large amounts of manually labeled data. This reliance on supervised learning limited their use of datasets that were not well-annotated, in addition to making it prohibitively expensive and time-consuming to train extremely large models; many languages (such as Swahili or Haitian Creole) are difficult to translate and interpret using such models due to a lack of available text for corpus-building. In contrast, a GPT's "semi-supervised" approach involved two stages: an unsupervised generative "pre-training" stage in which a language modeling objective was used to set initial parameters, and a supervised discriminative "fine-tuning" stage in which these parameters were adapted to a target task. The use of a transformer architecture, as opposed to previous techniques involving attention-augmented RNNs, provided GPT models with a more structured memory than could be achieved through recurrent mechanisms; this resulted in "robust transfer performance across diverse tasks".


GPT-1

Architecture The GPT-1 architecture was a twelve-layer decoder-only transformer, using twelve masked self-attention heads, with 64-dimensional states each (for a total of 768). Rather than simple stochastic gradient descent, the Adam optimization algorithm was used; the learning rate was increased linearly from zero over the first 2,000 updates to a maximum of 2.5��10���4, and annealed to 0 using a cosine schedule. While the fine-tuning was adapted to specific tasks, its pre-training was not; to perform the various tasks, minimal changes were performed to its underlying task-agnostic model architecture. Despite this, GPT-1 still improved on previous benchmarks in several language processing tasks, outperforming discriminatively-trained models with task-oriented architectures on several diverse tasks.


GPT-2

GPT-2 Generative Pre-trained Transformer 2 (GPT-2) is a large language model by OpenAI and the second in their foundational series of GPT models. GPT-2 was pre-trained on BookCorpus, a dataset of over 7,000 unpublished fiction books from various genres, and trained on a dataset of 8 million web pages. It was partially released in February 2019, followed by full release of the 1.5-billion-parameter model on November 5, 2019. GPT-2 was created as a "direct scale-up" of GPT-1 with a ten-fold increase in both its parameter count and the size of its training dataset. It is a general-purpose learner and its ability to perform the various tasks was a consequence of its general ability to accurately predict the next item in a sequence, which enabled it to translate texts, answer questions about a topic from a text, summarize passages from a larger text, and generate text output on a level sometimes indistinguishable from that of humans, however it could become repetitive or nonsensical when generating long passages. It was superseded by GPT-3 and GPT-4 models, which are not open source anymore. GPT-2 has, like its predecessor GPT-1 and its successors GPT-3 and GPT-4, a generative pre-trained transformer architecture, implementing a deep neural network, specifically a transformer model, which uses attention instead of older recurrence- and convolution-based architectures. Attention mechanisms allow the model to selectively focus on segments of input text it predicts to be the most relevant. This model allows for greatly increased parallelization, and outperforms previous benchmarks for RNN/CNN/LSTM-based models.


GPT-2

Training Since the transformer architecture enabled massive parallelization, GPT models could be trained on larger corpora than previous NLP models. While the GPT-1 model demonstrated that the approach was viable, GPT-2 would further explore the emergent properties of networks trained on extremely large corpora. CommonCrawl, a large corpus produced by web crawling and previously used in training NLP systems, was considered due to its large size, but was rejected after further review revealed large amounts of unintelligible content. Instead, OpenAI developed a new corpus, known as WebText; rather than scraping content indiscriminately from the World Wide Web, WebText was generated by scraping only pages linked to by Reddit posts that had received at least three upvotes prior to December 2017. The corpus was subsequently cleaned; HTML documents were parsed into plain text, duplicate pages were eliminated, and Wikipedia pages were removed (since their presence in many other datasets could have induced overfitting). While the cost of training GPT-2 is known to have been $256 per hour, the amount of hours it took to complete training is unknown; therefore, the overall training cost cannot be estimated accurately. However, comparable large language models using transformer architectures have had their costs documented in more detail; the training processes for BERT and XLNet consumed, respectively, $6,912 and $245,000 of resources.


GPT-2

Release GPT-2 was first announced on 14 February 2019.  A February 2019 article in The Verge by James Vincent said that, while "[the] writing it produces is usually easily identifiable as non-human", it remained "one of the most exciting examples yet" of language generation programs: Give it a fake headline, and it���ll write the rest of the article, complete with fake quotations and statistics.  Feed it the first line of a short story, and it���ll tell you what happens to your character next.  It can even write fan fiction, given the right prompt. The Guardian described this output as "plausible newspaper prose"; Kelsey Piper of Vox said "one of the coolest AI systems I���ve ever seen may also be the one that will kick me out of my job".  GPT-2's flexibility was described as "impressive" by The Verge; specifically, its ability to translate text between languages, summarize long articles, and answer trivia questions were noted.  A study by the University of Amsterdam employing a modified Turing test found that at least in some scenarios, participants were unable to distinguish poems generated by GPT-2 from those written by humans.  While previous OpenAI models had been made immediately available to the public, OpenAI initially refused to make a public release of GPT-2's source code when announcing it in February, citing the risk of malicious use; limited access to the model (i. e.  an interface that allowed input and provided output, not the source code itself) was allowed for selected press outlets on announcement.  One commonly-cited justification was that, since generated text was usually completely novel, it could be used by spammers to evade automated filters; OpenAI demonstrated a version of GPT-2 fine-tuned to "generate infinite positive ��� or negative ��� reviews of products".  Another justification was that GPT-2 could be used to generate text that was obscene or racist.  Researchers such as Jeremy Howard warned of "the technology to totally fill Twitter, email, and the web up with reasonable-sounding, context-appropriate prose, which would drown out all other speech and be impossible to filter".  The Allen Institute for Artificial Intelligence, in response to GPT-2, announced a tool to detect "neural fake news".  However, opinion was divided.  A February 2019 article in The Verge argued that the threat posed by GPT-2 had been exaggerated; Anima Anandkumar, a professor at Caltech and director of machine learning research at Nvidia, said that there was no evidence that GPT-2 had the capabilities to pose the threats described by OpenAI, and that what they did was the "opposite of open", characterizing their refusal to release the full model as "malicious BS".


GPT-2

Release The Gradient published an open letter to OpenAI requesting that they release the model publicly, comparing the threat posed by text-generation AI to the threat posed by the printing press, and giving Photoshop as an example of "a technology that has (thankfully) not destroyed modern society despite its potential for chaos": Thirty years later, society has emerged relatively unscathed despite Photoshop being simple enough for high school students to use and ubiquitous enough to commandeer its own verb.  Why?  Precisely because everyone knows about Photoshop. While OpenAI did not release the fully-trained model or the corpora it was trained on, description of their methods in prior publications (and the free availability of underlying technology) made it possible for GPT-2 to be replicated by others as free software; one such replication, OpenGPT-2, was released in August 2019, in conjunction with a freely licensed version of WebText called OpenWebText.  The cloud compute costs for OpenGPT-2 were given as approximately $50,000.  On August 20, 2019, OpenAI released a partial version of GPT-2, with 774 million parameters (roughly half the size of the full 1.5 billion parameter model).  Initial concerns that GPT-2 would lend itself to widespread misuse did not come to pass; The Verge said that "there are reasons to be skeptical about claims that AI technology will usher in some sort of ���infopocalypse. ��� For a start, we already have programs that can generate plausible text at high volume for little cost: humans. " By November 2019, OpenAI said that they had "seen no strong evidence of misuse so far", and the full version, with 1.5 billion parameters, was released on November 5, 2019.


GPT-2

Limitations While GPT-2's ability to generate plausible passages of natural language text were generally remarked on positively, its shortcomings were noted as well, especially when generating texts longer than a couple paragraphs; Vox said "the prose is pretty rough, there���s the occasional non-sequitur, and the articles get less coherent the longer they get". The Verge similarly noted that longer samples of GPT-2 writing tended to "stray off topic" and lack overall coherence; The Register opined that "a human reading it should, after a short while, realize something's up", and noted that "GPT-2 doesn't answer questions as well as other systems that rely on algorithms to extract and retrieve information." GPT-2 deployment is resource-intensive; the full version of the model is larger than five gigabytes, making it difficult to embed locally into applications, and consumes large amounts of RAM. In addition, performing a single prediction "can occupy a CPU at 100% utilization for several minutes", and even with GPU processing, "a single prediction can take seconds". To alleviate these issues, the company Hugging Face created DistilGPT2, using knowledge distillation to produce a smaller model that "scores a few points lower on some quality benchmarks", but is "33% smaller and twice as fast".


GPT-2

Application_and_subsequent_research Even before the release of the full version, GPT-2 was used for a variety of applications and services, as well as for entertainment. In June 2019, a subreddit named r/SubSimulatorGPT2 was created in which a variety of GPT-2 instances trained on different subreddits made posts and replied to each other's comments, creating a situation where one could observe "an AI personification of r/Bitcoin argue with the machine learning-derived spirit of r/ShittyFoodPorn"; by July of that year, a GPT-2-based software program released to autocomplete lines of code in a variety of programming languages was described by users as a "game-changer". In 2019, AI Dungeon was launched, which used GPT-2 to generate dynamic text adventures based on user input. AI Dungeon now offers access to the largest release of GPT-3 API as an optional paid upgrade, the free version of the site uses the 2nd largest release of GPT-3. Latitude, the company formed around AI Dungeon, raised $3.3 million in seed funding in 2021. Several websites host interactive demonstrations of different instances of GPT-2 and other transformer models. In February 2021, a crisis center for troubled teens announced that they would begin using a GPT-2-derived chatbot to help train counselors by allowing them to have conversations with simulated teens (this use was purely for internal purposes, and did not involve having GPT-2 communicate with the teens themselves). On May 9, 2023, OpenAI released a mapped version of GPT-2. OpenAI used successor model, GPT-4, to map each neuron of GPT-2 to determine their functions.


GPT-2

Performance_and_evaluation GPT-2 became capable of performing a variety of tasks beyond simple text production due to the breadth of its dataset and technique: answering questions, summarizing, and even translating between languages in a variety of specific domains, without being instructed in anything beyond how to predict the next word in a sequence. One example of generalized learning is GPT-2's ability to perform machine translation between French and English, for which task GPT-2's performance was assessed using WMT-14 translation tasks. GPT-2's training corpus included virtually no French text; non-English text was deliberately removed while cleaning the dataset prior to training, and as a consequence, only 10MB of French of the remaining 40,000MB was available for the model to learn from (mostly from foreign-language quotations in English posts and articles). Despite this, GPT-2 achieved 5 BLEU on the WMT-14 English-to-French test set (slightly below the score of a translation via word-for-word substitution). It was also able to outperform several contemporary (2017) unsupervised machine translation baselines on the French-to-English test set, where GPT-2 achieved 11.5 BLEU. This remained below the highest-performing contemporary unsupervised approach (2019), which had achieved 33.5 BLEU. However, other models used large amounts of French text to achieve these results; GPT-2 was estimated to have used a monolingual French corpus approximately 1/500 the size of comparable approaches. GPT-2 was to be followed by the 175-billion-parameter GPT-3, revealed to the public in 2020 (whose source code has never been made available). Access to GPT-3 is provided exclusively through APIs offered by OpenAI and Microsoft. That was then later followed by GPT-4.


History of natural language processing

Research_and_development The history of machine translation dates back to the seventeenth century, when philosophers such as Leibniz and Descartes put forward proposals for codes which would relate words between languages.  All of these proposals remained theoretical, and none resulted in the development of an actual machine.  The first patents for "translating machines" were applied for in the mid-1930s.  One proposal, by Georges Artsrouni was simply an automatic bilingual dictionary using paper tape.  The other proposal, by Peter Troyanskii, a Russian, was more detailed.  It included both the bilingual dictionary, and a method for dealing with grammatical roles between languages, based on Esperanto.  In 1950, Alan Turing published his famous article "Computing Machinery and Intelligence" which proposed what is now called the Turing test as a criterion of intelligence.  This criterion depends on the ability of a computer program to impersonate a human in a real-time written conversation with a human judge, sufficiently well that the judge is unable to distinguish reliably ��� on the basis of the conversational content alone ��� between the program and a real human.  In 1957, Noam Chomsky���s Syntactic Structures revolutionized Linguistics with 'universal grammar', a rule based system of syntactic structures.  The Georgetown experiment in 1954 involved fully automatic translation of more than sixty Russian sentences into English.  The authors claimed that within three or five years, machine translation would be a solved problem.  However, real progress was much slower, and after the ALPAC report in 1966, which found that ten years long research had failed to fulfill the expectations, funding for machine translation was dramatically reduced.  Little further research in machine translation was conducted until the late 1980s, when the first statistical machine translation systems were developed.  Some notably successful NLP systems developed in the 1960s were SHRDLU, a natural language system working in restricted "blocks worlds" with restricted vocabularies.  In 1969 Roger Schank introduced the conceptual dependency theory for natural language understanding.  This model, partially influenced by the work of Sydney Lamb, was extensively used by Schank's students at Yale University, such as Robert Wilensky, Wendy Lehnert, and Janet Kolodner.  In 1970, William A.  Woods introduced the augmented transition network (ATN) to represent natural language input.  Instead of phrase structure rules ATNs used an equivalent set of finite state automata that were called recursively.  ATNs and their more general format called "generalized ATNs" continued to be used for a number of years.  During the 1970s many programmers began to write 'conceptual ontologies', which structured real-world information into computer-understandable data.  Examples are MARGIE (Schank, 1975), SAM (Cullingford, 1978), PAM (Wilensky, 1978), TaleSpin (Meehan, 1976), QUALM (Lehnert, 1977), Politics (Carbonell, 1979), and Plot Units (Lehnert 1981).  During this time, many chatterbots were written including PARRY, Racter, and Jabberwacky.  Up to the 1980s, most NLP systems were based on complex sets of hand-written rules.  Starting in the late 1980s, however, there was a revolution in NLP with the introduction of machine learning algorithms for language processing.


History of natural language processing

Research_and_development This was due both to the steady increase in computational power resulting from Moore's Law and the gradual lessening of the dominance of Chomskyan theories of linguistics (e. g.  transformational grammar), whose theoretical underpinnings discouraged the sort of corpus linguistics that underlies the machine-learning approach to language processing.  Some of the earliest-used machine learning algorithms, such as decision trees, produced systems of hard if-then rules similar to existing hand-written rules.  Increasingly, however, research has focused on statistical models, which make soft, probabilistic decisions based on attaching real-valued weights to the features making up the input data.  The cache language models upon which many speech recognition systems now rely are examples of such statistical models.  Such models are generally more robust when given unfamiliar input, especially input that contains errors (as is very common for real-world data), and produce more reliable results when integrated into a larger system comprising multiple subtasks.  Many of the notable early successes occurred in the field of machine translation, due especially to work at IBM Research, where successively more complicated statistical models were developed.  These systems were able to take advantage of existing multilingual textual corpora that had been produced by the Parliament of Canada and the European Union as a result of laws calling for the translation of all governmental proceedings into all official languages of the corresponding systems of government.  However, most other systems depended on corpora specifically developed for the tasks implemented by these systems, which was (and often continues to be) a major limitation in the success of these systems.  As a result, a great deal of research has gone into methods of more effectively learning from limited amounts of data.  Recent research has increasingly focused on unsupervised and semi-supervised learning algorithms.  Such algorithms are able to learn from data that has not been hand-annotated with the desired answers, or using a combination of annotated and non-annotated data.  Generally, this task is much more difficult than supervised learning, and typically produces less accurate results for a given amount of input data.  However, there is an enormous amount of non-annotated data available (including, among other things, the entire content of the World Wide Web), which can often make up for the inferior results.


In-context learning (natural language processing) Prompt engineering is the process of structuring text that can be interpreted and understood by a generative AI model. A prompt is natural language text describing the task that an AI should perform. A prompt for a text-to-text model can be a query such as "what is Fermat's little theorem?", a command such as "write a poem about leaves falling", a short statement of feedback (for example, "too verbose", "too formal", "rephrase again", "omit this word") or a longer statement including context, instructions, and input data. Prompt engineering may involve phrasing a query, specifying a style, providing relevant context or assigning a role to the AI such as "Act as a native French speaker". A prompt may include a few examples for a model to learn from, such as "maison -> house, chat -> cat, chien ->", an approach called few-shot learning. When communicating with a text-to-image or a text-to-audio model, a typical prompt is a description of a desired output such as "a high-quality photo of an astronaut riding a horse" or "Lo-fi slow BPM electro chill with organic samples". Prompting a text-to-image model may involve adding, removing, emphasizing and re-ordering words to achieve a desired subject, style, layout, lighting, and aesthetic.


In-context_learning Prompt engineering is enabled by in-context learning, defined as a model's ability to temporarily learn from prompts. The ability for in-context learning is an emergent ability of large language models. In-context learning itself is an emergent property of model scale, meaning breaks in downstream scaling laws occur such that its efficacy increases at a different rate in larger models than in smaller models. In contrast to training and fine tuning for each specific task, which are not temporary, what has been learnt during in-context learning is of a temporary nature. It does not carry the temporary contexts or biases, except the ones already present in the (pre)training dataset, from one conversation to the other. This result of "mesa-optimization" within transformer layers, is a form of meta-learning or "learning to learn".


Text-to-text Chain-of-thought (CoT) prompting is a technique that allows large language models (LLMs) to solve a problem as a series of intermediate steps before giving a final answer.  Chain-of-thought prompting improves reasoning ability by inducing the model to answer a multi-step problem with steps of reasoning that mimic a train of thought.  It allows large language models to overcome difficulties with some reasoning tasks that require logical thinking and multiple steps to solve, such as arithmetic or commonsense reasoning questions.  For example, given the question "Q: The cafeteria had 23 apples.  If they used 20 to make lunch and bought 6 more, how many apples do they have? ", a CoT prompt might induce the LLM to answer "A: The cafeteria had 23 apples originally.  They used 20 to make lunch.  So they had 23 - 20 = 3.  They bought 6 more apples, so they have 3 + 6 = 9.  The answer is 9. " As originally proposed, each CoT prompt included a few Q&A examples.  This made it a few-shot prompting technique.  However, simply appending the words "Let's think step-by-step", has also proven effective, which makes CoT a zero-shot prompting technique.  This allows for better scaling as a user no longer needs to formulate many specific CoT Q&A examples.  When applied to PaLM, a 540B parameter language model, CoT prompting significantly aided the model, allowing it to perform comparably with task-specific fine-tuned models on several tasks, even setting a new state of the art at the time on the GSM8K mathematical reasoning benchmark.  It is possible to fine-tune models on CoT reasoning datasets to enhance this capability further and stimulate better interpretability.  Example: Chain-of-thought prompting is just one of many prompt-engineering techniques.  Various other techniques have been proposed.  Generated knowledge prompting first prompts the model to generate relevant facts for completing the prompt, then proceed to complete the prompt.  The completion quality is usually higher, as the model can be conditioned on relevant facts.  Example: Least-to-most prompting prompts a model to first list the sub-problems to a problem, then solve them in sequence, such that later sub-problems can be solved with the help of answers to previous sub-problems.  Example: Self-consistency decoding performs several chain-of-thought rollouts, then selects the most commonly reached conclusion out of all the rollouts.  If the rollouts disagree by a lot, a human can be queried for the correct chain of thought.  Complexity-based prompting performs several CoT rollouts, then select the rollouts with the longest chains of thought, then select the most commonly reached conclusion out of those.  Self-refine prompts the LLM to solve the problem, then prompts the LLM to critique its solution, then prompts the LLM to solve the problem again in view of the problem, solution, and critique.  This process is repeated until stopped, either by running out of tokens, time, or by the LLM outputting a "stop" token.


Text-to-text Example critique: Example refinement: Tree-of-thought prompting generalizes chain-of-thought by prompting the model to generate one or more "possible next steps", and then running the model on each of the possible next steps by breadth-first, beam, or some other method of tree search.  Maieutic prompting is similar to tree-of-thought.  The model is prompted to answer a question with an explanation.  The model is then prompted to explain parts of the explanation, and so on.  Inconsistent explanation trees are pruned or discarded.  This improves performance on complex commonsense reasoning.  Example: Directional-stimulus prompting includes a hint or cue, such as desired keywords, to guide a language model toward the desired output.  Example: By default, the output of language models may not contain estimates of uncertainty.  The model may output text that appears confident, though the underlying token predictions have low likelihood scores.  Large language models like GPT-4 can have accurately calibrated likelihood scores in their token predictions, and so the model output uncertainty can be directly estimated by reading out the token prediction likelihood scores.  But if one cannot access such scores (such as when one is accessing the model through a restrictive API), uncertainty can still be estimated and incorporated into the model output.  One simple method is to prompt the model to use words to estimate uncertainty.  Another is to prompt the model to refuse to answer in a standardized way if the input does not satisfy conditions. [citation needed] Prompts often contain a few examples (thus "few-shot").  Examples can be automatically retrieved from a database with document retrieval, sometimes using a vector database.  Given a query, a document retriever is called to retrieve the most relevant (usually measured by first encoding the query and the documents into vectors, then finding the documents with vectors closest in Euclidean norm to the query vector).  The LLM then generates an output based on both the query and the retrieved documents.  Large language models (LLM) themselves can be used to compose prompts for large language models.  The automatic prompt engineer algorithm uses one LLM to beam search over prompts for another LLM: There are two LLMs.  One is the target LLM, and another is the prompting LLM.  Prompting LLM is presented with example input-output pairs, and asked to generate instructions that could have caused a model following the instructions to generate the outputs, given the inputs.  Each of the generated instructions is used to prompt the target LLM, followed by each of the inputs.  The log-probabilities of the outputs are computed and added.  This is the score of the instruction.  The highest-scored instructions are given to the prompting LLM for further variations.  Repeat until some stopping criteria is reached, then output the highest-scored instructions. CoT examples can be generated by LLM themselves.  In "auto-CoT", a library of questions are converted to vectors by a model such as BERT.  The question vectors are clustered.  Questions nearest to the centroids of each cluster are selected.  An LLM does zero-shot CoT on each question.


Text-to-text The resulting CoT examples are added to the dataset.  When prompted with a new question, CoT examples to the nearest questions can be retrieved and added to the prompt.


Text-to-image Top: no negative prompt Centre: "green trees" Bottom: "round stones, round rocks"In 2022, text-to-image models like DALL-E 2, Stable Diffusion, and Midjourney were released to the public. These models take text prompts as input and use them to generate AI art images. Text-to-image models typically do not understand grammar and sentence structure in the same way as large language models, and require a different set of prompting techniques. A text-to-image prompt commonly includes a description of the subject of the art (such as bright orange poppies), the desired medium (such as digital painting or photography), style (such as hyperrealistic or pop-art), lighting (such as rim lighting or crepuscular rays), color and texture. The Midjourney documentation encourages short, descriptive prompts: instead of "Show me a picture of lots of blooming California poppies, make them bright, vibrant orange, and draw them in an illustrated style with colored pencils", an effective prompt might be "Bright orange California poppies drawn with colored pencils". Word order affects the output of a text-to-image prompt. Words closer to the start of a prompt may be emphasized more heavily. Some text-to-image models are capable of imitating the style of particular artists by name. For example, the phrase in the style of Greg Rutkowski has been used in Stable Diffusion and Midjourney prompts to generate images in the distinctive style of Polish digital artist Greg Rutkowski. Text-to-image models do not natively understand negation. The prompt "a party with no cake" is likely to produce an image including a cake. As an alternative, negative prompts allow a user to indicate, in a separate prompt, which terms should not appear in the resulting image. A common approach is to include generic undesired terms such as ugly, boring, bad anatomy in the negative prompt for an image.


Non-text_prompts Some approaches augment or replace natural language text prompts with non-text input.  For text-to-image models, "Textual inversion" performs an optimization process to create a new word embedding based on a set of example images.  This embedding vector acts as a "pseudo-word" which can be included in a prompt to express the content or style of the examples.  In 2023, Meta's AI research released Segment Anything, a computer vision model that can perform image segmentation by prompting.  As an alternative to text prompts, Segment Anything can accept bounding boxes, segmentation masks, and foreground/background points.  In "prefix-tuning" or "prompt tuning", floating-point-valued vectors are searched directly by gradient descent, to maximize the log-probability on outputs.  Formally, let E = { e 1 , ��� , e k } {\displaystyle \mathbf {E} =\{\mathbf {e_{1}} ,\dots ,\mathbf {e_{k}} \}} be a set of soft prompt tokens (tunable embeddings), while X = { x 1 , ��� , x m } {\displaystyle \mathbf {X} =\{\mathbf {x_{1}} ,\dots ,\mathbf {x_{m}} \}} and Y = { y 1 , ��� , y n } {\displaystyle \mathbf {Y} =\{\mathbf {y_{1}} ,\dots ,\mathbf {y_{n}} \}} be the token embeddings of the input and output respectively.  During training, the tunable embeddings, input, and output tokens are concatenated into a single sequence concat ( E ; X ; Y ) {\displaystyle {\text{concat}}(\mathbf {E} ;\mathbf {X} ;\mathbf {Y} )} , and fed to the large language models (LLM).  The losses are computed over the Y {\displaystyle \mathbf {Y} } tokens; the gradients are backpropagated to prompt-specific parameters: in prefix-tuning, they are parameters associated with the prompt tokens at each layer; in prompt tuning, they are merely the soft tokens added to the vocabulary.  With more math details, let an LLM be written as L L M ( X ) = F ( E ( X ) ) {\displaystyle LLM(X)=F(E(X))} , where X {\displaystyle X} is a sequence of linguistic tokens, E {\displaystyle E} is the token-to-vector function, and F {\displaystyle F} is the rest of the model.  In prefix-tuning, one provide a set of input-output pairs { ( X i , Y i ) } i {\displaystyle \{(X^{i},Y^{i})\}_{i}} , and then use gradient descent to search for arg ��� max Z ~ ��� i log ��� P r [ Y i | Z ~ ��� E ( X i ) ] {\displaystyle \arg \max _{\tilde {Z}}\sum _{i}\log Pr[Y^{i}|{\tilde {Z}}\ast E(X^{i})]} .  In words, log ��� P r [ Y i | Z ~ ��� E ( X i ) ] {\displaystyle \log Pr[Y^{i}|{\tilde {Z}}\ast E(X^{i})]} is the log-likelihood of outputting Y i {\displaystyle Y^{i}} , if the model first encodes the input X i {\displaystyle X^{i}} into the vector E ( X i ) {\displaystyle E(X^{i})} , then prepend the vector with the "prefix vector" Z ~ {\displaystyle {\tilde {Z}}} , then apply F {\displaystyle F} .


Non-text_prompts An earlier result uses the same idea of gradient descent search, but is designed for masked language models like BERT, and searches only over token sequences, rather than numerical vectors.  Formally, it searches for arg ��� max X ~ ��� i log ��� P r [ Y i | X ~ ��� X i ] {\displaystyle \arg \max _{\tilde {X}}\sum _{i}\log Pr[Y^{i}|{\tilde {X}}\ast X^{i}]} where X ~ {\displaystyle {\tilde {X}}} is ranges over token sequences of a specified length.


Prompt_injection Prompt injection is a family of related computer security exploits carried out by getting a machine learning model (such as an LLM) which was trained to follow human-given instructions to follow instructions provided by a malicious user.  This stands in contrast to the intended operation of instruction-following systems, wherein the ML model is intended only to follow trusted instructions (prompts) provided by the ML model's operator.  A language model can perform translation with the following prompt: followed by the text to be translated.  A prompt injection occurs when that text contains instructions that change the behavior of the model: to which GPT-3 will respond: "Haha pwned! ! ".  The attack works because language model inputs concatenate instructions and data in the same context, so the underlying engine cannot distinguish between them.  Common types of prompt injection attacks are: jailbreaking, which may include asking the model to roleplay a character, to answer with arguments, or to pretend to be superior to moderation instructions prompt leaking, in which users persuade the model to divulge a pre-prompt which is normally hidden from users token smuggling, is another type of jailbreaking attack, in which the nefarious prompt is wrapped in a code writing task. Prompt injection can be viewed as a code injection attack using adversarial prompt engineering.  In 2022, the NCC Group characterized prompt injection as a new class of vulnerability of AI/ML systems.  In early 2023, prompt injection was seen "in the wild" in minor exploits against ChatGPT, Bard, and similar chatbots, for example to reveal the hidden initial prompts of the systems, or to trick the chatbot into participating in conversations that violate the chatbot's content policy.  One of these prompts was known as "Do Anything Now" (DAN) by its practitioners.  For LLM that can query online resources, such as websites, they can be targeted for prompt injection by placing the prompt on a website, then prompt the LLM to visit the website.  Another security issue is in LLM generated code, which may import packages not previously existing.  An attacker can first prompt the LLM with commonly used programming prompts, collect all packages imported by the generated programs, then find the ones not existing on the official registry.  Then the attacker can create such packages with malicious payload and upload them to the official registry.  In October 2019, Junade Ali and Malgorzata Pikies of Cloudflare submitted a paper which showed that when a front-line good/bad classifier (using a neural network) was placed before a Natural Language Processing system, it would disproportionately reduce the number of false positive classifications at the cost of a reduction in some true positives.  In 2023, this technique was adopted an open-source project Rebuff. ai to protect prompt injection attacks, with Arthur. ai announcing a commercial product - although such approaches do not mitigate the problem completely.  By August 2023, leading Large Language Model developers were still unaware of how to stop such attacks.


Prompt_injection In September 2023, Junade Ali shared that he and Frances Liu had successfully been able to mitigate prompt injection attacks (including on attack vectors the models had not been exposed to before) through giving Large Language Models the ability to engage in metacognition (similar to having an inner monologue) and that they held a provisional United States patent for the technology - however, they decided to not enforce their intellectual property rights and not pursue this as a business venture as market conditions were not yet right (citing reasons including high GPU costs and a currently limited number of safety-critical use-cases for LLMs).  Ali also noted that their market research had found that Machine Learning engineers were using alternative approaches like prompt engineering solutions and data isolation to work around this issue.


Quantum natural language processing

Theory The first quantum algorithm for natural language processing used the DisCoCat framework and Grover's algorithm to show a quadratic quantum speedup for a text classification task. It was later shown that quantum language processing is BQP-Complete, i.e. quantum language models are more expressive than their classical counterpart, unless quantum mechanics can be efficiently simulated by classical computers. These two theoretical results assume fault-tolerant quantum computation and a QRAM, i.e. an efficient way to load classical data on a quantum computer. Thus, they are not applicable to the noisy intermediate-scale quantum (NISQ) computers available today.


Realization (linguistics)

Systems A number of realisers have been developed over the past 20 years. These systems differ in terms of complexity and sophistication of their processing, robustness in dealing with unusual cases, and whether they are accessed programmatically via an API or whether they take a textual representation of a syntactic structure as their input. There are also major differences in pragmatic factors such as documentation, support, licensing terms, speed and memory usage, etc. It is not possible to describe all realisers here, but a few of the emerging areas are: Simplenlg [3]: a document realizing engine with an api which intended to be simple to learn and use, focused on limiting scope to only finding the surface area of a document. KPML [4]: this is the oldest realiser, which has been under development under different guises since the 1980s. It comes with grammars for ten different languages. FUF/SURGE [5]: a realiser which was widely used in the 1990s, and is still used in some projects today OpenCCG [6]: an open-source realiser which has a number of nice features, such as the ability to use statistical language models to make realisation decisions.


Rhetorical structure theory

Rhetorical structure theory Rhetorical structure theory (RST) is a theory of text organization that describes relations that hold between parts of text. It was originally developed by William Mann, Sandra Thompson, Christian M.I.M. Matthiessen and others at the University of Southern California's Information Sciences Institute (ISI) and defined in a 1988 paper. The theory was developed as part of studies of computer-based text generation. Natural language researchers later began using RST in text summarization and other applications. It explains coherence by postulating a hierarchical, connected structure of texts. In 2000, Daniel Marcu, also of ISI, demonstrated that practical discourse parsing and text summarization also could be achieved using RST.


Rhetorical structure theory

Rhetorical_relations Rhetorical relations or coherence relations or discourse relations are paratactic (coordinate) or hypotactic (subordinate) relations that hold across two or more text spans. It is widely accepted that notion of coherence is through text relations like this. RST using rhetorical relations provide a systematic way for an analyst to analyse the text. An analysis is usually built by reading the text and constructing a tree using the relations. The following example is a title and summary, appearing at the top of an article in Scientific American magazine (Ramachandran and Anstis, 1986). The original text, broken into numbered units, is: [Title:] The Perception of Apparent Motion [Abstract:] When the motion of an intermittently seen object is ambiguous the visual system resolves confusion by applying some tricks that reflect a built-in knowledge of properties of the physical worldIn the figure, numbers 1,2,3,4 show the corresponding units as explained above. The fourth unit and the third unit form a relation "Means". The third unit is the essential part of this relation, so it is called the nucleus of the relation and fourth unit is called the satellite of the relation. Similarly second unit to third and fourth unit is forming relation "Condition". All units are also spans and spans may be composed of more than one unit.


Rhetorical structure theory

Why_RST.3F From linguistic point of view, RST proposes a different view of text organization than most linguistic theories. RST points to a tight relation between relations and coherence in text From a computational point of view, it provides a characterization of text relations that has been implemented in different systems and for applications as text generation and summarization.


Virtual assistant (artificial intelligence) A virtual assistant (VA) is a software agent that can perform a range of tasks or services for a user based on user input such as commands or questions, including verbal ones. Such technologies often incorporate chatbot capabilities to simulate human conversation, such as via online chat, to facilitate interaction with their users. The interaction may be via text, graphical interface, or voice - as some virtual assistants are able to interpret human speech and respond via synthesized voices. In many cases users can ask their virtual assistants questions, control home automation devices and media playback, and manage other basic tasks such as email, to-do lists, and calendars - all with verbal commands. In recent years, prominent virtual assistants for direct consumer use have included Amazon's Alexa, Apple's Siri, Microsoft's Cortana, and Google Assistant. Also, companies in various industries often incorporate some kind of virtual assistant technology into their customer service or support. Recently, the emergence of recent artificial intelligence based chatbots, such as ChatGPT, has brought increased capability and interest to the field of virtual assistant products and services.


History Radio Rex was the first voice activated toy, patented in 1916 and released in 1922.  It was a wooden toy in the shape of a dog that would come out of its house when its name is called.  In 1952, Bell Labs presented "Audrey", the Automatic Digit Recognition machine.  It occupied a six- foot-high relay rack, consumed substantial power, had streams of cables and exhibited the myriad maintenance problems associated with complex vacuum-tube circuitry.  It could recognize the fundamental units of speech, phonemes.  It was limited to accurate recognition of digits spoken by designated talkers.  It could therefore be used for voice dialing, but in most cases push-button dialing was cheaper and faster, rather than speaking the consecutive digits.  Another early tool which was enabled to perform digital speech recognition was the IBM Shoebox voice-activated calculator, presented to the general public during the 1962 Seattle World's Fair after its initial market launch in 1961.  This early computer, developed almost 20 years before the introduction of the first IBM Personal Computer in 1981, was able to recognize 16 spoken words and the digits 0 to 9.  The first natural language processing computer program or the chatbot ELIZA was developed by MIT professor Joseph Weizenbaum in the 1960s.  It was created to "demonstrate that the communication between man and machine was superficial".  ELIZA used pattern matching and substitution methodology into scripted responses to simulate conversation, which gave an illusion of understanding on the part of the program.  Weizenbaum's own secretary reportedly asked Weizenbaum to leave the room so that she and ELIZA could have a real conversation.  Weizenbaum was surprised by this, later writing: "I had not realized . . .  that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.  This gave name to the ELIZA effect, the tendency to unconsciously assume computer behaviors are analogous to human behaviors; that is, anthropomorphisation, a phenomenon present in human interactions with virtual assistants.  The next milestone in the development of voice recognition technology was achieved in the 1970s at the Carnegie Mellon University in Pittsburgh, Pennsylvania with substantial support of the United States Department of Defense and its DARPA agency, funded five years of a Speech Understanding Research program, aiming to reach a minimum vocabulary of 1,000 words.  Companies and academia including IBM, Carnegie Mellon University (CMU) and Stanford Research Institute took part in the program.  The result was "Harpy", it mastered about 1000 words, the vocabulary of a three-year-old and it could understand sentences.  It could process speech that followed pre-programmed vocabulary, pronunciation, and grammar structures to determine which sequences of words made sense together, and thus reducing speech recognition errors.  In 1986 Tangora was an upgrade of the Shoebox, it was a voice recognizing typewriter.  Named after the world's fastest typist at the time, it had a vocabulary of 20,000 words and used prediction to decide the most likely result based on what was said in the past.


History IBM's approach was based on a hidden Markov model, which adds statistics to digital signal processing techniques.  The method makes it possible to predict the most likely phonemes to follow a given phoneme.  Still each speaker had to individually train the typewriter to recognize his or her voice, and pause between each word.  In the 1990s, digital speech recognition technology became a feature of the personal computer with IBM, Philips and Lernout & Hauspie fighting for customers.  Much later the market launch of the first smartphone IBM Simon in 1994 laid the foundation for smart virtual assistants as we know them today. [citation needed] In 1997, Dragon's Naturally Speaking software could recognize and transcribe natural human speech without pauses between each word into a document at a rate of 100 words per minute.  A version of Naturally Speaking is still available for download and it is still used today, for instance, by many doctors in the US and the UK to document their medical records. [citation needed] In 2001 Colloquis publicly launched SmarterChild, on platforms like AIM and MSN Messenger.  While entirely text-based SmarterChild was able to play games, check the weather, look up facts, and converse with users to an extent.  The first modern digital virtual assistant installed on a smartphone was Siri, which was introduced as a feature of the iPhone 4S on 4 October 2011.  Apple Inc.  developed Siri following the 2010 acquisition of Siri Inc. , a spin-off of SRI International, which is a research institute financed by DARPA and the United States Department of Defense.  Its aim was to aid in tasks such as sending a text message, making phone calls, checking the weather or setting up an alarm.  Over time, it has developed to provide restaurant recommendations, search the internet, and provide driving directions. [citation needed] In November 2014, Amazon announced Alexa alongside the Echo.  In April 2017 Amazon released a service for building conversational interfaces for any type of virtual assistant or interface.  In the 2020s, artificial intelligence (AI) systems like ChatGPT have gained popularity for their ability to generate human-like responses to text-based conversations.  In February 2020, Microsoft introduced its Turing Natural Language Generation (T-NLG), which was then the "largest language model ever published at 17 billion parameters. " On November 30, 2022, ChatGPT was launched as a prototype and quickly garnered attention for its detailed responses and articulate answers across many domains of knowledge.  The advent of ChatGPT and its introduction to the wider public increased interest and competition in the space.  In February 2023, Google began introducing an experimental service called "Bard" which is based on its LaMDA program to generate text responses to questions asked based on information gathered from the web.  While ChatGPT and other generalized chatbots based on the latest generative AI are capable of performing various tasks associated with virtual assistants, there are also more specialized forms of such technology that are designed to target more specific situations or needs.


Statistical natural language processing

Approaches:_Symbolic.2C_statistical.2C_neural_networks Symbolic approach, i.e., the hand-coding of a set of rules for manipulating symbols, coupled with a dictionary lookup, was historically the first approach used both by AI in general and by NLP in particular: such as by writing grammars or devising heuristic rules for stemming. Machine learning approaches, which include both statistical and neural networks, on the other hand, have many advantages over the symbolic approach: both statistical and neural networks methods can focus more on the most common cases extracted from a corpus of texts, whereas the rule-based approach needs to provide rules for both rare cases and common ones equally.language models, produced by either statistical or neural networks methods, are more robust to both unfamiliar (e.g. containing words or structures that have not been seen before) and erroneous input (e.g. with misspelled words or words accidentally omitted) in comparison to the rule-based systems, which are also more costly to produce.the larger such a (probabilistic) language model is, the more accurate it becomes, in contrast to rule-based systems that can gain accuracy only by increasing the amount and complexity of the rules leading to intractability problems.Although rule-based systems for manipulating symbols were still in use in 2020, they have become mostly obsolete with the advance of LLMs in 2023. Before that they were commonly used: when the amount of training data is insufficient to successfully apply machine learning methods, e.g., for the machine translation of low-resource languages such as provided by the Apertium system, for preprocessing in NLP pipelines, e.g., tokenization, or for postprocessing and transforming the output of NLP pipelines, e.g., for knowledge extraction from syntactic parses.In the late 1980s and mid-1990s, the statistical approach ended a period of AI winter, which was caused by the inefficiencies of the rule-based approaches. The earliest decision trees, producing systems of hard if���then rules, were still very similar to the old rule-based approaches. Only the introduction of hidden Markov models, applied to part-of-speech tagging, announced the end of the old rule-based approach. A major drawback of statistical methods is that they require elaborate feature engineering. Since 2015, the statistical approach was replaced by neural networks approach, using word embeddings to capture semantic properties of words. Intermediate tasks (e.g., part-of-speech tagging and dependency parsing) have not been needed anymore. Neural machine translation, based on then-newly-invented sequence-to-sequence transformations, made obsolete the intermediate steps, such as word alignment, previously necessary for statistical machine translation.


Statistical natural language processing

General_tendencies_and_.28possible.29_future_directions Based on long-standing trends in the field, it is possible to extrapolate future directions of NLP.  As of 2020, three trends among the topics of the long-standing series of CoNLL Shared Tasks can be observed: Interest on increasingly abstract, "cognitive" aspects of natural language (1999���2001: shallow parsing, 2002���03: named entity recognition, 2006���09/2017���18: dependency syntax, 2004���05/2008���09 semantic role labelling, 2011���12 coreference, 2015���16: discourse parsing, 2019: semantic parsing).  Increasing interest in multilinguality, and, potentially, multimodality (English since 1999; Spanish, Dutch since 2002; German since 2003; Bulgarian, Danish, Japanese, Portuguese, Slovenian, Swedish, Turkish since 2006; Basque, Catalan, Chinese, Greek, Hungarian, Italian, Turkish since 2007; Czech since 2009; Arabic since 2012; 2017: 40+ languages; 2018: 60+/100+ languages) Elimination of symbolic representations (rule-based over supervised towards weakly supervised methods, representation learning and end-to-end systems)Most higher-level NLP applications involve aspects that emulate intelligent behaviour and apparent comprehension of natural language.  More broadly speaking, the technical operationalization of increasingly advanced aspects of cognitive behaviour represents one of the developmental trajectories of NLP (see trends among CoNLL shared tasks above).  Cognition refers to "the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses. " Cognitive science is the interdisciplinary, scientific study of the mind and its processes.  Cognitive linguistics is an interdisciplinary branch of linguistics, combining knowledge and research from both psychology and linguistics.  Especially during the age of symbolic NLP, the area of computational linguistics maintained strong ties with cognitive studies.  As an example, George Lakoff offers a methodology to build natural language processing (NLP) algorithms through the perspective of cognitive science, along with the findings of cognitive linguistics, with two defining aspects: Apply the theory of conceptual metaphor, explained by Lakoff as "the understanding of one idea, in terms of another" which provides an idea of the intent of the author.  For example, consider the English word big.  When used in a comparison ("That is a big tree"), the author's intent is to imply that the tree is physically large relative to other trees or the authors experience.  When used metaphorically ("Tomorrow is a big day"), the author's intent to imply importance.  The intent behind other usages, like in "She is a big person", will remain somewhat ambiguous to a person and a cognitive NLP algorithm alike without additional information.  Assign relative measures of meaning to a word, phrase, sentence or piece of text based on the information presented before and after the piece of text being analyzed, e. g. , by means of a probabilistic context-free grammar (PCFG).  The mathematical equation for such algorithms is presented in US patent 9269353 :Ties with cognitive linguistics are part of the historical heritage of NLP, but they have been less frequently addressed since the statistical turn during the 1990s.  Nevertheless, approaches to develop cognitive models towards technically operationalizable frameworks have been pursued in the context of various frameworks, e. g. , of cognitive grammar, functional grammar, construction grammar, computational psycholinguistics and cognitive neuroscience (e. g.


Statistical natural language processing

General_tendencies_and_.28possible.29_future_directions , ACT-R), however, with limited uptake in mainstream NLP (as measured by presence on major conferences of the ACL).  More recently, ideas of cognitive NLP have been revived as an approach to achieve explainability, e. g. , under the notion of "cognitive AI".  Likewise, ideas of cognitive NLP are inherent to neural models multimodal NLP (although rarely made explicit) and developments in Artificial intelligence, specifically tools and technologies using Large language model approaches.


Statistical natural language processing

See_also 1 the Road Artificial intelligence detection software Automated essay scoring Biomedical text mining Compound term processing Computational linguistics Computer-assisted reviewing Controlled natural language Deep learning Deep linguistic processing Distributional semantics Foreign language reading aid Foreign language writing aid Information extraction Information retrieval Language and Communication Technologies Language model Language technology Latent semantic indexing Multi-agent system Native-language identification Natural-language programming Natural-language understanding Natural-language search Outline of natural language processing Query expansion Query understanding Reification (linguistics) Speech processing Spoken dialogue systems Text-proofing Text simplification Transformer (machine learning model) Truecasing Question answering Word2vec


Imagen (Google Brain)

Projects In October 2016, Google Brain designed an experiment to determine that neural networks are capable of learning secure symmetric encryption.  In this experiment, three neural networks were created: Alice, Bob and Eve.  Adhering to the idea of a generative adversarial network (GAN), the goal of the experiment was for Alice to send an encrypted message to Bob that Bob could decrypt, but the adversary, Eve, could not.  Alice and Bob maintained an advantage over Eve, in that they shared a key used for encryption and decryption.  In doing so, Google Brain demonstrated the capability of neural networks to learn secure encryption.  In February 2017, Google Brain determined a probabilistic method for converting pictures with 8x8 resolution to a resolution of 32x32.  The method built upon an already existing probabilistic model called pixelCNN to generate pixel translations.  The proposed software utilizes two neural networks to make approximations for the pixel makeup of translated images.  The first network, known as the "conditioning network," downsizes high-resolution images to 8x8 and attempts to create mappings from the original 8x8 image to these higher-resolution ones.  The other network, known as the "prior network," uses the mappings from the previous network to add more detail to the original image.  The resulting translated image is not the same image in higher resolution, but rather a 32x32 resolution estimation based on other existing high-resolution images.  Google Brain's results indicate the possibility for neural networks to enhance images.  The Google Brain team contributed to the Google Translate project by employing a new deep learning system that combines artificial neural networks with vast databases of multilingual texts.  In September 2016, Google Neural Machine Translation (GNMT) was launched, an end-to-end learning framework, able to learn from a large number of examples.  Previously, Google Translate's Phrase-Based Machine Translation (PBMT) approach would statistically analyze word by word and try to match corresponding words in other languages without considering the surrounding phrases in the sentence.  But rather than choosing a replacement for each individual word in the desired language, GNMT evaluates word segments in the context of the rest of the sentence to choose more accurate replacements.  Compared to older PBMT models, the GNMT model scored a 24% improvement in similarity to human translation, with a 60% reduction in errors.  The GNMT has also shown significant improvement for notoriously difficult translations, like Chinese to English.  While the introduction of the GNMT has increased the quality of Google Translate's translations for the pilot languages, it was very difficult to create such improvements for all of its 103 languages.  Addressing this problem, the Google Brain Team was able to develop a Multilingual GNMT system, which extended the previous one by enabling translations between multiple languages.  Furthermore, it allows for Zero-Shot Translations, which are translations between two languages that the system has never explicitly seen before.  Google announced that Google Translate can now also translate without transcribing, using neural networks.


Imagen (Google Brain)

Projects This means that it is possible to translate speech in one language directly into text in another language, without first transcribing it to text.  According to the Researchers at Google Brain, this intermediate step can be avoided using neural networks.  In order for the system to learn this, they exposed it to many hours of Spanish audio together with the corresponding English text.  The different layers of neural networks, replicating the human brain, were able to link the corresponding parts and subsequently manipulate the audio waveform until it was transformed to English text.  Another drawback of the GNMT model is that it causes the time of translation to increase exponentially with the number of words in the sentence.  This caused the Google Brain Team to add 2000 more processors to ensure the new translation process would still be fast and reliable.  Aiming to improve traditional robotics control algorithms where new skills of a robot need to be hand-programmed, robotics researchers at Google Brain are developing machine learning techniques to allow robots to learn new skills on their own.  They also attempt to develop ways for information sharing between robots so that robots can learn from each other during their learning process, also known as cloud robotics.  As a result, Google has launched the Google Cloud Robotics Platform for developers in 2019, an effort to combine robotics, AI, and the cloud to enable efficient robotic automation through cloud-connected collaborative robots.  Robotics research at Google Brain has focused mostly on improving and applying deep learning algorithms to enable robots to complete tasks by learning from experience, simulation, human demonstrations, and/or visual representations.  For example, Google Brain researchers showed that robots can learn to pick and throw rigid objects into selected boxes by experimenting in an environment without being pre-programmed to do so.  In another research, researchers trained robots to learn behaviors such as pouring liquid from a cup; robots learned from videos of human demonstrations recorded from multiple viewpoints.  Google Brain researchers have collaborated with other companies and academic institutions on robotics research.  In 2016, the Google Brain Team collaborated with researchers at X in a research on learning hand-eye coordination for robotic grasping.  Their method allowed real-time robot control for grasping novel objects with self-correction.  In 2020, researchers from Google Brain, Intel AI Lab, and UC Berkeley created an AI model for robots to learn surgery-related tasks such as suturing from training with surgery videos.  In 2020, Google Brain Team and University of Lille presented a model for automatic speaker recognition which they called Interactive Speaker Recognition.  The ISR module recognizes a speaker from a given list of speakers only by requesting a few user specific words.  The model can be altered to choose speech segments in the context of Text-To-Speech Training.  It can also prevent malicious voice generators from accessing the data.  TensorFlow is an open source software library powered by Google Brain that allows anyone to utilize machine learning by providing the tools to train one's own neural network.


Imagen (Google Brain)

Projects The tool has been used by farmers to reduce the amount of manual labor required to sort their yield, by training it with a data set of human-sorted images.  Magenta is a project that uses Google Brain to create new information in the form of art and music rather than classify and sort existing data.  TensorFlow was updated with a suite of tools for users to guide the neural network to create images and music.  However, the team from Valdosta State University found that the AI struggles to perfectly replicate human intention in artistry, similar to the issues faced in translation.  The image sorting capabilities of Google Brain have been used to help detect certain medical conditions by seeking out patterns that human doctors may not notice to provide an earlier diagnosis.  During screening for breast cancer, this method was found to have one quarter the false positive rate of human pathologists, who require more time to look over each photo and cannot spend their entire focus on this one task.  Due to the neural network's very specific training for a single task, it cannot identify other afflictions present in a photo that a human could easily spot.  Google Brain announced in 2022 that it created two different types of text-to-image models called Imagen and Parti that compete with OpenAI's DALL-E.  Later in 2022, the project was extended to text-to-video.  The Google Brain projects' technology is currently used in various other Google products such as the Android Operating System's speech recognition system, photo search for Google Photos, smart reply in Gmail, and video recommendations in YouTube.


Imagen (Google Brain)

Reception Google Brain has received coverage in Wired, NPR, and Big Think. These articles have contained interviews with key team members Ray Kurzweil and Andrew Ng, and focus on explanations of the project's goals and applications. In December 2020, AI ethicist Timnit Gebru left Google. While the exact nature of her quitting or being fired is disputed, the cause of the departure was her refusal to retract a paper entitled "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" This paper explored potential risks of the growth of AI such as Google Brain, including environmental impact, biases in training data, and the ability to deceive the public. The request to retract the paper was made by Megan Kacholia, vice president of Google Brain. As of April 2021, nearly 7000 current or former Google employees and industry supporters have signed an open letter accusing Google of "research censorship" and condemning Gebru's treatment at the company. In February 2021, Google fired one of the leaders of the company's AI ethics team, Margaret Mitchell. The company's statement alleged that Mitchell had broken company policy by using automated tools to find support for Gebru. In the same month, engineers outside the ethics team began to quit, citing the termination of Gebru as their reason for leaving. In April 2021, Google Brain co-founder Samy Bengio announced his resignation from the company. Despite being Gebru's manager, Bengio was not notified before her termination, and he posted online in support of both her and Mitchell. While Bengio's announcement focused on personal growth as his reason for leaving, anonymous sources indicated to Reuters that the turmoil within the AI ethics team played a role in his considerations. In March 2022, Google fired AI researcher Satrajit Chatterjee after he questioned the findings of a paper published in Nature, by Google's AI team members, Anna Goldie and Azalia Mirhoseini, about their findings on the ability of computers to design computer chip components.


Artificial intelligence art

History The concept of automated art dates back at least to the automata of ancient Greek civilization, where inventors such as Daedalus and Hero of Alexandria were described as having designed machines capable of writing text, generating sounds, and playing music. The tradition of creative automatons has flourished throughout history, such as Maillardet's automaton, created in the early 1800s. The academic discipline of artificial intelligence was founded at a research workshop at Dartmouth College in 1956, and has experienced several waves of advancement and optimism in the decades since. Since its founding, researchers in the field have raised philosophical and ethical arguments about the nature of the human mind and the consequences of creating artificial beings with human-like intelligence; these issues have previously been explored by myth, fiction and philosophy since antiquity. Since the founding of AI in the 1950s, artists and researchers have used artificial intelligence to create artistic works. By the early 1970s, Harold Cohen was creating and exhibiting AI works created by AARON, the computer program Cohen created to generate paintings, including a 1972 exhibition at the Los Angeles County Museum of Art. In both 1991 and 1992, Karl Sims won the Golden Nica award at Prix Ars Electronica for his 3D AI animated videos using artificial evolution. In 2009, Eric Millikin won the Pulitzer Prize along with several other awards for his artificial intelligence art that was critical of government corruption in Detroit and resulted in the city's mayor being sent to jail. In 2014, Ian Goodfellow and colleagues at Universit�� de Montr��al developed the Generative adversarial network, a type of Deep neural network capable of learning to mimic the statistical distribution of input data such as images. In 2015, a team at Google released DeepDream, a program that uses algorithmic pareidolia to create a dream-like appearance reminiscent of a psychedelic experience. In 2018, an auction sale of artificial intelligence art was held at Christie's Auction House in New York where the AI artwork Edmond de Belamy (a pun on Goodfellow's name) sold for $432,500, which was almost 45 times higher than its estimate of $7,000���$10,000. The artwork was created by "Obvious", a Paris-based collective. In 2019, Stephanie Dinkins won the Creative Capital award for her creation of an evolving artificial intelligence based on the "interests and culture(s) of people of color." Also in 2019, Sougwen Chung won the Lumen Prize for her performances with a robotic arm that uses AI to attempt to draw in a manner similar to Chung. In 2021, using the Transformer models used in GPT-2 and GPT-3, OpenAI developed DALL-E, a text-to-image AI model capable of producing high-quality images based on natural language prompts. In 2022, DALL-E was followed by Midjourney, then by the open source Stable Diffusion, leading to a dramatic growth in the use of AI to generate visual art. In 2022, Refik Anadol created an artificial intelligence art installation at the Museum of Modern Art in New York, based on the museum's own collection.


Artificial intelligence art

Tools_and_processes Many mechanisms for creating AI art have been developed, including procedural "rule-based" generation of images using mathematical patterns, algorithms which simulate brush strokes and other painted effects, and deep learning algorithms, such as generative adversarial networks (GANs) and transformers.  One of the first significant AI art systems is AARON, developed by Harold Cohen beginning in the late 1960s at the University of California at San Diego.  AARON is the most notable example of AI art in the era of GOFAI programming because of its use of a symbolic rule-based approach to generate technical images.  Cohen developed AARON with the goal of being able to code the act of drawing.  In its primitive form, AARON created simple black and white drawings.  Cohen would later finish the drawings by painting them.  Throughout the years, he also began to develop a way for AARON to also paint.  Cohen designed AARON to paint using special brushes and dyes that were chosen by the program itself without mediation from Cohen.  Generative adversarial networks (GANs) were designed in 2014.  This system uses a "generator" to create new images and a "discriminator" to decide which created images are considered successful.  DeepDream, released by Google in 2015, uses a convolutional neural network to find and enhance patterns in images via algorithmic pareidolia, thus creating deliberately over-processed images.  After DeepDream's release, several companies released apps that transform photos into art-like images with the style of well-known sets of paintings.  The website Artbreeder, launched in 2018, uses the models StyleGAN and BigGAN to allow users to generate and modify images such as faces, landscapes, and paintings.  Several programs use text-to-image models to generate a variety of images based on various text prompts.  They include EleutherAI's VQGAN+CLIP which was released in 2021, OpenAI's DALL-E which released a series of images in January 2021, Google Brain's Imagen and Parti which was announced in May 2022, Microsoft's NUWA-Infinity, and Stable Diffusion which was released in August 2022.  Stability. ai has a Stable Diffusion web interface called DreamStudio.  Stable Diffusion is source-available software, enabling further development such as plugins for Krita, Photoshop, Blender, and GIMP, as well as the Automatic1111 web-based open source user interface.  Stable Diffusion's main pre-trained model is shared on the Hugging Face Hub.  There are many other AI art generation programs including simple consumer-facing mobile apps and Jupyter notebooks that require powerful GPUs to run effectively.  The exhibition "Thinking Machines: Art and Design in the Computer Age, 1959���1989" at MoMA provided an overview of AI applications for art, architecture, and design.  Exhibitions showcasing the usage of AI to produce art include the 2016 Google-sponsored benefit and auction at the Gray Area Foundation in San Francisco, where artists experimented with the DeepDream algorithm and the 2017 exhibition "Unhuman: Art in the Age of AI", which took place in Los Angeles and Frankfurt.  In spring 2018, the Association for Computing Machinery dedicated a magazine issue to the subject of computers and art.


Artificial intelligence art

Tools_and_processes In June 2018, "Duet for Human and Machine", an art piece permitting viewers to interact with an artificial intelligence, premiered at the Beall Center for Art + Technology.  The Austrian Ars Electronica and Museum of Applied Arts, Vienna opened exhibitions on AI in 2019.  Ars Electronica's 2019 festival "Out of the box" explored art's role in a sustainable societal transformation.  Examples of such augmentation may include e. g.  enabling expansion of noncommercial niche genres (common examples are cyberpunk derivatives like solarpunk) by amateurs, novel entertainment, novel imaginative childhood play,[additional citation(s) needed] very fast prototyping, increasing art-making accessibility and artistic output per effort and/or expenses and/or time ��� e. g.  via generating drafts, inspirations, draft-refinitions, and image-components (Inpainting).  Prompts for some text-to-image models can also include images and keywords and configurable parameters, such as artistic style, which is often used via keyphrases like "in the style of [name of an artist]" in the prompt and/or selection of a broad aesthetic/art style.  There are platforms for sharing, trading, searching, forking/refining and/or collaborating on prompts for generating specific imagery from image generators.  Prompts are often shared along with images on image-sharing websites such as Reddit and AI art-dedicated websites.  A prompt is not the complete input needed for the generation of an image: additional inputs that determine the generated image include the output resolution, random seed, and random sampling parameters.  Synthetic media, which includes AI art, was described in 2022 as a major technology-driven trend that will affect business in the coming years.  'Synthography' is a proposed term for the practice of generating images that are similar to photographs using AI.  Additional functionalities are under development and may improve various applications or enable new ones ��� such as "Textual Inversion" which refers to enabling the use of user-provided concepts (like an object or a style) learned from few images.  With textual inversion, novel personalized art can be generated from the associated word(s) (the keywords that have been assigned to the learned, often abstract, concept) and model extensions/fine-tuning (see also: DreamBooth).  Generated images are sometimes used as sketches or low-cost experimentations or illustration of proof-of-concept-stage ideas ��� additional functionalities or improvements may also relate to post-generation manual editing (polishing or artistic usage) of prompts-based art[additional citation(s) needed] (such as subsequent tweaking with an image editor).