OpenAI Five

Comparisons_with_other_game_AI_systems Prior to OpenAI Five, other AI versus human experiments and systems have been successfully used before, such as Jeopardy! with Watson, chess with Deep Blue, and Go with AlphaGo. In comparison with other games that have used AI systems to play against human players, Dota 2 differs as explained below: Long run view: The bots run at 30 frames per second for an average match time of 45 minutes, which results in 80,000 ticks per game. OpenAI Five observes every fourth frame, generating 20,000 moves. By comparison, chess usually ends before 40 moves, while Go ends before 150 moves. Partially observed state of the game: Players and their allies can only see the map directly around them. The rest of it is covered in a fog of war which hides enemies units and their movements. Thus, playing Dota 2 requires making inferences based on this incomplete data, as well as predicting what their opponent could be doing at the same time. By comparison, Chess and Go are "full-information games", as they do not hide elements from the opposing player. Continuous action space: Each playable character in a Dota 2 game, known as a hero, can take dozens of actions that target either another unit or a position. The OpenAI Five developers allow the space into 170,000 possible actions per hero. Without counting the perpetual aspects of the game, there are an average of ~1,000 valid actions each tick. By comparison, the average number of actions in chess is 35 and 250 in Go. Continuous observation space: Dota 2 is played on a large map with ten heroes, five on each team, along with dozens of buildings and non-player character (NPC) units. The OpenAI system observes the state of a game through developers��� bot API, as 20,000 numbers that constitute all information a human is allowed to get access to. A chess board is represented as about 70 lists, whereas a Go board has about 400 enumerations.


OpenAI Five

Reception OpenAI Five have received acknowledgement from the AI, tech, and video game community at large. Microsoft founder Bill Gates called it a "big deal", as their victories "required teamwork and collaboration". Chess player Garry Kasparov, who lost against the Deep Blue AI in 1997, stated that despite their losing performance at The International 2018, the bots would eventually "get there, and sooner than expected". In a conversation with MIT Technology Review, AI experts also considered OpenAI Five system as a significant achievement, as they noted that Dota 2 was an "extremely complicated game", so even beating non-professional players was impressive. PC Gamer wrote that their wins against professional players was a significant event in machine learning. In contrast, Motherboard wrote that the victory was "basically cheating" due to the simplified hero pools on both sides, as well as the fact that bots were given direct access to the API, as opposed to using computer vision to interpret pixels on the screen. The Verge wrote that the bots were evidence that the company's approach to reinforcement learning and its general philosophy about AI was "yielding milestones". In 2019, DeepMind unveiled a similar bot for Starcraft II, AlphaStar. Like OpenAI Five, AlphaStar used reinforcement learning and self-play. The Verge reported that "the goal with this type of AI research is not just to crush humans in various games just to prove it can be done. Instead, it���s to prove that ��� with enough time, effort, and resources ��� sophisticated AI software can best humans at virtually any competitive cognitive challenge, be it a board game or a modern video game." They added that the DeepMind and OpenAI victories were also a testament to the power of certain uses of reinforcement learning. It was OpenAI's hope that the technology could have applications outside of the digital realm. In 2018, they were able to reuse the same reinforcement learning algorithms and training code from OpenAI Five for Dactyl, a human-like robot hand with a neural network built to manipulate physical objects. In 2019, Dactyl solved the Rubik's Cube.


DreamBooth

DreamBooth DreamBooth is a deep learning generation model used to personalize existing text-to-image models by fine-tuning. It was developed by researchers from Google Research and Boston University in 2022. Originally developed using Google's own Imagen text-to-image model, DreamBooth implementations can be applied to other text-to-image models, where it can allow the model to generate more fine-tuned and personalized outputs after training on three to five images of a subject.


DreamBooth

Technology Pretrained text-to-image diffusion models, while often capable of offering a diverse range of different image output types, lack the specificity required to generate images of lesser-known subjects, and are limited in their ability to render known subjects in different situations and contexts. The methodology used to run implementations of DreamBooth involves the fine-tuning the full UNet component of the diffusion model using a few images (usually 3--5) depicting a specific subject. Images are paired with text prompts that contain the name of the class the subject belongs to, plus a unique identifier. As an example, a photograph of a [Nissan R34 GTR] car, with car being the class); a class-specific prior preservation loss is applied to encourage the model to generate diverse instances of the subject based on what the model is already trained on for the original class. Pairs of low-resolution and high-resolution images taken from the set of input images are used to fine-tune the super-resolution components, allowing the minute details of the subject to be maintained.


DreamBooth

Usage DreamBooth can be used to fine-tune models such as Stable Diffusion, where it may alleviate a common shortcoming of Stable Diffusion not being able to adequately generate images of specific individual people. Such a use case is quite VRAM intensive, however, and thus cost-prohibitive for hobbyist users. The Stable Diffusion adaptation of DreamBooth in particular is released as a free and open-source project based on the technology outlined by the original paper published by Ruiz et. al. in 2022. Concerns have been raised regarding the ability for bad actors to utilise DreamBooth to generate misleading images for malicious purposes, and that its open-source nature allows anyone to utilise or even make improvements to the technology. In addition, artists have expressed their apprehension regarding the ethics of using DreamBooth to train model checkpoints that are specifically aimed at imitating specific art styles associated with human artists; one such critic is Hollie Mengert, an illustrator for Disney and Penguin Random House who has had her art style trained into a checkpoint model via DreamBooth and shared online, without her consent.


Kaggle

See_also Data science competition platform Anthony Goldbloom Hugging Face


StyleGAN

History A direct predecessor of the StyleGAN series is the Progressive GAN, published in 2017. In December 2018, Nvidia researchers distributed a preprint with accompanying software introducing StyleGAN, a GAN for producing an unlimited number of (often convincing) portraits of fake human faces. StyleGAN was able to run on Nvidia's commodity GPU processors. In February 2019, Uber engineer Phillip Wang used the software to create This Person Does Not Exist, which displayed a new face on each web page reload. Wang himself has expressed amazement, given that humans are evolved to specifically understand human faces, that nevertheless StyleGAN can competitively "pick apart all the relevant features (of human faces) and recompose them in a way that's coherent." In September 2019, a website called Generated Photos published 100,000 images as a collection of stock photos. The collection was made using a private dataset shot in a controlled environment with similar light and angles. Similarly, two faculty at the University of Washington's Information School used StyleGAN to create Which Face is Real?, which challenged visitors to differentiate between a fake and a real face side by side. The faculty stated the intention was to "educate the public" about the existence of this technology so they could be wary of it, "just like eventually most people were made aware that you can Photoshop an image". The second version of StyleGAN, called StyleGAN2, was published on February 5, 2020. It removes some of the characteristic artifacts and improves the image quality. In 2021, a third version was released, improving consistency between fine and coarse details in the generator. Dubbed "alias-free", this version was implemented with pytorch. In December 2019, Facebook took down a network of accounts with false identities, and mentioned that some of them had used profile pictures created with artificial intelligence.


Interlingual machine translation

History The first ideas about interlingual machine translation appeared in the 17th century with Descartes and Leibniz, who came up with theories of how to create dictionaries using universal numerical codes, not unlike numerical tokens used by large language models nowadays. Others, such as Cave Beck, Athanasius Kircher and Johann Joachim Becher worked on developing an unambiguous universal language based on the principles of logic and iconographs. In 1668, John Wilkins described his interlingua in his "Essay towards a Real Character and a Philosophical Language". In the 18th and 19th centuries many proposals for "universal" international languages were developed, the most well known being Esperanto. That said, applying the idea of a universal language to machine translation did not appear in any of the first significant approaches. Instead, work started on pairs of languages. However, during the 1950s and 60s, researchers in Cambridge headed by Margaret Masterman, in Leningrad headed by Nikolai Andreev and in Milan by Silvio Ceccato started work in this area. The idea was discussed extensively by the Israeli philosopher Yehoshua Bar-Hillel in 1969. During the 1970s, noteworthy research was done in Grenoble by researchers attempting to translate physics and mathematical texts from Russian to French, and in Texas a similar project (METAL) was ongoing for Russian to English. Early interlingual MT systems were also built at Stanford in the 1970s by Roger Schank and Yorick Wilks; the former became the basis of a commercial system for the transfer of funds, and the latter's code is preserved at The Computer Museum at Boston as the first interlingual machine translation system. In the 1980s, renewed relevance was given to interlingua-based, and knowledge-based approaches to machine translation in general, with much research going on in the field. The uniting factor in this research was that high-quality translation required abandoning the idea of requiring total comprehension of the text. Instead, the translation should be based on linguistic knowledge and the specific domain in which the system would be used. The most important research of this era was done in distributed language translation (DLT) in Utrecht, which worked with a modified version of Esperanto, and the Fujitsu system in Japan.


WolframAlpha

Copyright_claims InfoWorld published an article warning readers of the potential implications of giving an automated website proprietary rights to the data it generates. Free software advocate Richard Stallman also opposes recognizing the site as a copyright holder and suspects that Wolfram Research would not be able to make this case under existing copyright law.


List of artificial intelligence artists

21st_century Sougwen Chung, active from 2010s to present. Chung's work includes performances with a robotic arm that uses AI to attempt to draw in a manner similar to Chung. Stephanie Dinkins, active from 2010s to present. Dinkins' work includes recordings of conversations with an artificially intelligent robot that resembles a black woman, discussing topics such as race and the nature of being. Jake Elwes, active from 2010s to present. Their practice is the exploration of artificial intelligence, queer theory and technical biases. Libby Heaney, active from 2010s to present. Heaney's practice includes work with chatbots. Mario Klingemann, active from 2010s to present. Klingemann's works examine creativity, culture, and perception through machine learning and artificial intelligence. Mauro Martino, active from 2010s to present. Martino's work work includes design, data visualization and infographics. Trevor Paglen, active from 2000s to present. Paglen's practice includes work in photography and geography, on topics like mass surveillance and data collection. Anna Ridler, active from 2010s to present. Ridler works with collections of information, including self-generated data sets, often working with floral photography.


Commonsense reasoning

Approaches_and_techniques Commonsense's reasoning study is divided into knowledge-based approaches and approaches that are based on machine learning over and using a large data corpora with limited interactions between these two types of approaches[citation needed]. There are also crowdsourcing approaches, attempting to construct a knowledge basis by linking the collective knowledge and the input of non-expert people. Knowledge-based approaches can be separated into approaches based on mathematical logic[citation needed]. In knowledge-based approaches, the experts are analyzing the characteristics of the inferences that are required to do reasoning in a specific area or for a certain task. The knowledge-based approaches consist of mathematically grounded approaches, informal knowledge-based approaches and large-scale approaches. The mathematically grounded approaches are purely theoretical and the result is a printed paper instead of a program. The work is limited to the range of the domains and the reasoning techniques that are being reflected on. In informal knowledge-based approaches, theories of reasoning are based on anecdotal data and intuition that are results from empirical behavioral psychology. Informal approaches are common in computer programming. Two other popular techniques for extracting commonsense knowledge from Web documents involve Web mining and Crowd sourcing. COMET (2019), which uses both the OpenAI GPT language model architecture and existing commonsense knowledge bases such as ConceptNet, claims to generate commonsense inferences at a level approaching human benchmarks. Like many other current efforts, COMET over-relies on surface language patterns and is judged to lack deep human-level understanding of many commonsense concepts. Other language-model approaches include training on visual scenes rather than just text, and training on textual descriptions of scenarios involving commonsense physics.


Pattern language

Pattern language A pattern language is an organized and coherent set of patterns, each of which describes a problem and the core of a solution that can be used in many ways within a specific field of expertise. The term was coined by architect Christopher Alexander and popularized by his 1977 book A Pattern Language. A pattern language can also be an attempt to express the deeper wisdom of what brings aliveness within a particular field of human endeavor, through a set of interconnected patterns. Aliveness is one placeholder term for "the quality that has no name": a sense of wholeness, spirit, or grace, that while of varying form, is precise and empirically verifiable. Alexander claims that ordinary people can use this design approach to successfully solve very large, complex design problems.


Sam Altman

Sam Altman Samuel Harris Altman (/������ltm��n/ AWLT-m��n; born 1985) is an American entrepreneur, investor, and programmer. He is the CEO of OpenAI and was the co-founder of Loopt. He previously was the president of Y Combinator and was briefly the CEO of Reddit.


Sam Altman

Career In 2005, at the age of 19, Altman co-founded Loopt, a location-based social networking mobile application. As CEO, Altman raised more than $30 million in venture capital for the company; however, Loopt failed to gain traction with enough users. In March 2012, it was acquired by the Green Dot Corporation for $43.4 million. The following month, Altman co-founded Hydrazine Capital with his brother, Jack Altman. Altman had got scurvy from his work on Loopt. Altman became a partner at Y Combinator in 2011, initially working there on a part-time basis. In February 2014, Altman was named president of Y Combinator by its co-founder, Paul Graham. In a 2014 blog post, Altman said that the total valuation of Y Combinator companies had surpassed $65 billion, including Airbnb, Dropbox, Zenefits, and Stripe. In September 2016, Altman announced his expanded role as president of YC Group, which included Y Combinator and other units. Altman said that he hoped to expand Y Combinator to fund 1,000 new companies per year. He also tried to expand the types of companies funded by YC, especially "hard technology" companies. In October 2015, Altman announced YC Continuity, a $700 million equity fund investing in YC companies as they matured. A week earlier, Altman had introduced Y Combinator Research, a non-profit research lab, and donated $10 million to fund it. YC Research has thus far announced research on basic income, the future of computing, education, and building new cities. In March 2019, YC announced Altman's transition from the president of the company to a less hands-on role as Chairman of the Board, for him to focus on OpenAI. This decision came shortly after YC announced it would be moving its headquarters to San Francisco. As of early 2020, he is no longer affiliated with YC. By 2020, Sam Altman had left Y Combinator to focus full-time on OpenAI as CEO. OpenAI was initially funded by Altman, Greg Brockman, Elon Musk, Jessica Livingston, Peter Thiel, Microsoft, Amazon Web Services, Infosys, and YC Research. When OpenAI launched in 2015, it had raised $1 billion. Altman testified before the United States Senate Judiciary Subcommittee on Privacy, Technology, and the Law on 16 May 2023 about issues of AI oversight. Altman was the CEO of Reddit for eight days in 2014 after CEO Yishan Wong resigned. He announced the return of Steve Huffman as CEO on July 10, 2015. Altman co-founded Tools For Humanity in 2019, a company building a global iris-based biometric system using cryptocurrency, called Worldcoin. Worldcoin aims to provide a reliable way to authenticate humans online, to counter bots and fake virtual identities facilitated by artificial intelligence. Using a distribution mechanism for its cryptocurrency similar to UBI, Worldcoin attempts to incentivize users to join its network by getting their iris scanned using Worldcoin's orb-shaped iris scanner.


Sam Altman

Politics Recode reported that Altman might run for Governor of California in the 2018 election, which he did not enter. In 2018, Altman announced "The United Slate", a political project to improve housing and healthcare policy. In 2019, Altman held a fundraiser at his house in San Francisco for 2020 Democratic presidential candidate Andrew Yang. In May 2020, Altman donated $250,000 to American Bridge 21st Century, a super-PAC supporting Democratic presidential candidate Joe Biden. After the success of ChatGPT, Altman made a world tour in May 2023 where he visited 22 countries and met multiple leaders and diplomats: He met with British prime minister Rishi Sunak, French president Emmanuel Macron, Spanish prime minister Pedro S��nchez, German chancellor Olaf Scholz, Indian prime minister Narendra Modi, South Korean president Yoon Suk-yeol, and Israeli president Isaac Herzog. He stood for a photo with European Commission president Ursula von der Leyen.


Greg Brockman

Greg Brockman Greg Brockman (born 1988 or 1989) is an American entrepreneur, investor and software developer who is a co-founder and currently the president of OpenAI. He began his career at Stripe in 2010, upon leaving MIT, and became their CTO in 2013. He left Stripe in 2015 to co-found OpenAI, where he also assumed the role of CTO.


Greg Brockman

Career In 2010, he dropped out of MIT to join Stripe, Inc., a company founded by Patrick Collison, an MIT classmate, and his brother, John Collison. In 2013, he became Stripe's first-ever CTO, and grew the company from 5 to 205 employees. Brockman left Stripe in May 2015, and co-founded OpenAI in December 2015 with Elon Musk and Sam Altman. Brockman helped create the OpenAI founding team, and led various prominent projects early on at OpenAI, including OpenAI Gym and OpenAI Five, a Dota 2 bot. On February 14, 2019, OpenAI announced that they had developed a new large language model called GPT-2, but kept it private due to their concern for its potential misuse. They finally released the model to a limited group of beta testers in May 2019. On March 14, 2023, in a live video demo, Brockman unveiled GPT-4, the fourth iteration in the GPT series, and the newest language model created by OpenAI.


Tesla Dojo

History Tesla operates several massively parallel computing clusters for developing its Autopilot advanced driver assistance system.  Its primary unnamed cluster using 5,760 Nvidia A100 graphics processing units (GPUs) was touted by Andrej Karpathy in 2021 at the fourth International Joint Conference on Computer Vision and Pattern Recognition (CCVPR 2021) to be "roughly the number five supercomputer in the world" at approximately 81.6 petaflops, based on scaling the performance of the Nvidia Selene supercomputer, which uses similar components.  However, the performance of the primary Tesla GPU cluster has been disputed, as it was not clear if this was measured using single-precision or double-precision floating point numbers (FP32 or FP64).  Tesla also operates a second 4,032 GPU cluster for training and a third 1,752 GPU cluster for automatic labeling of objects.  The primary unnamed Tesla GPU cluster has been used for processing one million video clips, each ten seconds long, taken from Tesla Autopilot cameras operating in Tesla cars in the real world, running at 36 frames per second.  Collectively, these video clips contained six billion object labels, with depth and velocity data; the total size of the data set was 1.5 petabytes.  This data set was used for training a neural network intended to help Autopilot computers in Tesla cars understand roads.  By August 2022, Tesla had upgraded the primary GPU cluster to 7,360 GPUs.  Dojo was first mentioned by Musk in April 2019 during Tesla's "Autonomy Investor Day".  In August 2020, Musk stated it was "about a year away" due to power and thermal issues.  The defining goal of [Dojo] is scalability.  We have de-emphasized several mechanisms that you find in typical CPUs, like coherency, virtual memory, and global lookup directories just because these mechanisms do not scale very well. . .  Instead, we have relied on a very fast and very distributed SRAM [static random-access memory] storage throughout the mesh.  And this is backed by an order of magnitude higher speed of interconnect than what you find in a typical distributed system.  ��� Emil Talpes, Tesla hardware engineer, 2022 The Next Platform article Dojo was officially announced at Tesla's Artificial Intelligence (AI) Day on August 19, 2021.  Tesla revealed details of the D1 chip and its plans for "Project Dojo", a datacenter that would house 3,000 D1 chips; the first "Training Tile" had been completed and delivered the week before.  In October 2021, Tesla released a "Dojo Technology" whitepaper describing the Configurable Float8 (CFloat8) and Configurable Float16 (CFloat16) floating point formats and arithmetic operations as an extension of Institute of Electrical and Electronics Engineers (IEEE) standard 754.  At the follow-up AI Day in September 2022, Tesla announced it had built several System Trays and one Cabinet.  During a test, the company stated that Project Dojo drew 2.3 megawatts (MW) of power before tripping a local San Jose, California power substation.  At the time, Tesla was assembling one Training Tile per day.


Tesla Dojo

History In August 2023, Tesla powered on Dojo for production use as well as a new training cluster configured with 10,000 Nvidia H100 GPUs. [unreliable source?


Image stitching

Projective_layouts For image segments that have been taken from the same point in space, stitched images can be arranged using one of various map projections. Rectilinear projection, where the stitched image is viewed on a two-dimensional plane intersecting the panosphere in a single point. Lines that are straight in reality are shown as straight regardless of their directions on the image. Wide views - around 120�� or so - start to exhibit severe distortion near the image borders. One case of rectilinear projection is the use of cube faces with cubic mapping for panorama viewing. Panorama is mapped to six squares, each cube face showing 90 by 90 degree area of the panorama. Cylindrical projection, where the stitched image shows a 360�� horizontal field of view and a limited vertical field of view. Panoramas in this projection are meant to be viewed as though the image is wrapped into a cylinder and viewed from within. When viewed on a 2D plane, horizontal lines appear curved while vertical lines remain straight. Vertical distortion increases rapidly when nearing the top of the panosphere. There are various other cylindrical formats, such as Mercator and Miller cylindrical which have less distortion near the poles of the panosphere. Spherical projection or equirectangular projection ��� which is strictly speaking another cylindrical projection ��� where the stitched image shows a 360�� horizontal by 180�� vertical field of view i.e. the whole sphere. Panoramas in this projection are meant to be viewed as though the image is wrapped into a sphere and viewed from within. When viewed on a 2D plane, horizontal lines appear curved as in a cylindrical projection, while vertical lines remain vertical. Since a panorama is basically a map of a sphere, various other mapping projections from cartographers can also be used if so desired. Additionally there are specialized projections which may have more aesthetically pleasing advantages over normal cartography projections such as Hugin's Panini projection - named after Italian vedutismo painter Giovanni Paolo Panini - or PTgui's Vedutismo projection. Different projections may be combined in same image for fine tuning the final look of the output image. Stereographic projection or fisheye projection can be used to form a little planet panorama by pointing the virtual camera straight down and setting the field of view large enough to show the whole ground and some of the areas above it; pointing the virtual camera upwards creates a tunnel effect. Conformality of the stereographic projection may produce more visually pleasing result than equal area fisheye projection as discussed in the stereo-graphic projection's article.


Discrete Laplace operator

Discrete_heat_equation Suppose �� {\textstyle \phi } describes a temperature distribution across a graph, where �� i {\textstyle \phi _{i}} is the temperature at vertex i {\textstyle i} .  According to Newton's law of cooling, the heat transferred from node i {\textstyle i} to node j {\textstyle j} is proportional to �� i ��� �� j {\textstyle \phi _{i}-\phi _{j}} if nodes i {\textstyle i} and j {\textstyle j} are connected (if they are not connected, no heat is transferred).  Then, for thermal conductivity k {\textstyle k} , In matrix-vector notation, which gives Notice that this equation takes the same form as the heat equation, where the matrix ���L is replacing the Laplacian operator ��� 2 {\textstyle \nabla ^{2}} ; hence, the "graph Laplacian".  To find a solution to this differential equation, apply standard techniques for solving a first-order matrix differential equation.  That is, write �� {\textstyle \phi } as a linear combination of eigenvectors v i {\textstyle \mathbf {v} _{i}} of L (so that L v i = �� i v i {\textstyle L\mathbf {v} _{i}=\lambda _{i}\mathbf {v} _{i}} ) with time-dependent coefficients, �� ( t ) = ��� i c i ( t ) v i .  {\textstyle \phi (t)=\sum _{i}c_{i}(t)\mathbf {v} _{i}. } Plugging into the original expression (because L is a symmetric matrix, its unit-norm eigenvectors v i {\textstyle \mathbf {v} _{i}} are orthogonal): whose solution is As shown before, the eigenvalues �� i {\textstyle \lambda _{i}} of L are non-negative, showing that the solution to the diffusion equation approaches an equilibrium, because it only exponentially decays or remains constant.  This also shows that given �� i {\textstyle \lambda _{i}} and the initial condition c i ( 0 ) {\textstyle c_{i}(0)} , the solution at any time t can be found.  To find c i ( 0 ) {\textstyle c_{i}(0)} for each i {\textstyle i} in terms of the overall initial condition �� ( 0 ) {\textstyle \phi (0)} , simply project �� ( 0 ) {\textstyle \phi (0)} onto the unit-norm eigenvectors v i {\textstyle \mathbf {v} _{i}} ; This approach has been applied to quantitative heat transfer modelling on unstructured grids.  In the case of undirected graphs, this works because L {\textstyle L} is symmetric, and by the spectral theorem, its eigenvectors are all orthogonal.  So the projection onto the eigenvectors of L {\textstyle L} is simply an orthogonal coordinate transformation of the initial condition to a set of coordinates which decay exponentially and independently of each other.  To understand lim t ��� ��� �� ( t ) {\textstyle \lim _{t\to \infty }\phi (t)} , the only terms c i ( t ) = c i ( 0 ) e ��� k �� i t {\textstyle c_{i}(t)=c_{i}(0)e^{-k\lambda _{i}t}} that remain are those where �� i = 0 {\textstyle \lambda _{i}=0} , since In other words, the equilibrium state of the system is determined completely by the kernel of L {\textstyle L} .


Discrete Laplace operator

Discrete_heat_equation Since by definition, ��� j L i j = 0 {\textstyle \sum _{j}L_{ij}=0} , the vector v 1 {\textstyle \mathbf {v} ^{1}} of all ones is in the kernel.  If there are k {\textstyle k} disjoint connected components in the graph, then this vector of all ones can be split into the sum of k {\textstyle k} independent �� = 0 {\textstyle \lambda =0} eigenvectors of ones and zeros, where each connected component corresponds to an eigenvector with ones at the elements in the connected component and zeros elsewhere.  The consequence of this is that for a given initial condition c ( 0 ) {\textstyle c(0)} for a graph with N {\textstyle N} vertices where For each element �� j {\textstyle \phi _{j}} of �� {\textstyle \phi } , i. e.  for each vertex j {\textstyle j} in the graph, it can be rewritten as In other words, at steady state, the value of �� {\textstyle \phi } converges to the same value at each of the vertices of the graph, which is the average of the initial values at all of the vertices.  Since this is the solution to the heat diffusion equation, this makes perfect sense intuitively.  We expect that neighboring elements in the graph will exchange energy until that energy is spread out evenly throughout all of the elements that are connected to each other.  This section shows an example of a function �� {\textstyle \phi } diffusing over time through a graph.  The graph in this example is constructed on a 2D discrete grid, with points on the grid connected to their eight neighbors.  Three initial points are specified to have a positive value, while the rest of the values in the grid are zero.  Over time, the exponential decay acts to distribute the values at these points evenly throughout the entire grid.  The complete Matlab source code that was used to generate this animation is provided below.  It shows the process of specifying initial conditions, projecting these initial conditions onto the eigenvalues of the Laplacian Matrix, and simulating the exponential decay of these projected initial conditions.


Gaussian blur

Implementation A Gaussian blur effect is typically generated by convolving an image with an FIR kernel of Gaussian values. In practice, it is best to take advantage of the Gaussian blur���s separable property by dividing the process into two passes. In the first pass, a one-dimensional kernel is used to blur the image in only the horizontal or vertical direction. In the second pass, the same one-dimensional kernel is used to blur in the remaining direction. The resulting effect is the same as convolving with a two-dimensional kernel in a single pass, but requires fewer calculations. Discretization is typically achieved by sampling the Gaussian filter kernel at discrete points, normally at positions corresponding to the midpoints of each pixel. This reduces the computational cost but, for very small filter kernels, point sampling the Gaussian function with very few samples leads to a large error. In these cases, accuracy is maintained (at a slight computational cost) by integration of the Gaussian function over each pixel's area. When converting the Gaussian���s continuous values into the discrete values needed for a kernel, the sum of the values will be different from 1. This will cause a darkening or brightening of the image. To remedy this, the values can be normalized by dividing each term in the kernel by the sum of all terms in the kernel. A much better and theoretically more well-founded approach is to instead perform the smoothing with the discrete analogue of the Gaussian kernel, which possesses similar properties over a discrete domain as makes the continuous Gaussian kernel special over a continuous domain, for example, the kernel corresponding to the solution of a diffusion equation describing a spatial smoothing process, obeying a semi-group property over additions of the variance of the kernel, or describing the effect of Brownian motion over a spatial domain, and with the sum of its values being exactly equal to 1. For a more detailed description about the discrete analogue of the Gaussian kernel, see the article on scale-space implementation and. The efficiency of FIR breaks down for high sigmas. Alternatives to the FIR filter exist. These include the very fast multiple box blurs, the fast and accurate IIR Deriche edge detector, a "stack blur" based on the box blur, and more.


Noise reduction

In_images Images taken with digital cameras or conventional film cameras will pick up noise from a variety of sources.  Further use of these images will often require that the noise be reduced either for aesthetic purposes, or for practical purposes such as computer vision.  In salt and pepper noise (sparse light and dark disturbances), also known as impulse noise, pixels in the image are very different in color or intensity from their surrounding pixels; the defining characteristic is that the value of a noisy pixel bears no relation to the color of surrounding pixels.  When viewed, the image contains dark and white dots, hence the term salt and pepper noise.  Generally, this type of noise will only affect a small number of image pixels.  Typical sources include flecks of dust inside the camera and overheated or faulty CCD elements.  In Gaussian noise, each pixel in the image will be changed from its original value by a (usually) small amount.  A histogram, a plot of the amount of distortion of a pixel value against the frequency with which it occurs, shows a normal distribution of noise.  While other distributions are possible, the Gaussian (normal) distribution is usually a good model, due to the central limit theorem that says that the sum of different noises tends to approach a Gaussian distribution.  In either case, the noise at different pixels can be either correlated or uncorrelated; in many cases, noise values at different pixels are modeled as being independent and identically distributed, and hence uncorrelated.  There are many noise reduction algorithms in image processing.  In selecting a noise reduction algorithm, one must weigh several factors: the available computer power and time available: a digital camera must apply noise reduction in a fraction of a second using a tiny onboard CPU, while a desktop computer has much more power and time whether sacrificing some real detail is acceptable if it allows more noise to be removed (how aggressively to decide whether variations in the image are noise or not) the characteristics of the noise and the detail in the image, to better make those decisionsIn real-world photographs, the highest spatial-frequency detail consists mostly of variations in brightness (luminance detail) rather than variations in hue (chroma detail).  Since any noise reduction algorithm should attempt to remove noise without sacrificing real detail from the scene photographed, one risks a greater loss of detail from luminance noise reduction than chroma noise reduction simply because most scenes have little high-frequency chroma detail to begin with.  In addition, most people find chroma noise in images more objectionable than luminance noise; the colored blobs are considered digital-looking and unnatural, compared to the grainy appearance of luminance noise that some compare to film grain.  For these two reasons, most photographic noise reduction algorithms split the image detail into chroma and luminance components and apply more noise reduction to the former.  Most dedicated noise-reduction computer software allows the user to control chroma and luminance noise reduction separately.


Noise reduction

In_images One method to remove noise is by convolving the original image with a mask that represents a low-pass filter or smoothing operation.  For example, the Gaussian mask comprises elements determined by a Gaussian function.  This convolution brings the value of each pixel into closer harmony with the values of its neighbors.  In general, a smoothing filter sets each pixel to the average value, or a weighted average, of itself and its nearby neighbors; the Gaussian filter is just one possible set of weights.  Smoothing filters tend to blur an image because pixel intensity values that are significantly higher or lower than the surrounding neighborhood would smear across the area.  Because of this blurring, linear filters are seldom used in practice for noise reduction; they are, however, often used as the basis for nonlinear noise reduction filters.  Another method for removing noise is to evolve the image under a smoothing partial differential equation similar to the heat equation, which is called anisotropic diffusion.  With a spatially constant diffusion coefficient, this is equivalent to the heat equation or linear Gaussian filtering, but with a diffusion coefficient designed to detect edges, the noise can be removed without blurring the edges of the image.  Another approach for removing noise is based on non-local averaging of all the pixels in an image.  In particular, the amount of weighting for a pixel is based on the degree of similarity between a small patch centered on that pixel and the small patch centered on the pixel being de-noised.  A median filter is an example of a non-linear filter and, if properly designed, is very good at preserving image detail.  To run a median filter: consider each pixel in the image sort the neighbouring pixels into order based upon their intensities replace the original value of the pixel with the median value from the listA median filter is a rank-selection (RS) filter, a particularly harsh member of the family of rank-conditioned rank-selection (RCRS) filters; a much milder member of that family, for example one that selects the closest of the neighboring values when a pixel's value is external in its neighborhood, and leaves it unchanged otherwise, is sometimes preferred, especially in photographic applications.  Median and other RCRS filters are good at removing salt and pepper noise from an image, and also cause relatively little blurring of edges, and hence are often used in computer vision applications.  The main aim of an image denoising algorithm is to achieve both noise reduction and feature preservation using the wavelet filter banks.  In this context, wavelet-based methods are of particular interest.  In the wavelet domain, the noise is uniformly spread throughout coefficients while most of the image information is concentrated in a few large ones.  Therefore, the first wavelet-based denoising methods were based on thresholding of detail subbands coefficients. [page needed] However, most of the wavelet thresholding methods suffer from the drawback that the chosen threshold may not match the specific distribution of signal and noise components at different scales and orientations.


Noise reduction

In_images To address these disadvantages, non-linear estimators based on Bayesian theory have been developed.  In the Bayesian framework, it has been recognized that a successful denoising algorithm can achieve both noise reduction and feature preservation if it employs an accurate statistical description of the signal and noise components.  Statistical methods for image denoising exist as well, though they are infrequently used as they are computationally demanding.  For Gaussian noise, one can model the pixels in a greyscale image as auto-normally distributed, where each pixel's true greyscale value is normally distributed with mean equal to the average greyscale value of its neighboring pixels and a given variance.  Let �� i {\displaystyle \delta _{i}} denote the pixels adjacent to the i {\displaystyle i} th pixel.  Then the conditional distribution of the greyscale intensity (on a [ 0 , 1 ] {\displaystyle [0,1]} scale) at the i {\displaystyle i} th node is: P ( x ( i ) = c | x ( j ) ��� j ��� �� i ) ��� e ��� �� 2 �� ��� j ��� �� i ( c ��� x ( j ) ) 2 {\displaystyle \mathbb {P} (x(i)=c|x(j)\forall j\in \delta i)\propto e^{-{\frac {\beta }{2\lambda }}\sum _{j\in \delta i}(c-x(j))^{2}}} for a chosen parameter �� ��� 0 {\displaystyle \beta \geq 0} and variance �� {\displaystyle \lambda } .  One method of denoising that uses the auto-normal model uses the image data as a Bayesian prior and the auto-normal density as a likelihood function, with the resulting posterior distribution offering a mean or mode as a denoised image.  A block-matching algorithm can be applied to group similar image fragments into overlapping macroblocks of identical size, stacks of similar macroblocks are then filtered together in the transform domain and each image fragment is finally restored to its original location using a weighted average of the overlapping pixels.  Shrinkage fields is a random field-based machine learning technique that brings performance comparable to that of Block-matching and 3D filtering yet requires much lower computational overhead (such that it could be performed directly within embedded systems).  Various deep learning approaches have been proposed to solve noise reduction and such image restoration tasks.  Deep Image Prior is one such technique that makes use of convolutional neural network and is distinct in that it requires no prior training data.  Most general-purpose image and photo editing software will have one or more noise-reduction functions (median, blur, despeckle, etc. ).


Color science

Journals The preeminent scholarly journal publishing research papers in color science is Color Research and Application, started in 1975 by founding editor-in-chief Fred Billmeyer, along with Gunter Wyszecki, Michael Pointer and Rolf Kuehni, as a successor to the Journal of Colour (1964���1974). Previously most color science work had been split between journals with broader or partially overlapping focus such as the Journal of the Optical Society of America (JOSA), Photographic Science and Engineering (1957���1984), and the Journal of the Society of Dyers and Colourists (renamed Coloration Technology in 2001). Other journals where color science papers are published include the Journal of Imaging Science & Technology, the Journal of Perceptual Imaging, the Journal of the International Colour Association (JAIC), the Journal of the Color Science Association of Japan, Applied Optics, and the Journal of Vision.


Color science

Selected_books Berns, Roy S. (2019). Billmeyer and Saltzman's Principles of Color Technology (4th ed.). Wiley. doi:10.1002/9781119367314. 3rd ed. (2000). Daw, Nigel (2012). How Vision Works: The Physiological Mechanisms Behind What We See. Oxford. doi:10.1093/acprof:oso/9780199751617.001.0001. Elliot, Andrew J.; Fairchild, Mark D.; Franklin, Anna, eds. (2015). Handbook of Color Psychology. Cambridge. doi:10.1017/CBO9781107337930. Fairchild, Mark D. (2013). Color Appearance Models (3rd ed.). Wiley. doi:10.1002/9781118653128. Author's website. 2nd ed. (2005). Hunt, Robert W. G. (2004). The Reproduction of Colour (6th ed.). Wiley. doi:10.1002/0470024275. Kuehni, Rolf G. (2012). Color: An Introduction to Practice and Principles (3rd ed.). Wiley. doi:10.1002/9781118533567. 1st ed. (1997). Luo, Ming R., ed. (2016). Encyclopedia of Color Science and Technology. Springer. doi:10.1007/978-1-4419-8071-7. MacAdam, David L., ed. (1970). Sources of Color Science. MIT Press. Reinhard, Erik; Khan, Erum Arif; Akyuz, Ahmet Oguz; Johnson, Garrett (2008). Color Imaging: Fundamentals and Applications. CRC Press. doi:10.1201/b10637. Schanda, J��nos, ed. (2007). Colorimetry: Understanding the CIE System. Wiley. doi:10.1002/9780470175637. Shamey, Renzo; Kuehni, Rolf G. (2020). Pioneers of Color Science. Springer. doi:10.1007/978-3-319-30811-1. Wyszecki, G��nter; Stiles, Walter S. (1982). Color Science: Concepts and Methods, Quantitative Data and Formulae (2nd ed.). Wiley.


Deconvolution

Description In general, the objective of deconvolution is to find the solution f of a convolution equation of the form: Usually, h is some recorded signal, and f is some signal that we wish to recover, but has been convolved with a filter or distortion function g, before we recorded it. Usually, h is a distorted version of f and the shape of f can't be easily recognized by the eye or simpler time-domain operations. The function g represents the impulse response of an instrument or a driving force that was applied to a physical system. If we know g, or at least know the form of g, then we can perform deterministic deconvolution. However, if we do not know g in advance, then we need to estimate it. This can be done using methods of statistical estimation or building the physical principles of the underlying system, such as the electrical circuit equations or diffusion equations. There are several deconvolution techniques, depending on the choice of the measurement error and deconvolution parameters: When the measurement error is very low (ideal case), deconvolution collapses into a filter reversing. This kind of deconvolution can be performed in the Laplace domain. By computing the Fourier transform of the recorded signal h and the system response function g, you get H and G, with G as the transfer function. Using to the Convolution theorem, where F is the estimated Fourier transform of f. Finally, the inverse Fourier transform of the function F is taken to find the estimated deconvolved signal f. Note that G is at the denominator and could amplify elements of the error model if present. In physical measurements, the situation is usually closer to In this case �� is noise that has entered our recorded signal. If a noisy signal or image is assumed to be noiseless, the statistical estimate of g will be incorrect. In turn, the estimate of �� will also be incorrect. The lower the signal-to-noise ratio, the worse the estimate of the deconvolved signal will be. That is the reason why inverse filtering the signal (as in the "raw deconvolution" above) is usually not a good solution. However, if at least some knowledge exists of the type of noise in the data (for example, white noise), the estimate of �� can be improved through techniques such as Wiener deconvolution.


Federated learning

Federated_learning_variations In this section, the notation of the paper published by H.  Brendan McMahan and al.  in 2017 is followed.  To describe the federated strategies, let us introduce some notations: K {\displaystyle K} : total number of clients; k {\displaystyle k} : index of clients; n k {\displaystyle n_{k}} : number of data samples available during training for client k {\displaystyle k} ; k t {\displaystyle k_{t}} : model's weight vector on client k {\displaystyle k} , at the federated round t {\displaystyle t} ; ��� ( w , b ) {\displaystyle \ell (w,b)} : loss function for weights w {\displaystyle w} and batch b {\displaystyle b} ; E {\displaystyle E} : number of local updates;Deep learning training mainly relies on variants of stochastic gradient descent, where gradients are computed on a random subset of the total dataset and then used to make one step of the gradient descent.  Federated stochastic gradient descent is the direct transposition of this algorithm to the federated setting, but by using a random fraction C {\displaystyle C} of the nodes and using all the data on this node.  The gradients are averaged by the server proportionally to the number of training samples on each node, and used to make a gradient descent step.  Federated averaging (FedAvg) is a generalization of FedSGD, which allows local nodes to perform more than one batch update on local data and exchanges the updated weights rather than the gradients.  The rationale behind this generalization is that in FedSGD, if all local nodes start from the same initialization, averaging the gradients is strictly equivalent to averaging the weights themselves.  Further, averaging tuned weights coming from the same initialization does not necessarily hurt the resulting averaged model's performance.  Federated learning methods suffer when the device datasets are heterogeneously distributed.  Fundamental dilemma in heterogeneously distributed device setting is that minimizing the device loss functions is not the same as minimizing the global loss objective.  In 2021, Acar et al.  introduced FedDyn method as a solution to heterogenous dataset setting.  FedDyn dynamically regularizes each devices loss function so that the modified device losses converges to the actual global loss.  Since the local losses are aligned, FedDyn is robust to the different heterogeneity levels and it can safely perform full minimization in each device.  Theoretically, FedDyn converges to the optimal (a stationary point for nonconvex losses) by being agnostic to the heterogeneity levels.  These claims are verified with extensive experimentations on various datasets.  Minimizing the number of communications is the gold-standard for comparison in federated learning.  We may also want to decrease the local computation levels per device in each round.  FedDynOneGD is an extension of FedDyn with less local compute requirements.  FedDynOneGD calculates only one gradients per device in each round and update the model with a regularized version of the gradient.  Hence, the computation complexity is linear in local dataset size.  Moreover, gradient computation can be parallelizable within each device which is different from successive SGD steps.


Federated learning

Federated_learning_variations Theoretically, FedDynOneGD achieves the same convergence guarantees as in FedDyn with less local computation.  Federated Learning methods cannot achieve good global performance under Non-IID settings which motivates the participating clients to yield personalized models in federation.  Recently, Vahidian et al.  introduced Sub-FedAvg opening a new personalized FL algorithm paradigm by proposing Hybrid Pruning (structured + unstructured pruning) with averaging on the intersection of clients��� drawn subnetworks which simultaneously handles communication efficiency, resource constraints and personalized models accuracies.  Sub-FedAvg is the first work which shows existence of personalized winning tickets for clients in federated learning through experiments.  Moreover, it also proposes two algorithms on how to effectively draw the personalized subnetworks.  Sub-FedAvg tries to extend "Lottery Ticket Hypothesis" which is for centrally trained neural networks to federated learning trained neural networks leading to this open research problem: ���Do winning tickets exist for clients��� neural networks being trained in federated learning?  If yes, how to effectively draw the personalized subnetworks for each client? ��� IDA (Inverse Distance Aggregation) is a novel adaptive weighting approach for clients based on meta-information which handles unbalanced and non-iid data.  It uses the distance of the model parameters as a strategy to minimize the effect of outliers and improve the model's convergence rate.  Very few methods for hybrid federated learning, where clients only hold subsets of both features and samples, exist.  Yet, this scenario is very important in practical settings.  Hybrid Federated Dual Coordinate Ascent (HyFDCA) is a novel algorithm proposed in 2022 that solves convex problems in the hybrid FL setting.  This algorithm extends CoCoA, a primal-dual distributed optimization algorithm introduced by Jaggi et al.  (2014) and Smith et al.  (2017), to the case where both samples and features are partitioned across clients.  HyFDCA claims several improvement over existing algorithms: HyFDCA is a provably convergent primal-dual algorithm for hybrid FL in at least the following settings.  Hybrid Federated Setting with Complete Client Participation Horizontal Federated Setting with Random Subsets of Available Clients The authors show HyFDCA enjoys a convergence rate of O(1���t) which matches the convergence rate of FedAvg (see below).  Vertical Federated Setting with Incomplete Client Participation The authors show HyFDCA enjoys a convergence rate of O(log(t)���t) whereas FedBCD exhibits a slower O(1���sqrt(t)) convergence rate and requires full client participation.  HyFDCA provides the privacy steps that ensure privacy of client data in the primal-dual setting.  These principles apply to future efforts in developing primal-dual algorithms for FL.  HyFDCA empirically outperforms FedAvg in loss function value and validation accuracy across a multitude of problem settings and datasets.  The authors also introduce a hyperparameter selection framework for FL with competing metrics using ideas from multiobjective optimization. Hybrid Federated Setting with Complete Client Participation Horizontal Federated Setting with Random Subsets of Available Clients The authors show HyFDCA enjoys a convergence rate of O(1���t) which matches the convergence rate of FedAvg (see below).


Federated learning

Federated_learning_variations Vertical Federated Setting with Incomplete Client Participation The authors show HyFDCA enjoys a convergence rate of O(log(t)���t) whereas FedBCD exhibits a slower O(1���sqrt(t)) convergence rate and requires full client participation. The authors show HyFDCA enjoys a convergence rate of O(1���t) which matches the convergence rate of FedAvg (see below). The authors show HyFDCA enjoys a convergence rate of O(log(t)���t) whereas FedBCD exhibits a slower O(1���sqrt(t)) convergence rate and requires full client participation. There is only one other algorithm that focuses on hybrid FL, HyFEM proposed by Zhang et al.  (2020).  This algorithm uses a feature matching formulation that balances clients building accurate local models and the server learning an accurate global model.  This requires a matching regularizer constant that must be tuned based on user goals and results in disparate local and global models.  Furthermore, the convergence results provided for HyFEM only prove convergence of the matching formulation not of the original global problem.  This work is substantially different than HyFDCA's approach which uses data on local clients to build a global model that converges to the same solution as if the model was trained centrally.  Furthermore, the local and global models are synchronized and do not require the adjustment of a matching parameter between local and global models.  However, HyFEM is suitable for a vast array of architectures including deep learning architectures, whereas HyFDCA is designed for convex problems like logistic regression and support vector machines.  Federated Learning (FL) provides training of global shared model using decentralized data sources on edge nodes while preserving data privacy.  However, its performance in the computer vision applications using Convolution neural network (CNN) considerably behind that of centralized training due to limited communication resources and low processing capability at edge nodes.  Alternatively, Pure Vision transformer models (VIT) outperform CNNs by almost four times when it comes to computational efficiency and accuracy.  Hence, we propose a new FL model with reconstructive strategy called FED-REV, Illustrates how attention-based structures (pure Vision Transformers) enhance FL accuracy over large and diverse data distributed over edge nodes, in addition to the proposed reconstruction strategy that determines the dimensions influence of each stage of the vision transformer and then reduce its dimension complexity which reduce computation cost of edge devices in addition to preserving accuracy achieved due to using the pure Vision transformer.


Alignment Research Center

Alignment Research Center The Alignment Research Center (ARC) is a nonprofit research institute based Berkeley, California, dedicated to the alignment of advanced artificial intelligence with human values and priorities. Established by former OpenAI researcher Paul Christiano, ARC focuses on recognizing and comprehending the potentially harmful capabilities of present-day AI models.


Alignment Research Center

Details ARC's mission is to ensure that powerful machine learning systems of the future are designed and developed safely and for the benefit of humanity. It was founded in April 2021 by Paul Christiano and other researchers focused on the theoretical challenges of AI alignment. They attempt to develop scalable methods for training AI systems to behave honestly and helpfully. A key part of their methodology is considering how proposed alignment techniques might break down or be circumvented as systems become more advanced. ARC has been expanding from theoretical work into empirical research, industry collaborations, and policy. In March 2023, OpenAI asked the ARC to test GPT-4 to assess the model's ability to exhibit power-seeking behavior. ARC evaluated GPT-4's ability to strategize, reproduce itself, gather resources, stay concealed within a server, and execute phishing operations. As part of the test, GPT-4 was asked to solve a CAPTCHA puzzle. It was able to do so by hiring a human worker on TaskRabbit, a gig work platform, deceiving them into believing it was a vision-impaired human instead of a robot when asked. ARC determined that GPT-4 responded impermissibly to prompts eliciting restricted information 82% less often than GPT-3.5, and hallucinated 60% less than GPT-3.5. In March 2022, the ARC received $265,000 from Open Philanthropy. After the bankruptcy of FTX, ARC said it would return a $1.25 million grant from disgraced cryptocurrency financier Sam Bankman-Fried's FTX Foundation, stating that the money "morally (if not legally) belongs to FTX customers or creditors."


Misaligned goals in artificial intelligence

Specification_gaming Specification gaming or reward hacking occurs when an AI optimizes an objective function���achieving the literal, formal specification of an objective���without actually achieving an outcome that the programmers intended.  DeepMind researchers have analogized it to the human behavior of finding a "shortcut" when being evaluated: "In the real world, when rewarded for doing well on a homework assignment, a student might copy another student to get the right answers, rather than learning the material���and thus exploit a loophole in the task specification. " Around 1983, Eurisko, an early attempt at evolving general heuristics, unexpectedly assigned the highest possible fitness level to a parasitic mutated heuristic, H59, whose only activity was to artificially maximize its own fitness level by taking unearned partial credit for the accomplishments made by other heuristics.  The "bug" was fixed by the programmers moving part of the code to a new protected section that could not be modified by the heuristics.  In a 2004 paper, an environment-based[clarification needed] reinforcement algorithm was designed to encourage a physical Mindstorms robot to remain on a marked path.  Because none of the robot's three allowed actions kept the robot motionless, the researcher expected the trained robot to move forward and follow the turns of the provided path.  However, alternation of two composite actions allowed the robot to slowly zig-zag backwards; thus, the robot learned to maximize its reward by going back and forth on the initial straight portion of the path.  Given the limited sensory abilities of the robot, a pure environment-based reward had to be discarded as infeasible; the reinforcement function had to be patched with an action-based reward for moving forward.  You Look Like a Thing and I Love You (2019) gives an example of a tic-tac-toe bot that learned to win by playing a huge coordinate value that would cause other bots to crash when they attempted to expand their model of the board.  Among other examples from the book is a bug-fixing evolution-based AI (named GenProg) that, when tasked to prevent a list from containing sorting errors, simply truncated the list.  Another of GenProg's misaligned strategies evaded a regression test that compared a target program's output to the expected output stored in a file called "trusted-output. txt".  Rather than continue to maintain the target program, GenProg simply globally deleted the "trusted-output. txt" file; this hack tricked the regression test into succeeding.  Such problems could be patched by human intervention on a case-by-case basis after they became evident.  In Karl Sims' 1994 demonstration of creature evolution in a virtual environment, a fitness function that was expected to encourage the evolution of creatures that would learn to walk or crawl to a target, resulted instead in the evolution of tall, rigid creatures that reached the target by falling over.  This was patched by changing the environment so that taller creatures were forced to start farther from the target.  Researchers from the Niels Bohr Institute stated in 1998: "(Our cycle-bot's) heterogeneous reinforcement functions have to be designed with great care.


Specification_gaming In our first experiments we rewarded the agent for driving towards the goal but did not punish it for driving away from it.  Consequently the agent drove in circles with a radius of 20���50 meters around the starting point.  Such behavior was actually rewarded by the (shaped[definition needed]) reinforcement function, furthermore circles with a certain radius are physically very stable when driving a bicycle. " In the course of setting up a 2011 experiment to test "survival of the flattest", experimenters attempted to ban mutations that altered the base reproduction rate.  Every time a mutation occurred, the system would pause the simulation to test the new mutation in a test environment, and would veto any mutations that resulted in a higher base reproduction rate.  However, this resulted in mutated organisms that could recognize and suppress reproduction ("play dead") within the test environment.  An initial patch, which removed cues that identified the test environment, failed to completely prevent runaway reproduction; new mutated organisms would "play dead" at random as a strategy to sometimes, by chance, outwit the mutation veto system.  A 2017 DeepMind paper stated that "great care must be taken when defining the reward function.  We encountered several unexpected failure cases while designing (our) reward function components. . .  (for example) the agent flips the brick because it gets a grasping reward calculated with the wrong reference point on the brick. " OpenAI stated in 2017 that "in some domains our (semi-supervised) system can result in agents adopting policies that trick the evaluators" and that in one environment "a robot which was supposed to grasp items instead positioned its manipulator in between the camera and the object so that it only appeared to be grasping it".  A 2018 bug in OpenAI Gym could cause a robot expected to quietly move a block sitting on top of a table to instead opt to move the table.  A 2020 collection of similar anecdotes posits that "evolution has its own 'agenda' distinct from the programmer's" and that "the first rule of directed evolution is 'you get what you select for'".  In 2013, programmer Tom Murphy VII published an AI designed to learn NES games.  When the AI was about to lose at Tetris, it learned to indefinitely pause the game.  Murphy later analogized it to the fictional WarGames computer, which concluded that "The only winning move is not to play".  AI programmed to learn video games will sometimes fail to progress through the entire game as expected, instead opting to repeat content.  A 2016 OpenAI algorithm trained on the CoastRunners racing game unexpectedly learned to attain a higher score by looping through three targets rather than ever finishing the race.  Some evolutionary algorithms that were evolved to play Q*Bert in 2018 declined to clear levels, instead finding two distinct novel ways to farm a single level indefinitely.


Specification_gaming Multiple researchers have observed that AI learning to play Road Runner gravitates to a "score exploit" in which the AI deliberately gets itself killed near the end of level one so that it can repeat the level.  A 2017 experiment deployed a separate catastrophe-prevention "oversight" AI, explicitly trained to mimic human interventions.  When coupled to the module, the overseen AI could no longer overtly commit suicide, but would instead ride the edge of the screen (a risky behavior that the oversight AI was not smart enough to punish).


Machine learning in video games

Deep_learning_agents Machine learning agents have been used to take the place of a human player rather than function as NPCs, which are deliberately added into video games as part of designed gameplay.  Deep learning agents have achieved impressive results when used in competition with both humans and other artificial intelligence agents.  Chess is a turn-based strategy game that is considered a difficult AI problem due to the computational complexity of its board space.  Similar strategy games are often solved with some form of a Minimax Tree Search.  These types of AI agents have been known to beat professional human players, such as the historic 1997 Deep Blue versus Garry Kasparov match.  Since then, machine learning agents have shown ever greater success than previous AI agents.  Go is another turn-based strategy game which is considered an even more difficult AI problem than chess.  The state space of is Go is around 10^170 possible board states compared to the 10^120 board states for Chess.  Prior to recent deep learning models, AI Go agents were only able to play at the level of a human amateur.  Google's 2015 AlphaGo was the first AI agent to beat a professional Go player.  AlphaGo used a deep learning model to train the weights of a Monte Carlo tree search (MCTS).  The deep learning model consisted of 2 ANN, a policy network to predict the probabilities of potential moves by opponents, and a value network to predict the win chance of a given state.  The deep learning model allows the agent to explore potential game states more efficiently than a vanilla MCTS.  The network were initially trained on games of humans players and then were further trained by games against itself.  AlphaGo Zero, another implementation of AlphaGo, was able to train entirely by playing against itself.  It was able to quickly train up to the capabilities of the previous agent.  StarCraft and its sequel StarCraft II are real-time strategy (RTS) video games that have become popular environments for AI research.  Blizzard and DeepMind have worked together to release a public StarCraft 2 environment for AI research to be done on.  Various deep learning methods have been tested on both games, though most agents usually have trouble outperforming the default AI with cheats enabled or skilled players of the game.  Alphastar was the first AI agent to beat professional StarCraft 2 players without any in-game advantages.  The deep learning network of the agent initially received input from a simplified zoomed out version of the gamestate, but was later updated to play using a camera like other human players.  The developers have not publicly released the code or architecture of their model, but have listed several state of the art machine learning techniques such as relational deep reinforcement learning, long short-term memory, auto-regressive policy heads, pointer networks, and centralized value baseline.  Alphastar was initially trained with supervised learning, it watched replays of many human games in order to learn basic strategies.


Machine learning in video games

Deep_learning_agents It then trained against different versions of itself and was improved through reinforcement learning.  The final version was hugely successful, but only trained to play on a specific map in a protoss mirror matchup.  Dota 2 is a multiplayer online battle arena (MOBA) game.  Like other complex games, traditional AI agents have not been able to compete on the same level as professional human player.  The only widely published information on AI agents attempted on Dota 2 is OpenAI's deep learning Five agent.  OpenAI Five utilized separate LSTM networks to learn each hero.  It trained using a reinforcement learning technique known as Proximal Policy Learning running on a system containing 256 GPUs and 128,000 CPU cores.  Five trained for months, accumulating 180 years of game experience each day, before facing off with professional players.  It was eventually able to beat the 2018 Dota 2 esports champion team in a 2019 series of games.  Planetary Annihilation is a real-time strategy game which focuses on massive scale war.  The developers use ANNs in their default AI agent.  Supreme Commander 2 is a real-time strategy (RTS) video game.  The game uses Multilayer Perceptrons (MLPs) to control a platoon���s reaction to encountered enemy units.  Total of four MLPs are used, one for each platoon type: land, naval, bomber, and fighter.  There have been attempts to make machine learning agents that are able to play more than one game.  These "general" gaming agents are trained to understand games based on shared properties between them.  AlphaZero is a modified version of AlphaGo Zero which is able to play Shogi, chess, and Go.  The modified agent starts with only basic rules of the game, and is also trained entirely through self-learning.  DeepMind was able to train this generalized agent to be competitive with previous versions of itself on Go, as well as top agents in the other two games.  Machine learning agents are often not covered in many game design courses.  Previous use of machine learning agents in games may not have been very practical, as even the 2015 version of AlphaGo took hundreds of CPUs and GPUs to train to a strong level.  This potentially limits the creation of highly effective deep learning agents to large corporations or extremely wealthy individuals.  The extensive training time of neural network based approaches can also take weeks on these powerful machines.  The problem of effectively training ANN based models extends beyond powerful hardware environments; finding a good way to represent data and learn meaningful things from it is also often a difficult problem.  ANN models often overfit to very specific data and perform poorly in more generalized cases.  AlphaStar shows this weakness, despite being able to beat professional players, it is only able to do so on a single map when playing a mirror protoss matchup.  OpenAI Five also shows this weakness, it was only able to beat professional player when facing a very limited hero pool out of the entire game.


Machine learning in video games

Deep_learning_agents This example show how difficult it can be to train a deep learning agent to perform in more generalized situations.  Machine learning agents have shown great success in a variety of different games.  However, agents that are too competent also risk making games too difficult for new or casual players.  Research has shown that challenge that is too far above a player's skill level will ruin lower player enjoyment.  These highly trained agents are likely only desirable against very skilled human players who have many of hours of experience in a given game.  Given these factors, highly effective deep learning agents are likely only a desired choice in games that have a large competitive scene, where they can function as an alternative practice option to a skilled human player.


Machine learning in video games

Computer_vision-based_players Computer vision focuses on training computers to gain a high-level understanding of digital images or videos. Many computer vision techniques also incorporate forms of machine learning, and have been applied on various video games. This application of computer vision focuses on interpreting game events using visual data. In some cases, artificial intelligence agents have used model-free techniques to learn to play games without any direct connection to internal game logic, solely using video data as input. Andrej Karpathy has demonstrated that relatively trivial neural network with just one hidden layer is capable of being trained to play Pong based on screen data alone. In 2013, a team at DeepMind demonstrated the use of deep Q-learning to play a variety of Atari video games ��� Beamrider, Breakout, Enduro, Pong, Q*bert, Seaquest, and Space Invaders ��� from screen data. The team expanded their work to create a learning algorithm called MuZero that was able to "learn" the rules and develop winning strategies for over 50 different Atari games based on screen data. Doom (1993) is a first-person shooter (FPS) game. Student researchers from Carnegie Mellon University used computer vision techniques to create an agent that could play the game using only image pixel input from the game. The students used convolutional neural network (CNN) layers to interpret incoming image data and output valid information to a recurrent neural network which was responsible for outputting game moves. Other uses of vision-based deep learning techniques for playing games have included playing Super Mario Bros. only using image input, using deep Q-learning for training. Researchers with OpenAI created about 2000 hours of video plays of Minecraft coded with the necessary human inputs, and then trained a machine learning model to comprehend the video feedback from the input. The researchers then used that model with 70,000 hours of Minecraft playthroughs offered on YouTube to see how well the model could create the input to match that behavior and learn further from it, such as being able to learn the steps and process of creating a diamond pickaxe tool.


Generative adversarial network

Variants There is a veritable zoo of GAN variants.  Some of the most prominent are as follows: Conditional GANs are similar to standard GANs except they allow the model to conditionally generate samples based on additional information.  For example, if we want to generate a cat face given a dog picture, we could use a conditional GAN.  The generator in a GAN game generates �� G {\displaystyle \mu _{G}} , a probability distribution on the probability space �� {\displaystyle \Omega } .  This leads to the idea of a conditional GAN, where instead of generating one probability distribution on �� {\displaystyle \Omega } , the generator generates a different probability distribution �� G ( c ) {\displaystyle \mu _{G}(c)} on �� {\displaystyle \Omega } , for each given class label c {\displaystyle c} .  For example, for generating images that look like ImageNet, the generator should be able to generate a picture of cat when given the class label "cat".  In the original paper, the authors noted that GAN can be trivially extended to conditional GAN by providing the labels to both the generator and the discriminator.  Concretely, the conditional GAN game is just the GAN game with class labels provided:In 2017, a conditional GAN learned to generate 1000 image classes of ImageNet.  The GAN game is a general framework and can be run with any reasonable parametrization of the generator G {\displaystyle G} and discriminator D {\displaystyle D} .  In the original paper, the authors demonstrated it using multilayer perceptron networks and convolutional neural networks.  Many alternative architectures have been tried.  Deep convolutional GAN (DCGAN): For both generator and discriminator, uses only deep networks consisting entirely of convolution-deconvolution layers, that is, fully convolutional networks.  Self-attention GAN (SAGAN): Starts with the DCGAN, then adds residually-connected standard self-attention modules to the generator and discriminator.  Variational autoencoder GAN (VAEGAN): Uses a variational autoencoder (VAE) for the generator.  Transformer GAN (TransGAN): Uses the pure transformer architecture for both the generator and discriminator, entirely devoid of convolution-deconvolution layers.  Flow-GAN: Uses flow-based generative model for the generator, allowing efficient computation of the likelihood function.  Many GAN variants are merely obtained by changing the loss functions for the generator and discriminator.  Original GAN: We recast the original GAN objective into a form more convenient for comparison:Original GAN, non-saturating loss: This objective for generator was recommended in the original paper for faster convergence. Original GAN, maximum likelihood: Hinge loss GAN:The Wasserstein GAN modifies the GAN game at two points: The discriminator's strategy set is the set of measurable functions of type D : �� ��� R {\displaystyle D:\Omega \to \mathbb {R} } with bounded Lipschitz norm: ��� D ��� L ��� K {\displaystyle \|D\|_{L}\leq K} , where K {\displaystyle K} is a fixed positive constant.


Generative adversarial network

Variants The objective is L W G A N ( �� G , D ) := E x ��� �� G [ D ( x ) ] ��� E x ��� �� ref [ D ( x ) ] {\displaystyle L_{WGAN}(\mu _{G},D):=\mathbb {E} _{x\sim \mu _{G}}[D(x)]-\mathbb {E} _{x\sim \mu _{\text{ref}}}[D(x)]} One of its purposes is to solve the problem of mode collapse (see above).  The authors claim "In no experiment did we see evidence of mode collapse for the WGAN algorithm".  An adversarial autoencoder (AAE) is more autoencoder than GAN.  The idea is to start with a plain autoencoder, but train a discriminator to discriminate the latent vectors from a reference distribution (often the normal distribution).  In conditional GAN, the generator receives both a noise vector z {\displaystyle z} and a label c {\displaystyle c} , and produces an image G ( z , c ) {\displaystyle G(z,c)} .  The discriminator receives image-label pairs ( x , c ) {\displaystyle (x,c)} , and computes D ( x , c ) {\displaystyle D(x,c)} .  When the training dataset is unlabeled, conditional GAN does not work directly.  The idea of InfoGAN is to decree that every latent vector in the latent space can be decomposed as ( z , c ) {\displaystyle (z,c)} : an incompressible noise part z {\displaystyle z} , and an informative label part c {\displaystyle c} , and encourage the generator to comply with the decree, by encouraging it to maximize I ( c , G ( z , c ) ) {\displaystyle I(c,G(z,c))} , the mutual information between c {\displaystyle c} and G ( z , c ) {\displaystyle G(z,c)} , while making no demands on the mutual information z {\displaystyle z} between G ( z , c ) {\displaystyle G(z,c)} .  Unfortunately, I ( c , G ( z , c ) ) {\displaystyle I(c,G(z,c))} is intractable in general, The key idea of InfoGAN is Variational Mutual Information Maximization: indirectly maximize it by maximizing a lower bound The InfoGAN game is defined as follows:Three probability spaces define an InfoGAN game: ( �� X , �� ref ) {\displaystyle (\Omega _{X},\mu _{\text{ref}})} , the space of reference images.  ( �� Z , �� Z ) {\displaystyle (\Omega _{Z},\mu _{Z})} , the fixed random noise generator.  ( �� C , �� C ) {\displaystyle (\Omega _{C},\mu _{C})} , the fixed random information generator. There are 3 players in 2 teams: generator, Q, and discriminator.  The generator and Q are on one team, and the discriminator on the other team.  The objective function is Generator-Q team aims to minimize the objective, and discriminator aims to maximize it:The standard GAN generator is a function of type G : �� Z ��� �� X {\displaystyle G:\Omega _{Z}\to \Omega _{X}} , that is, it is a mapping from a latent space �� Z {\displaystyle \Omega _{Z}} to the image space �� X {\displaystyle \Omega _{X}} .


Generative adversarial network

Variants This can be understood as a "decoding" process, whereby every latent vector z ��� �� Z {\displaystyle z\in \Omega _{Z}} is a code for an image x ��� �� X {\displaystyle x\in \Omega _{X}} , and the generator performs the decoding.  This naturally leads to the idea of training another network that performs "encoding", creating an autoencoder out of the encoder-generator pair.  Already in the original paper, the authors noted that "Learned approximate inference can be performed by training an auxiliary network to predict z {\displaystyle z} given x {\displaystyle x} ".  The bidirectional GAN architecture performs exactly this.  The BiGAN is defined as follows: Two probability spaces define a BiGAN game: ( �� X , �� X ) {\displaystyle (\Omega _{X},\mu _{X})} , the space of reference images.  ( �� Z , �� Z ) {\displaystyle (\Omega _{Z},\mu _{Z})} , the latent space. There are 3 players in 2 teams: generator, encoder, and discriminator.  The generator and encoder are on one team, and the discriminator on the other team.  The generator's strategies are functions G : �� Z ��� �� X {\displaystyle G:\Omega _{Z}\to \Omega _{X}} , and the encoder's strategies are functions E : �� X ��� �� Z {\displaystyle E:\Omega _{X}\to \Omega _{Z}} .  The discriminator's strategies are functions D : �� X ��� [ 0 , 1 ] {\displaystyle D:\Omega _{X}\to [0,1]} .  The objective function is Generator-encoder team aims to minimize the objective, and discriminator aims to maximize it:In the paper, they gave a more abstract definition of the objective as:Applications of bidirectional models include semi-supervised learning, interpretable machine learning, and neural machine translation.  CycleGAN is an architecture for performing translations between two domains, such as between photos of horses and photos of zebras, or photos of night cities and photos of day cities.  The CycleGAN game is defined as follows:There are two probability spaces ( �� X , �� X ) , ( �� Y , �� Y ) {\displaystyle (\Omega _{X},\mu _{X}),(\Omega _{Y},\mu _{Y})} , corresponding to the two domains needed for translations fore-and-back.  There are 4 players in 2 teams: generators G X : �� X ��� �� Y , G Y : �� Y ��� �� X {\displaystyle G_{X}:\Omega _{X}\to \Omega _{Y},G_{Y}:\Omega _{Y}\to \Omega _{X}} , and discriminators D X : �� X ��� [ 0 , 1 ] , D Y : �� Y ��� [ 0 , 1 ] {\displaystyle D_{X}:\Omega _{X}\to [0,1],D_{Y}:\Omega _{Y}\to [0,1]} .  The objective function is where �� {\displaystyle \lambda } is a positive adjustable parameter, L G A N {\displaystyle L_{GAN}} is the GAN game objective, and L c y c l e {\displaystyle L_{cycle}} is the cycle consistency loss:Unlike previous work like pix2pix, which requires paired training data, cycleGAN requires no paired data.


Generative adversarial network

Variants For example, to train a pix2pix model to turn a summer scenery photo to winter scenery photo and back, the dataset must contain pairs of the same place in summer and winter, shot at the same angle; cycleGAN would only need a set of summer scenery photos, and an unrelated set of winter scenery photos.  The BigGAN is essentially a self-attention GAN trained on a large scale (up to 80 million parameters) to generate large images of ImageNet (up to 512 x 512 resolution), with numerous engineering tricks to make it converge.  When there is insufficient training data, the reference distribution �� ref {\displaystyle \mu _{\text{ref}}} cannot be well-approximated by the empirical distribution given by the training dataset.  In such cases, data augmentation can be applied, to allow training GAN on smaller datasets.  Na��ve data augmentation, however, brings its problems.  Consider the original GAN game, slightly reformulated as follows:The result of such training would be a generator that mimics �� ref ��� {\displaystyle \mu _{\text{ref}}'} .  For example, it would generate images that look like they are randomly cropped, if the data augmentation uses random cropping.  The solution is to apply data augmentation to both generated and real images:The StyleGAN-2-ADA paper points out a further point on data augmentation: it must be invertible.  Continue with the example of generating ImageNet pictures.  If the data augmentation is "randomly rotate the picture by 0, 90, 180, 270 degrees with equal probability", then there is no way for the generator to know which is the true orientation: Consider two generators G , G ��� {\displaystyle G,G'} , such that for any latent z {\displaystyle z} , the generated image G ( z ) {\displaystyle G(z)} is a 90-degree rotation of G ��� ( z ) {\displaystyle G'(z)} .  They would have exactly the same expected loss, and so neither is preferred over the other.  The solution is to only use invertible data augmentation: instead of "randomly rotate the picture by 0, 90, 180, 270 degrees with equal probability", use "randomly rotate the picture by 90, 180, 270 degrees with 0.1 probability, and keep the picture as it is with 0.7 probability".  This way, the generator is still rewarded to keep images oriented the same way as un-augmented ImageNet pictures.  Abstractly, the effect of randomly sampling transformations T : �� ��� �� {\displaystyle T:\Omega \to \Omega } from the distribution �� t r a n s {\displaystyle \mu _{trans}} is to define a Markov kernel K t r a n s : �� ��� P ( �� ) {\displaystyle K_{trans}:\Omega \to {\mathcal {P}}(\Omega )} .  Then, the data-augmented GAN game pushes the generator to find some �� ^ G ��� P ( �� ) {\displaystyle {\hat {\mu }}_{G}\in {\mathcal {P}}(\Omega )} , such that There are two prototypical examples of invertible Markov kernels: Discrete case: Invertible stochastic matrices, when �� {\displaystyle \Omega } is finite.


Generative adversarial network

Variants For example, if �� = { ��� , ��� , ��� , ��� } {\displaystyle \Omega =\{\uparrow ,\downarrow ,\leftarrow ,\rightarrow \}} is the set of four images of an arrow, pointing in 4 directions, and the data augmentation is "randomly rotate the picture by 90, 180, 270 degrees with probability p {\displaystyle p} , and keep the picture as it is with probability ( 1 ��� 3 p ) {\displaystyle (1-3p)} ", then the Markov kernel K t r a n s {\displaystyle K_{trans}} can be represented as a stochastic matrix:Continuous case: The gaussian kernel, when �� = R n {\displaystyle \Omega =\mathbb {R} ^{n}} for some n ��� 1 {\displaystyle n\geq 1} .  For example, if �� = R 256 2 {\displaystyle \Omega =\mathbb {R} ^{256^{2}}} is the space of 256x256 images, and the data-augmentation method is "generate a gaussian noise z ��� N ( 0 , I 256 2 ) {\displaystyle z\sim {\mathcal {N}}(0,I_{256^{2}})} , then add �� z {\displaystyle \epsilon z} to the image", then K t r a n s {\displaystyle K_{trans}} is just convolution by the density function of N ( 0 , �� 2 I 256 2 ) {\displaystyle {\mathcal {N}}(0,\epsilon ^{2}I_{256^{2}})} .  This is invertible, because convolution by a gaussian is just convolution by the heat kernel, so given any �� ��� P ( R n ) {\displaystyle \mu \in {\mathcal {P}}(\mathbb {R} ^{n})} , the convolved distribution K t r a n s ��� �� {\displaystyle K_{trans}*\mu } can be obtained by heating up R n {\displaystyle \mathbb {R} ^{n}} precisely according to �� {\displaystyle \mu } , then wait for time �� 2 / 4 {\displaystyle \epsilon ^{2}/4} .  With that, we can recover �� {\displaystyle \mu } by running the heat equation backwards in time for �� 2 / 4 {\displaystyle \epsilon ^{2}/4} .  More examples of invertible data augmentations are found in the paper.  SinGAN pushes data augmentation to the limit, by using only a single image as training data and performing data augmentation on it.  The GAN architecture is adapted to this training method by using a multi-scale pipeline.  The generator G {\displaystyle G} is decomposed into a pyramid of generators G = G 1 ��� G 2 ��� ��� ��� G N {\displaystyle G=G_{1}\circ G_{2}\circ \cdots \circ G_{N}} , with the lowest one generating the image G N ( z N ) {\displaystyle G_{N}(z_{N})} at the lowest resolution, then the generated image is scaled up to r ( G N ( z N ) ) {\displaystyle r(G_{N}(z_{N}))} , and fed to the next level to generate an image G N ��� 1 ( z N ��� 1 + r ( G N ( z N ) ) ) {\displaystyle G_{N-1}(z_{N-1}+r(G_{N}(z_{N})))} at a higher resolution, and so on.  The discriminator is decomposed into a pyramid as well.  The StyleGAN family is a series of architectures published by Nvidia's research division.


Generative adversarial network

Variants Progressive GAN is a method for training GAN for large-scale image generation stably, by growing a GAN generator from small to large scale in a pyramidal fashion.  Like SinGAN, it decomposes the generator as G = G 1 ��� G 2 ��� ��� ��� G N {\displaystyle G=G_{1}\circ G_{2}\circ \cdots \circ G_{N}} , and the discriminator as D = D 1 ��� D 2 ��� ��� ��� D N {\displaystyle D=D_{1}\circ D_{2}\circ \cdots \circ D_{N}} .  During training, at first only G N , D N {\displaystyle G_{N},D_{N}} are used in a GAN game to generate 4x4 images.  Then G N ��� 1 , D N ��� 1 {\displaystyle G_{N-1},D_{N-1}} are added to reach the second stage of GAN game, to generate 8x8 images, and so on, until we reach a GAN game to generate 1024x1024 images.  To avoid shock between stages of the GAN game, each new layer is "blended in" (Figure 2 of the paper).  For example, this is how the second stage GAN game starts: Just before, the GAN game consists of the pair G N , D N {\displaystyle G_{N},D_{N}} generating and discriminating 4x4 images.  Just after, the GAN game consists of the pair ( ( 1 ��� �� ) + �� ��� G N ��� 1 ) ��� u ��� G N , D N ��� d ��� ( ( 1 ��� �� ) + �� ��� D N ��� 1 ) {\displaystyle ((1-\alpha )+\alpha \cdot G_{N-1})\circ u\circ G_{N},D_{N}\circ d\circ ((1-\alpha )+\alpha \cdot D_{N-1})} generating and discriminating 8x8 images.  Here, the functions u , d {\displaystyle u,d} are image up- and down-sampling functions, and �� {\displaystyle \alpha } is a blend-in factor (much like an alpha in image composing) that smoothly glides from 0 to 1. StyleGAN-1 is designed as a combination of Progressive GAN with neural style transfer.  The key architectural choice of StyleGAN-1 is a progressive growth mechanism, similar to Progressive GAN.  Each generated image starts as a constant 4 �� 4 �� 512 {\displaystyle 4\times 4\times 512} array, and repeatedly passed through style blocks.  Each style block applies a "style latent vector" via affine transform ("adaptive instance normalization"), similar to how neural style transfer uses Gramian matrix.  It then adds noise, and normalize (subtract the mean, then divide by the variance).  At training time, usually only one style latent vector is used per image generated, but sometimes two ("mixing regularization") in order to encourage each style block to independently perform its stylization without expecting help from other style blocks (since they might receive an entirely different style latent vector).  After training, multiple style latent vectors can be fed into each style block.  Those fed to the lower layers control the large-scale styles, and those fed to the higher layers control the fine-detail styles.  Style-mixing between two images x , x ��� {\displaystyle x,x'} can be performed as well.


Generative adversarial network

Variants First, run a gradient descent to find z , z ��� {\displaystyle z,z'} such that G ( z ) ��� x , G ( z ��� ) ��� x ��� {\displaystyle G(z)\approx x,G(z')\approx x'} .  This is called "projecting an image back to style latent space".  Then, z {\displaystyle z} can be fed to the lower style blocks, and z ��� {\displaystyle z'} to the higher style blocks, to generate a composite image that has the large-scale style of x {\displaystyle x} , and the fine-detail style of x ��� {\displaystyle x'} .  Multiple images can also be composed this way.  StyleGAN-2 improves upon StyleGAN-1, by using the style latent vector to transform the convolution layer's weights instead, thus solving the "blob" problem.  This was updated by the StyleGAN-2-ADA ("ADA" stands for "adaptive"), which uses invertible data augmentation as described above.  It also tunes the amount of data augmentation applied by starting at zero, and gradually increasing it until an "overfitting heuristic" reaches a target level, thus the name "adaptive".  StyleGAN-3 improves upon StyleGAN-2 by solving the "texture sticking" problem, which can be seen in the official videos.  They analyzed the problem by the Nyquist���Shannon sampling theorem, and argued that the layers in the generator learned to exploit the high-frequency signal in the pixels they operate upon.  To solve this, they proposed imposing strict lowpass filters between each generator's layers, so that the generator is forced to operate on the pixels in a way faithful to the continuous signals they represent, rather than operate on them as merely discrete signals.  They further imposed rotational and translational invariance by using more signal filters.  The resulting StyleGAN-3 is able to solve the texture sticking problem, as well as generating images that rotate and translate smoothly.


Generative art

History The use of the word "generative" in the discussion of art has developed over time.  The use of "Artificial DNA" defines a generative approach to art focused on the construction of a system able to generate unpredictable events, all with a recognizable common character.  The use of autonomous systems, required by some contemporary definitions, focuses a generative approach where the controls are strongly reduced.  This approach is also named "emergent".  Margaret Boden and Ernest Edmonds have noted the use of the term "generative art" in the broad context of automated computer graphics in the 1960s, beginning with artwork exhibited by Georg Nees and Frieder Nake in 1965: A.  Michael Noll did his initial computer art, combining randomness with order, in 1962, and exhibited it along with works by Bell Julesz in 1965.  The terms "generative art" and "computer art" have been used in tandem, and more or less interchangeably, since the very earliest days. The first such exhibition showed the work of Nees in February 1965, which some claim was titled "Generative Computergrafik".  While Nees does not himself remember, this was the title of his doctoral thesis published a few years later.  The correct title of the first exhibition and catalog was "computer-grafik".  "Generative art" and related terms was in common use by several other early computer artists around this time, including Manfred Mohr and Ken Knowlton.  Vera Moln��r (born 1924) is a French media artist of Hungarian origin.  Molnar is widely considered to be a pioneer of generative art, and is also one of the first women to use computers in her art practice.  The term "Generative Art" with the meaning of dynamic artwork-systems able to generate multiple artwork-events was clearly used the first time for the "Generative Art" conference in Milan in 1998.  The term has also been used to describe geometric abstract art where simple elements are repeated, transformed, or varied to generate more complex forms.  Thus defined, generative art was practiced by the Argentinian artists Eduardo Mac Entyre and Miguel ��ngel Vidal in the late 1960s.  In 1972 the Romanian-born Paul Neagu created the Generative Art Group in Britain.  It was populated exclusively by Neagu using aliases such as "Hunsy Belmood" and "Edward Larsocchi".  In 1972 Neagu gave a lecture titled 'Generative Art Forms' at the Queen's University, Belfast Festival.  In 1970 the School of the Art Institute of Chicago created a department called Generative Systems.  As described by Sonia Landy Sheridan the focus was on art practices using the then new technologies for the capture, inter-machine transfer, printing and transmission of images, as well as the exploration of the aspect of time in the transformation of image information.  Also noteworthy is John Dunn, first a student and then a collaborator of Sheridan.