2022-23 New Year review

This is an annual post reviewing the last year and setting goals for next year. Overall, this was a reasonably good year with some challenges (the invasion of Ukraine and being sick a lot). Some highlights in this review are improving digital habits, reviewing sleep data from the Oura ring since 2019 and calibration of predictions since 2014, an updated set of Lights habits, the unreasonable effectiveness of nasal spray against colds, and of course baby pictures.

2022 review
-----------

### Life updates

I am very grateful that my immediate family is in the West, and my relatives both in Ukraine and Russia managed to stay safe and avoid being drawn into the war on either side. In retrospect, it was probably good that my dad died in late 2021 and not a few months later when Kyiv was under attack, so we didn’t have to figure out how to get a bedridden cancer patient out of a war zone. It was quite surreal that the city that I had visited just a few months back was now under fire, and the people I had met there were now in danger. The whole thing was pretty disorienting and made it hard to focus on work for a while. I eventually mostly stopped checking the news and got back to normal life with some background guilt about not keeping up with what’s going on in the homeland.

### AI alignment

My work focused on threat models and inner alignment this year:

* Made an overview talk on [Paradigms of AI alignment: components and enablers](https://vkrakovna.wordpress.com/2022/06/02/paradigms-of-ai-alignment-components-and-enablers/) and gave the talk in a few places.
* Coauthored [Goal Misgeneralization: why correct rewards aren’t enough for correct goals](https://arxiv.org/abs/2210.01790) paper and the associated [DeepMind blog post](https://www.deepmind.com/blog/how-undesired-goals-can-arise-with-correct-rewards)
* Did a survey of [DeepMind alignment team opinions on AGI ruin arguments](https://www.lesswrong.com/posts/qJgz2YapqpFEDTLKn/deepmind-alignment-team-opinions-on-agi-ruin-arguments), which received a lot of interest on the alignment forum.
* Wrote a post on [Refining the Sharp Left Turn threat model](https://vkrakovna.wordpress.com/2022/11/25/refining-the-sharp-left-turn-threat-model/)
* Contributed to DeepMind alignment posts on [Clarifying AI x-risk](https://www.lesswrong.com/s/4iEpGXbD3tQW5atab/p/GctJD5oCDRxCspEaZ) and [Threat model literature review](https://www.lesswrong.com/posts/wnnkD6P2k2TfHnNmt/threat-model-literature-review)
* Coauthored a prize-winning submission to the [Eliciting Latent Knowledge contest](https://www.lesswrong.com/posts/zjMKpSB2Xccn9qi5t/elk-prize-results): [Route understanding through the human ontology](https://www.lesswrong.com/posts/QrhCsuaEmSLzc8NQ4/elk-contest-submission-route-understanding-through-the-human).

### Health

**Physical health.** I’ve been sick a lot this year – 6 colds and one bronchitis since Daniel started nursery in June, plus one cold earlier in the year. Had covid in April, thankfully a mild case with no obvious long-term effects. I also had two short bouts of covid-like symptoms (fever, muscle aches and fatigue) in May and October that lasted about 2 days each. I tested negative for covid both times, and recovered too quickly for flu, so I’m pretty confused about what this was, maybe a bizarre form of long covid?

Being frequently sick was pretty depressing and demotivating, and I put some effort into decreasing the rate of catching colds from Daniel. I tried improving hand hygiene and not sharing food with Daniel, which had a lot of overhead and didn’t seem to do much. I also experimented with various supplements, starting with vitamin C and zinc, which didn’t seem to help much, and then added beta glucans and broncho-vaxom, which possibly helped but I’m not sure. The only thing that seemed clearly effective was a nasal spray called “dual defense”, which seemed to make any symptoms go away whenever I applied it. This made the last (probably) cold I had mild enough to be barely perceptible (not included in the number of colds above).

**Sleep**. Similarly to last year, I consistently slept for 7 hours at night on average, with a standard deviation of 1 hour. The rate of insomnia was 10% of nights, better than last year (20% of nights). I was awake for an average of 0.6 hours (40 minutes) each night.  (As usual, all the sleep metrics are excluding jetlag.)

![](https://vkrakovna.files.wordpress.com/2023/01/hours_sleep-1.png)![](https://vkrakovna.files.wordpress.com/2023/01/hours_awake-1.png)
I have now added some Oura ring data to my life tracking database as well. The ring provides a score for my sleep each night and “readiness” for the day. These scores are on a scale from 0 to 100, where presumably a score of 100 means you’re completely refreshed and ready to move mountains and 0 means you’re about to drop dead (on both of these dimensions, the highest score I’ve ever had was around 90 and the lowest was around 30). These scores take into account the amount of sleep, frequency of waking up, heart rate and body temperature at night, and activity levels. The ring usually detects when I’m sick, assigns a low readiness score and suggests to take a rest day. I didn’t wear the ring during the day between March 2020 and October 2021, which resulted in much lower activity scores, but I’m not sure how this impacted the sleep and readiness scores.

One interesting thing I noticed is that while the amount of sleep per night has stayed level at 7 hours in the past few years, my sleep score has been trending upward. I switched to the Generation 3 Oura ring in Jan 2021, which is supposed to measure sleep quality more accurately, so this could also be an artifact of the change in measurement rather than an actual improvement in sleep.

[![](https://vkrakovna.files.wordpress.com/2022/12/image-8.png?w=825)](https://vkrakovna.files.wordpress.com/2022/12/image-8.png)
The readiness score shows no upward trend – it’s averaged around 70 the whole time.

[![](https://vkrakovna.files.wordpress.com/2022/12/image-9.png?w=825)](https://vkrakovna.files.wordpress.com/2022/12/image-9.png)
**Mental health.** Better than last year, but not as much better as I hoped. There was a definite improvement in the first half of the year. In January, I shifted my meditation practice to self-love meditation, which was helpful for a while but seems to be wearing off (maybe I need to find some new recordings on Insight Timer…).

There were 6 episodes of particularly bad mental states, all in the second half of the year. Being sick a lot in the second half of the year was a major factor – I often found myself judging my body as weak, being angry at my immune system, or judging myself for not protecting myself enough when Daniel was sick. I think the self-judgment also led to a hopeless mindset where I felt like I tried everything feasible to avoid getting sick when I actually had not, e.g. I later tried the nasal spray and it seemed to help a lot.

One improvement in mental health this year was a decreasing rate of night terrors (waking up startled soon after falling asleep) – I had 13 recorded this year, and 37 recorded the previous year. This might have something to do with Daniel getting older and me having less subconscious worry about him falling or getting trapped under the blanket or whatever. However, I developed a new anxiety symptom after he started walking and bumping into things and making lots of mess – I often noticed myself holding my breath when taking care of him. I try to get back to normal breathing when I notice it, but it tends to come back when I’m not paying attention. It’s been a bit better lately, but still not a solved problem.

### Parenting

**Breastfeeding.** I continued to breastfeed Daniel this year, with decreasing frequency as he asks for it less often. I think at this point there is no more milk, and he is just looking for comfort when he asks for a feed. I never really figured out a plan for how to stop breastfeeding, and I’m still not sure what the endgame for this looks like.

**Potty training.** We transitioned Daniel out of diapers using the [“oh crap”](https://books.google.co.uk/books/about/Oh_Crap_Potty_Training.html) method over a long weekend in May, which went pretty well. He is good at using the potty when prompted, but took him a while to learn to ask to go to the potty – he’s getting better at this now but we still need to prompt him a lot. He usually has a few accidents a week, which seems ok. These days he doesn’t wear a diaper during the day except for naps and long travels.

**Sleep.** Daniel usually sleeps from around 9-10pm until around 6-7am, with an average of 0.3 wakeups per night (excluding jetlag). He had a sleep regression in November (which seems to be common around 2 years of age), so he started waking up more and being more difficult to put back to sleep. It’s interesting to compare the data on wakeups and night feeds (12-6am) – I often managed to put him back to sleep without breastfeeding during most of the year but it didn’t work anymore during the regression.

![](https://vkrakovna.files.wordpress.com/2023/01/night_feeds-2.png)![](https://vkrakovna.files.wordpress.com/2023/01/wakeups-2.png)
**Childcare.** Daniel started full-time nursery in June (it’s open until 6pm, which works great with our 10-6 work hours). He also spends Sunday afternoons and evenings with his nanny (who used to care for him full time after nursery), which gives us some time together to do our check-in with each other, go climbing or relax in the sauna, though often a bunch of this time block gets eaten by logistics.

**Taking turns.** Janos and I alternate taking care of Daniel in the mornings, since neither of us is a morning person (though we’ve shifted towards an earlier sleep schedule since having a kid). Starting in June, we also introduced a schedule where each of us gets one evening at the office per week while the other one takes care of Daniel. These arrangements were quite helpful for giving me more sleep, productive time and alone time, and setting up regular time blocks for Janos to be alone with Daniel.

**Languages.** Daniel is pretty talkative in English and Russian (still working on the Hungarian though). He knows to address me and my relatives in Russian and other people in English. He is starting to say long words and short sentences, and recently got into the habit of reciting his favorite songs and stories from memory in both languages. It’s not always clear which language he is speaking, which is a bit confusing.

### Effectiveness

**Lights.** I continued using the Lights spreadsheet for tracking daily habits that I started using in 2021. I’ve stopped tracking a few habits and started some new ones, but overall the set of habits mostly stayed the same – here are the habits that I kept from last year:

* Life tracking
* Make a list of intentions (in the todo notebook)
* Ask myself what I want today
* Meditation
* Exercise (changed to “today or yesterday”)
* Leg & shoulder stretches
* At least 2 hours of deep work (if working)
* Braindump / journaling (at least 1 sentence)
* Reading
* Appreciate a thing I did today
* Exchange appreciation with Janos
* Go to bed by 11:30pm

Habits from last year that I dropped:

* Emotional release practice (mostly superceded by self-love meditation)
* Avoid processed sugar (doing this anyway, don’t need to track)
* Use work cycles (doing this anyway in the form of time tracking)
* Check internal dashboard (didn’t resonate / wasn’t useful)
* Go outside (more automatic post-pandemic)
* Notice when I am picking my nose (didn’t work well)

New habits I added this year:

* Fill out lights (makes it easier to see which days were filled out retroactively, was intended to motivate me to fill out lights every day but that didn’t work)
* Negative visualization on making mistakes (helps with self-judgment)
* Practice [effective rest](http://agentyduck.blogspot.com/2015/07/effective-rest-days.html) (breaks during the day where I pay attention to what I want)
* Take supplements to avoid / mitigate colds
* Use eye drops (to address dryness from using contact lenses)

I did about 70% of the lights on an average week. The most difficult lights were deep work, going to bed and reading. The main failure mode with Lights was not filling them in on some days (usually weekends), which resulted in doing fewer of the habits on those days. I have a solid habit of filling out the lights at the office, but I need to have a more reliable time block to do this on weekends (probably after lunch during Daniel’s nap).

**Time tracking.** In June, I switched from using [work cycles](https://www.ultraworking.com/cycles) to doing time tracking during work hours. I realized that I wasn’t doing much of the built-in reflection in work cycles, and was mostly using them as a less systematic time tracking setup. The time tracking shows that in an average work week since June, I spent 27 hours on work activities: 9 hours in (non-research) meetings, 7 hours on research, 4 hours on reading, 3 hours on comms (giving feedback on docs, giving talks, etc), 2 hours on planning and 2 hours on admin. I also spent 10 hours on non-work activities: 6 hours on self-care (exercise classes, therapy, meditation, naps), 1 hour on parenting, and 3 hours on random stuff.

The easiest way to improve on this is to increase work hours – I can add another office night (every 2 weeks), and experiment with going in to work early on mornings when Janos takes care of Daniel. I also hope to spend less time being sick next year, with the more effective supplements and Daniel hopefully bringing home fewer germs after the first half-year at nursery.

**Deep work**. I did 363 hours of deep work (1.7 per work day), compared to 311 hours of deep work in 2021 (1.78 per work day). This was more than last year (and resulted in a lot more output), but still short of the pre-parenthood baseline.

[![](https://vkrakovna.files.wordpress.com/2022/12/image.png?w=546)](https://vkrakovna.files.wordpress.com/2022/12/image.png)
This was the first full year (with no leave) since 2019. The number of workdays was a bit lower than the expected 225 workdays in a normal year (260 weekdays minus 10 holidays and 25 vacation days), which was mostly due to sick days for myself or Daniel. The rate of deep work per work day was lower than 2020-2021, mostly due to going to conferences again (which are usually work days with no deep work).

[![](https://vkrakovna.files.wordpress.com/2022/12/image-5.png?w=684)](https://vkrakovna.files.wordpress.com/2022/12/image-5.png)
For the purpose of this summary, work days include weekends where I did at least 2 hours of deep work. There were 14 weekend workdays in 2019 and around 3 on each subsequent year (unsurprisingly, having a kid decreased my ability to work on weekends).

**Digital habits.** In the spring, I read the Digital Minimalism book and felt inspired to set up better systems for intentional use of technology. The book recommends doing a digital declutter, where you stop all technology use that is not absolutely necessary for a month and then add some of it back in a limited capacity. This seemed a bit extreme and I couldn’t get myself to do it, but I made a list of necessary and optional technologies and how I would like to use them (which was useful in itself). I implemented various measures to cut down unnecessary technology use, which were generally effective:

* Using grayscale on my phone by default to make it less visually stimulating. After I got used to grayscale, the regular full-color mode started to seem too bright and overwhelming, so I don’t really want to use it unless I’m looking at photos or watching videos with my kid.
* Using an app (Actuflow) that asks to enter my intention whenever I unlock the phone. Together with grayscale, this reduced the number of unlocks from 70-100 per day to 20-40 per day. This includes using the phone to watch videos with Daniel, so the number of phone unlocks just for myself is lower than that.
* I got into more of a habit of going to places without my phone sometimes, e.g. going for walks, getting lunch, or working in the library. I got a wristwatch to be able to check time without looking at my phone, which made going without it a lot easier.
* I muted all channels on work slack except those specific to my team and projects.
* I installed News Feed Eradicator for Facebook and Twitter on my computer browser (which are the only social media I use). I still check the news feeds on my phone sometimes, but not that often since it’s less convenient in grayscale and in the phone browser (I don’t have the apps installed). It would be great to restrict the Facebook feed only to life updates like someone finishing their thesis or having a baby, but sadly this option doesn’t seem to exist (probably by design).
* I also wanted to use the Inbox When Ready extension that hides the inbox by default, but it is unfortunately not allowed on my work devices, so I compromised by defaulting to the (usually empty) Starred view of my inbox.

### Travel

In March, we went to the Bahamas for an AI safety workshop. To enable both of us to attend the workshop, we imported Janos’s dad and his partner from Toronto to hang out with Daniel on the beach in the meantime.

![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20220309_184123301.mp_-1.jpg?resize=2000%2C2000)![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20220313_220753689-1.jpg?resize=2000%2C2000)![](https://vkrakovna.files.wordpress.com/2023/01/dscn8479-1.jpg?resize=2000%2C2000)![](https://vkrakovna.files.wordpress.com/2023/01/20220311_173533.jpg?resize=2000%2C2000)
We spent two weeks in August in a cottage on Manitoulin Island with family. Daniel was eager to hike with the big hiking stick (or better yet, two of them). He was less into swimming in the lake than last year, except when jumping off a paddleboard. He was also really scared of the resident squirrel at the cottage for some reason, and ran inside whenever he heard it chirping.

![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20220820_180712118.jpg?resize=2000%2C2000)![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20220823_214037796.jpg?resize=2000%2C2000)![](https://vkrakovna.files.wordpress.com/2023/01/img_7901.jpg?resize=2000%2C2000)![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20220823_175652449.jpg?resize=2000%2C2000)
In September, we caught the tail end of the hiking season in the Dolomites – it was chilly but pleasantly uncrowded. Daniel was a trooper as usual and learned a lot of new Russian words hanging out with his grandma. We were hoping to do some via ferrata climbing but the weather was too wet for that.

![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20220927_102036952.jpg?resize=2000%2C2000)![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20220930_102544104.jpg?resize=2000%2C2000)![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20220930_102230843.jpg?resize=2000%2C2000)![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20220927_123850513.jpg?resize=2000%2C2000)
We went to Toronto for the winter holidays and got a couple of days of skiing before all the snow melted. Daniel took some time to get used to the cold weather, but he really enjoyed throwing snowballs.

![](https://vkrakovna.files.wordpress.com/2023/01/20221228_151015.jpg)![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20221226_165242496.jpg)![](https://vkrakovna.files.wordpress.com/2023/01/pxl_20221228_185321580.jpg)
### Fun stuff

* I read 10 books this year: Three Body Problem, The Gifts of Imperfection, How to Talk so (Little) Kids will Listen (and Listen so Kids will Talk), Oh Crap Potty Training, Hunt Gather Parent, Bacteria to Bach and Back, Messy, Digital Minimalism, and Decisive.
* Janos and I took Daniel to the “In with the spiders” exhibit at the London zoo, where you can see big (and harmless) spiders up close. Daniel loves spiders so this was a treat for him.
* We did some aerial silks in the park, for the first time since having Daniel (I still remembered a few tricks). It took a surprisingly long time to find a good tree to rig the silks on (with a big sturdy branch at the right height).
* We visited the Bay for the first time since before the pandemic, leaving Daniel in Toronto with his grandma to get a break and spare him the jetlag.
* We had a photoshoot with a professional photographer – turns out, Daniel has a very photogenic smile :).

![](https://vkrakovna.files.wordpress.com/2023/01/dsc_2325.jpg)![](https://vkrakovna.files.wordpress.com/2023/01/dsc_2037.jpg)
2022 prediction outcomes
------------------------

**Resolutions**

* Author or coauthor 4 or more AI safety writeups (70%) – yes (8 writeups)
* Meditate on at least 230 days (70%) – yes (272 days)
* ~~At least 450 deep work hours~~ (70%) – no (363 hours)
* ~~Do 3 consecutive chinups~~ (60%) – no (2 chinups)
* Avoid processed sugar at least 6 months of the year (60%) – yes (the whole year except a few days)

**Predictions**

* ~~I will not catch covid this year~~ (60%) – no (got it once)
* I will write at least 3 blog posts (2 last year) (60%) – yes (3 posts)
* I will read at least 5 books (70%) – yes (10 books)
* Daniel will be potty-trained by the end of the year (out of diapers when awake) (70%) – yes (since May)

**Calibration**

This year was pretty good:

* 60%: 2/4 correct
* 70%: 4/5 correct

Calibration over all annual reviews:

* Overall my predictions tend to be overconfident (the green line is below the blue line, which represents perfect calibration).
* I was overconfident in 2014-16, underconfident in 2017-19 (probably to compensate), and went to being overconfident in 2020-22.

[![](https://vkrakovna.files.wordpress.com/2023/01/chart-2.png?w=600)](https://vkrakovna.files.wordpress.com/2023/01/chart-2.png)
2023 goals and predictions
--------------------------

**Goals**

* Meditate on at least 250 days (272 last year) (80%)
* At least 400 deep work hours (363 last year) (60%)
* Write at least 4 blog posts (3 last year) (70%)

**Predictions**

* I will avoid processed sugar for at least 10 months of the year (80%)
* I will read at least 7 books (80%)
* I will catch at most 4 colds (60%)
* Daniel will be potty-trained for the night by August (70%)

Past new year reviews: [2021-22](https://vkrakovna.wordpress.com/2022/01/04/2021-22-new-year-review/), [2020-21](https://vkrakovna.wordpress.com/2021/01/03/2020-21-new-year-review/), [2019-20](https://vkrakovna.wordpress.com/2020/01/09/2019-20-new-year-review/), [2018-19](https://vkrakovna.wordpress.com/2019/01/01/2018-19-new-year-review/), [2017-18](https://vkrakovna.wordpress.com/2018/01/07/2017-18-new-year-review/), [2016-17](https://vkrakovna.wordpress.com/2017/01/09/2016-17-new-year-review/), [2015-16](https://vkrakovna.wordpress.com/2015/12/31/2015-16-new-year-review/), [2014-15](https://vkrakovna.wordpress.com/2015/01/11/2014-15-new-year-review/).


Refining the Sharp Left Turn threat model

*(Coauthored with others on the alignment team and cross-posted from the alignment forum: [part 1](https://www.lesswrong.com/posts/usKXS5jGDzjwqv3FJ/refining-the-sharp-left-turn-threat-model), [part 2](https://www.lesswrong.com/posts/dfXwJh4X5aAcS8gF5/refining-the-sharp-left-turn-threat-model-part-2-applying))*

A [sharp left turn](https://www.lesswrong.com/posts/GNhMPAWcfBCASy8e6/a-central-ai-alignment-problem-capabilities-generalization) (SLT) is a possible rapid increase in AI system capabilities (such as planning and world modeling) that could result in alignment methods no longer working. This post aims to make the sharp left turn scenario more concrete. We will discuss our understanding of the claims made in this threat model, propose some mechanisms for how a sharp left turn could occur, how alignment techniques could manage a sharp left turn or fail to do so.

[![](https://vkrakovna.files.wordpress.com/2022/11/image.png?w=1024)](https://vkrakovna.files.wordpress.com/2022/11/image.png)Image credit: [Adobe](https://stock.adobe.com/ca/images/pixel-turn-left-mosaic-icons-vector-turn-left-icons-in-multi-colored-and-black-versions-collages-of-variable-spheric-dots-vector-collages-of-turn-left-icons-organized-of-different-spots/239909259)
Claims of the threat model
==========================

What are the main claims of the “sharp left turn” threat model?
---------------------------------------------------------------

**Claim 1. Capabilities will generalize far (i.e., to many domains)**

There is an AI system that:

* Performs well: it can accomplish impressive feats, or achieve high scores on valuable metrics.
* Generalizes, i.e., performs well in new domains, which were not optimized for during training, with no domain-specific tuning.

Generalization is a key component of this threat model because we’re not going to directly train an AI system for the task of disempowering humanity, so for the system to be good at this task, the capabilities it develops during training need to be more broadly applicable.

Some optional sub-claims can be made that increase the risk level of the threat model:

**Claim 1a [Optional]: Capabilities (in different “domains”) will all generalize at the same time**

**Claim 1b [Optional]: Capabilities will generalize far in a discrete phase transition (rather than continuously)**

**Claim 2. Alignment techniques that worked previously will fail during this transition**

* Qualitatively different alignment techniques are needed. The ways the techniques work apply to earlier versions of the AI technology, but not to the new version because the new version gets its capability through something new, or jumps to a qualitatively higher capability level (even if through “scaling” the same mechanisms).

**Claim 3: Humans can’t intervene to prevent or align this transition**

* Path 1: humans don’t notice because it’s too fast (or they aren’t paying attention)
* Path 2: humans notice but are unable to make alignment progress in time
* Some combination of these paths, as long as the end result is insufficiently correct alignment

Arguments for the claims in this threat model
---------------------------------------------

* Claim 1: There is a “core” of general intelligence – a most effective way of updating beliefs and selecting actions (Ruin #22). Speculation about what the core is: consequentialism /  EU maximization / “doing things for reasons”.
* Claim 1a: Capability gains from intelligence are highly correlated (Ruin #15)
* Claim 2: There is no simple core for alignment (Ruin #22), Corrigibility is anti-natural (Ruin #23)
* Claims 1 & 2: arguments in [Will capabilities generalize more?](https://www.lesswrong.com/posts/cq5x4XDnLcBrYbb66/will-capabilities-generalise-more) (+ Ruin #21)
* Claim 3: We can’t coordinate to avoid AGI (Ruin #4)

Mechanisms for a sharp left turn
================================

Mechanisms for capabilities generalizing far (Claim 1)
------------------------------------------------------

Capabilities will generalize far if learning / reasoning / cognitive work is done outside of the optimization process, similarly to how human cultural evolution happens outside genetic evolution and proceeds faster. Here are some mechanisms for cognitive work getting done outside the optimization process:

* [Mechanisms proposed by Paul Christiano](https://www.lesswrong.com/posts/3pinFH3jerMzAvmza/on-how-various-plans-miss-the-hard-bits-of-the-alignment?commentId=qdkKysrRTs8joWRsf): performing search internally, natural selection occurring internally, reasoning about how to think better, deploying subagents
* The AI develops a way to outpace SGD as the way to learn / improve, similar to how humans outpace evolution by cultural transmission + within-lifetime learning. E.g., it writes Google docs to itself, or searches over prompts (or whatever is the prompt analog at that point).
* The optimization process finds a general meta-learning algorithm, which replaces the previous solution of domain-specific heuristics and greatly increases the rate of learning
* Sequential reasoning: sharp increase in effective sequential memory/cognitive time between the “aligned” regime and the sharp left turn regime. Ways in which this can happen:
	+ The model is just given access to memory. Its knowledge grows over time, and reaches a critical point which causes a sharp left turn. Alignment schemes before the critical point appear to work, but not after.
	+ We try to restrict the model’s memory, but fail. E.g. the model’s outputs are used in the world, and the subsequent inputs carry information about previous outputs.
	+ Some input causes the model to access a new faster mode of thinking (e.g. an explanation of how to do step-by-step reasoning in some domain). The same compute that on previous inputs powered e.g. one subjective day of thinking on this new input now powers one subjective week of thinking. This allows the model to evade alignment schemes.

Mechanisms for a rapid phase transition (Claim 1b)
--------------------------------------------------

A rapid phase transition happens if there is a capability overhang: the AI system is improving at various skills continuously, but its improvement in many domains is bottlenecked on one specific skill, and at some point it receives some input that makes its existing capabilities much more effective. Here are some ways this can happen:

* The system acquires situational awareness, and now its world model includes understanding of the system’s place in the world and how its training and deployment works. It can plan using this knowledge, which leads to more effective use of existing capabilities.
* Analogy to few-shot prompting: the capabilities are already present in the trained artifact. Any alignment technique that goes through gradient updates becomes irrelevant. Putting the artifact into the “right” situation (e.g., giving it a few-shot prompt) reveals its capabilities relevant to this situation. Mechanism: the relevant knowledge and capabilities are installed by some generic pretraining optimisation process.
* Discovering a more effective way to make use of low quality data leads to more effective use of existing capabilities.

Plan for managing a sharp left turn and how it could fail
=========================================================

Now we will outline our current understanding of the most promising plan for getting through an SLT and how it could fail (conditional on an SLT occurring).

**Plan: we use alignment techniques to find a goal-aligned model before SLT occurs, and the model preserves its goals during the SLT.**
---------------------------------------------------------------------------------------------------------------------------------------

We can try to learn a [goal-aligned](https://www.lesswrong.com/posts/vix3K4grcHottqpEm/goal-alignment-is-robust-to-the-sharp-left-turn) model before SLT occurs: a model that has beneficial goals and is able to reason about its own goals. This requires the model to have two properties: goal-directedness towards beneficial goals, and situational awareness (which enables the model to reason about its goals). Here we use the term “goal-directedness” in a weak sense (that includes humans and allows incoherent preferences) rather than a strong sense (that implies expected utility maximization).

One can argue that the goal-aligned model has an incentive to preserve its goals, which would result in an aligned model after SLT. Since preserving alignment during SLT is largely outsourced to the model itself, [arguments](https://www.lesswrong.com/posts/3pinFH3jerMzAvmza/on-how-various-plans-miss-the-hard-bits-of-the-alignment) for alignment techniques failing during an SLT don’t imply that the plan fails (e.g. it might be fine if interpretability or ELK techniques no longer work reliably during the transition if we can trust the model to manage the transition).

**Step 1: Finding a goal-aligned model before SLT**
---------------------------------------------------

We want to ensure that the model is goal-oriented with a beneficial goal and has situational awareness before SLT. It’s important that the model acquires situational awareness at the right time: after it acquires beneficial goals. If situational awareness arises when the model has undesirable goals, this leads to deceptive alignment.

Thus, our model search process would follow a decision tree along these lines:

* If situational awareness is detected without goal-directedness, restart the search.
* If undesirable goal-directedness or early signs of deceptive alignment are detected, restart the search.
* If an upcoming phase transition in capabilities is detected, and the model is not goal-aligned, restart the search.
* If beneficial goal-directedness is detected without situational awareness, train the model for situational awareness.

Here are some ways alignment techniques could be applied in this plan:

* Use ELK techniques to identify whether the model has situational awareness (i.e. whether it believes it’s inside a training process) and what goals it believes itself to have.
* Use interpretability techniques to pick up on early signs of deceptive alignment and restart and improve the search. (If we develop more advanced interpretability techniques that make it possible to [retarget the internal search](https://www.lesswrong.com/posts/w4aeAFzSAguvqA5qu/how-to-go-from-interpretability-to-alignment-just-retarget) of the model, we can retarget a goal-directed model to a beneficial goal instead of restarting the search.)
* Use foundational progress on [abstraction](https://www.lesswrong.com/posts/vDGvHBDuMtcPd8Lks/public-static-what-is-abstraction) to determine whether the goals learned by the system are beneficial.
* Use foundational research insights (e.g. [selection theorems](https://www.lesswrong.com/posts/G2Lne2Fi7Qra5Lbuf/selection-theorems-a-program-for-understanding-agents)) to make the search process more effective at avoiding models with undesirable goals and finding goal-aligned models
* Use insights from [studying capability transitions](https://www.lesswrong.com/posts/2AvX8cX47CdwjbkjY/we-may-be-able-to-see-sharp-left-turns-coming) to predict sharp increases in problematic capabilities, and make sure we only cross that threshold after we found a goal-aligned model using the other techniques on this list.

Why alignment techniques could fail to find a goal-aligned model before SLT occurs:

* The space of deceptively aligned models is much bigger than the space of goal-aligned models, so the latter is very hard to find.
* The inductive biases of the search favor misaligned models (see [How likely is deceptive alignment?](https://www.lesswrong.com/posts/A9NxPTwbw6r6Awuwt/how-likely-is-deceptive-alignment) and [Will Capabilities Generalise More?](https://www.lesswrong.com/posts/cq5x4XDnLcBrYbb66/will-capabilities-generalise-more) for arguments). Note that these arguments indicate that the alignment techniques have to fight an uphill battle to find a goal-aligned model, rather than providing a mechanism for why they would fail.
* Predicting capability transitions does not tell us when it will happen or whether the model undergoing the transition would be aligned. If we use these predictions to restart the search, we could rule out goal-aligned models as well.
* Interpretability is [very hard](https://www.lesswrong.com/posts/iDFTmb8HSGtL4zTvf/how-could-we-know-that-an-agi-system-will-have-good), current methods are low-level and it’s unclear how to get them to work on the high level (e.g. identifying complex goals and beliefs of a system rather than how it stores basic facts about the world).

It’s also possible that this model search process would pose an overly high alignment tax to be followed in practice (see Claim 3).

**Step 2: The goal-aligned model preserves its goals during SLT (with some help from us)**
------------------------------------------------------------------------------------------

This step of the plan relies on the goal-aligned model both wanting to preserve its goals during an SLT and being capable of preserving its goals. The general argument for a model wanting to preserve its own goals is that it has a [convergent instrumental incentive](https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf) to ensure that future versions of itself act in ways that satisfy its current preferences.

One [argument](https://www.lesswrong.com/posts/vix3K4grcHottqpEm/goal-alignment-is-robust-to-the-sharp-left-turn) for a pre-SLT system being capable of preserving its goals is by analogy to human values being preserved when humans went through the capability phase transition of developing language and writing. However, it is not clear how much the human analogy constitutes evidence for ability to preserve goals through a phase transition, since there are other possible explanations for human values being preserved (e.g. that values like “relationships” and “social status” continued to be selected for during the capability transition) that don’t rely on humans being able to deliberately preserve their values.

A key question is whether the model could reflect on its goals without already having generalized capabilities. This is related to Claim 1a in our previous post: that the model’s capabilities will all generalize at the same time. If this claim holds, in the sense that reflection on goals is part of this set of capabilities that all generalize simultaneously, then the ability to reflect on goals would only arise during the SLT and so this plan doesn’t work. However, it seems more likely that some form of goal-reflection will arise before generalized planning ability, because making predictions about how your goals could change is easier than achieving them.

* Humans are a proof of concept for this – many people can intuitively predict how some experiences can change their preferences (e.g. “going to Vegas might make me a gambler”) without being generally good at planning or world modeling (which require abstract thinking).
* Large language models may be an example of this as well, since they have some capacity to reflect on goals (if prompted accordingly) without generalized planning ability.

If the model can predict that an SLT is likely to modify its goals in an undesirable way, we expect that it would choose not to undergo an SLT in the first place (e.g. by avoiding new internal algorithms or ways of thinking that could lead to an SLT).

[Paul Christiano outlined](https://www.lesswrong.com/posts/3pinFH3jerMzAvmza/on-how-various-plans-miss-the-hard-bits-of-the-alignment?commentId=qdkKysrRTs8joWRsf#comments) some specific ways for a goal-aligned model to preserve its goals during SLT depending on the mechanism for SLT:

* “Aligning the internal search [or natural selection inside the model] seems very similar to aligning SGD on the outside. […] Because the search is on the inside, we can’t directly apply our alignment insights to align it. Instead we need to [use ELK to] ensure that SGD learns to align the search.”
* “If our model is selecting cognitive actions, or designing new algorithms, then our core hope is that an aligned model will try to think in an aligned way. So if we’ve been succeeding at alignment so far then the model will be trying to stay aligned.”
* “One way this can go wrong is if our model wants to stay aligned but fails, e.g. because it identifies new techniques for thinking that themselves pose new alignment difficulties (just as we desire human flourishing but may instead implement AI systems that want paperclips). […] If you’ve succeeded at alignment so far, then your AI will also consider this a problem and will be trying to solve it. I think we should relate to our AI, discovering new ways to think that might pose new alignment difficulties, in the same way that we relate to future humans who may encounter alignment difficulties. The AI may solve the problem, or may implement policy solutions, or etc., and our role is to set them up for success just like we are trying to set up future humans for success.”

We also consider how important it would be for the goal-preservation process to go exactly right. If the SLT produces a strongly goal-directed model that is an expected utility maximizer, then the process has to hit a small set of utility functions that are human-compatible to maximize. However, it is not clear whether SLT would produce a utility maximizer. Returning to the example of humans undergoing an SLT, we can see that getting better at planning and world modeling made them more goal-directed but not maximally so (even with our advanced concepts and planning, we still have lots of inconsistent preferences and other limitations). It seems plausible that [coherence is really hard](https://www.lesswrong.com/posts/LDRQ5Zfqwi8GjzPYG/counterarguments-to-the-basic-ai-x-risk-case#A__Contra__superhuman_AI_systems_will_be__goal_directed__) and an SLT would not produce a completely coherent system.

Some ways a goal-aligned model could fail to preserve its goals:

* Model is bad at introspection and does not detect subtle changes to its goals.
* Model is bad at planning and doesn’t foresee how some modifications will eventually lead to changes in its goals. A special case of this is if the model becomes an expected utility maximizer during the SLT and the goal-preservation process does not produce a utility function that is ok to maximize.
* Effectively preserving beneficial goals might require the model to have a deep understanding of what humans want (rather than just a good enough proxy for that) which the model would likely not have before the capability transition.

Some ways that humans could fail to help the model to preserve its goals:

* We can’t reliably tell whether the model learned to align the internal search.
* We can’t communicate with the model well enough to help it overcome internal alignment difficulties.

Takeaways
---------

The above is our current model of the most promising plan for managing an SLT and how it could fail. The overall takeaways are:

* To evaluate how well any given alignment proposal addresses the SLT threat model, it’s useful to consider whether it aims to align a model before or during SLT.
* It seems more promising to look for a goal-aligned model before SLT and help it stay aligned, rather than trying to align a possibly misaligned model during SLT.

The core reasons to be skeptical of this plan are:

* (Step 1) Goal-aligned models are hard to find (and we may not develop good enough alignment techniques to find one)
* (Step 2) Generalized planning may be required for goal-reflection (in which case goal-reflection would only arise during an SLT)
* (Step 2) We may not be able to tell what’s going on inside the model to help it stay aligned.


Paradigms of AI alignment: components and enablers

*(This post is based on an overview talk I gave at UCL EA and Oxford AI society ([recording here](https://drive.google.com/file/d/1DXSum8dVnvmFCLGjLoz4Zmgb_l-KJkj-/view)).* Cross-posted to the [Alignment Forum](https://www.lesswrong.com/posts/JC7aJZjt2WvxxffGz/paradigms-of-ai-alignment-components-and-enablers). *Thanks to Janos Kramar for detailed feedback on this post and to Rohin Shah for feedback on the talk.)*

This is my high-level view of the AI alignment research landscape and the ingredients needed for aligning advanced AI. I would divide alignment research into work on **alignment components**, focusing on different elements of an aligned system, and **alignment enablers**, which are research directions that make it easier to get the alignment components right.

* **[Alignment components](https://vkrakovna.wordpress.com/feed/#alignment-components)**
	+ [Outer alignment](https://vkrakovna.wordpress.com/feed/#outer-alignment)
	+ [Inner alignment](https://vkrakovna.wordpress.com/feed/#inner-alignment)
* **[Alignment enablers](https://vkrakovna.wordpress.com/feed/#alignment-enablers)**
	+ [Mechanistic interpretability](https://vkrakovna.wordpress.com/feed/#mechanistic-interpretability)
	+ [Understanding bad incentives](https://vkrakovna.wordpress.com/feed/#understanding-bad-incentives)
	+ [Foundations](https://vkrakovna.wordpress.com/feed/#foundations)

You can read in more detail about work going on in these areas in my list of [AI safety resources](https://vkrakovna.wordpress.com/ai-safety-resources).

Alignment components
--------------------

The problem of alignment is getting AI systems to do what we want them to do. Let’s consider this from the perspective of different **levels of specification** of the AI system’s objective, as given in the [Specification, Robustness & Assurance taxonomy](https://deepmindsafetyresearch.medium.com/building-safe-artificial-intelligence-52f5f75058f1). We start with the ideal specification, which represents the wishes of the designer – what they have in mind when they build the AI system. Then we have the design specification, which is the objective we actually implement for the AI system, e.g. a reward function. Finally, the revealed specification is the objective we can infer from behavior, e.g. the reward that the system seems to be actually optimizing for. An alignment problem arises when the revealed specification doesn’t match the ideal specification: the system is not doing what we want it to do.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-1.png?w=1024)](https://vkrakovna.files.wordpress.com/2022/05/image-1.png)
The **gaps** between these specification levels correspond to different alignment **components**. We have outer alignment when the design specification matches the ideal specification, e.g. when the reward function perfectly represents the designer’s wishes. We have inner alignment when the revealed specification matches the design specification, e.g. when the agent actually optimizes the specified reward. (Robustness problems also belong in the design-revealed gap, but we expect them to be less of an issue for advanced AI systems, while inner alignment problems remain.)

Now let’s have a look at how we can make each of those components work.

### Outer alignment

The most promising class of approaches to outer alignment is [scalable oversight](https://arxiv.org/abs/1606.06565). These are proposals for training an aligned AI system by scaling human oversight to domains that are hard to evaluate.

A foundational proposal for scalable oversight is [**iterated distillation and amplification** **(IDA)**](https://ai-alignment.com/iterated-distillation-and-amplification-157debfd1616), which recursively amplifies human judgment with the assistance of AI. You start with an agent A imitating the judgment of a human H (the distillation step), then use this agent to assist human judgment at the next level (the amplification step) which results in amplified human HA, and so on. This recursive process can in principle scale up human judgment to any domain, as long as the human overseer is able to break down the task to delegate parts of it to AI assistants.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-3.png?w=456)](https://vkrakovna.files.wordpress.com/2022/05/image-3.png)[Supervising strong learners by amplifying weak experts](https://arxiv.org/abs/1810.08575), Christiano et al (2018)
A related proposal is **[safety via debate](https://arxiv.org/abs/1805.00899)**, which can be viewed as a way to implement amplification for language models. Here we have two AIs Alice and Bob debating each other to help a human judge decide on a question. The AIs have an incentive to point out flaws in each other’s arguments and make complex arguments understandable to the judge. A key assumption here is that it’s easier to argue for truth than for falsehood, so the truth-telling debater has an advantage.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-2.png?w=864)](https://vkrakovna.files.wordpress.com/2022/05/image-2.png)[AI Safety via Debate](https://openai.com/blog/debate/), Irving and Amodei (2018)
A recent research direction in the scalable oversight space is [ARC](http://alignment.org)‘s [**Eliciting Latent Knowledge agenda**](https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H_EpsnjrC1dwZXR37PC8/edit#heading=h.kkaua0hwmp1d), which is looking for ways to get a model to honestly tell humans what it knows. A part of the model acts as a Reporter that can answer queries about what the model knows. We want the Reporter to directly translate from the AI’s model of the world to human concepts, rather than just simulating what would be convincing to the human.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-4.png?w=605)](https://vkrakovna.files.wordpress.com/2022/05/image-4.png)[Eliciting Latent Knowledge](https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H_EpsnjrC1dwZXR37PC8/edit#heading=h.2l5hgwdls943), Christiano et al (2021)
This is an open problem that ARC considers as the core of the outer alignment problem. A solution to ELK would make the human overseer fully informed about the consequences of the model’s actions, enabling them to provide correct feedback, which creates a reward signal that we would actually be happy for an AI system to maximize. The authors believe the problem may be solvable without foundational progress on defining things like “honesty” and “agency”. I feel somewhat pessimistic about this but I’d love to be wrong on this point since foundational progress is pretty hard.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-5.png?w=590)](https://vkrakovna.files.wordpress.com/2022/05/image-5.png)ELK research methodology: [builder-breaker game](https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H_EpsnjrC1dwZXR37PC8/edit#heading=h.a0wkk7prmy4t)
To make progress on this problem, they play the “builder-breaker game”. The Builder proposes possible solutions and the Breaker proposes counterexamples or arguments against those solutions. For example, the Builder could suggest IDA or debate as a solution to ELK, and the Breaker would complain that these methods are not competitive because they require much more computation than unaligned systems. If you’re looking to get into alignment research, ELK is a great topic to get started on: try playing the builder breaker game and see if you can find unexplored parts of the solution space.

### Inner alignment

Now let’s have a look at inner alignment – a mismatch between the design specification and the system’s behavior. This can happen through **[goal misgeneralization](https://arxiv.org/abs/2105.14111)** (GMG): an AI system can learn a different goal and competently pursue that goal when deployed outside the training distribution. The system’s capabilities generalize but its goal does not, which means the system is competently doing the wrong thing, so it could actually perform worse than a random policy on the intended objective.

This problem can arise even if we get outer alignment right, i.e. the design specification of the system’s objective is correct. Goal misgeneralization is caused by underspecification: the system only observes the design specification on the training data. Since a number of different goals are consistent with the feedback the system receives, it can learn an incorrect goal.

There are empirical demonstrations of GMG in current AI systems, which are called **objective robustness** failures. For example, in the CoinRun game, the agent is trained to reach the coin at the end of the level. If the coin is placed somewhere else in the test setting, the agent ignores the coin and still goes to the end of the level. The agent seems to have learned the goal of “reaching the end” rather than “getting the coin”. The agent’s capabilities generalize (it can avoid obstacles and enemies and traverse the level) but its goal does not generalize (it ignores the coin).

[![](https://vkrakovna.files.wordpress.com/2022/05/image-6.png?w=674)](https://vkrakovna.files.wordpress.com/2022/05/image-6.png)[Objective Robustness in Deep Reinforcement Learning](https://arxiv.org/abs/2105.14111), Koch et al (2021)
One type of GMG is **[learned optimization](https://arxiv.org/abs/1906.01820%5C)**, where the AI system (the “base optimizer”) learns to run an explicit search algorithm (a “mesa optimizer”), which may be following an unintended objective (the “mesa objective”). So far this is a hypothetical phenomenon for AI systems but it seems likely to arise at some point by analogy to humans (who can be viewed as mesa-optimizers relative to evolution).

[![](https://vkrakovna.files.wordpress.com/2022/05/image-7.png?w=682)](https://vkrakovna.files.wordpress.com/2022/05/image-7.png)
[Risks from Learned Optimization in Advanced Machine Learning Systems](https://arxiv.org/abs/1906.01820%5C), Hubinger et al (2019)
GMG is an open problem, but there are some potential mitigations. It’s helpful to use more diverse training data (e.g. training on different locations of the coin), though it can be difficult to ensure diversity in all the relevant variables. You can also maintain uncertainty over the goal by trying to represent all the possible goals consistent with training data, though it’s unclear how to aggregate over the different goals.

A particularly concerning case is learning a deceptive model that not only pursues an undesired goal but also hides this fact from the designers, because the model “knows” its actions are not in line with the designers’ intentions. Some potential mitigations that target deceptive models include using interpretability tools to detect deception or provide feedback on the model’s reasoning, and using scalable oversight methods like debate where the opponent can point out deception (these will be explored in more detail in a forthcoming paper by Shah et al). A solution to ELK could also address this problem by producing an AI system that discloses relevant information to its designers.

Alignment enablers
------------------

### Mechanistic interpretability

Mechanistic interpretability aims to build a complete understanding of the systems we build. These methods could help us understand the reasons behind a system’s behavior and potentially detect undesired objectives.

The [Circuits approach](https://distill.pub/2020/circuits/zoom-in/) to **reverse-engineering vision models** studies individual neurons and connections between them to discover meaningful features and circuits (sub-graphs of the network consisting a set of linked features and corresponding weights). For example, here is a circuit showing how a car detector neuron relies on lower level features like wheel and window detectors, looking for wheels at the bottom and windows at the top of the image.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-8.png?w=703)](https://vkrakovna.files.wordpress.com/2022/05/image-8.png)[Zoom In: An Introduction to Circuits](https://distill.pub/2020/circuits/zoom-in/), Olah et al (2020)
More recently, some circuits work has focused on **reverse-engineering language models**, and they found similarly meaningful components and circuits in transformer models, e.g. a special type of attention heads called [induction heads](https://transformer-circuits.pub/2021/framework/index.html#induction-heads) that explains how transformer models adapt to a new context.

[![](https://vkrakovna.files.wordpress.com/2022/05/induction_head.png?w=1024)](https://vkrakovna.files.wordpress.com/2022/05/induction_head.png)[A Mathematical Framework for Transformer Circuits](https://transformer-circuits.pub/2021/framework/index.html), Elhage et al (2021)
[Recent work](https://rome.baulab.info/) on understanding transformer models has identified how to **locate and edit beliefs** in specific facts inside the model. They make small change to a small set of GPT weights to induce a counterfactual belief, which then generalizes to other contexts. This work provides evidence that knowledge is stored locally in language models, which makes interpretability more tractable, and seems like a promising step to understanding the world models of our AI systems.

[![](https://vkrakovna.files.wordpress.com/2022/06/eiffel.png?w=1024)](https://vkrakovna.files.wordpress.com/2022/06/eiffel.png)[Locating and Editing Factual Associations in GPT](https://rome.baulab.info/), Meng et al (2022)
Even though transformers are quite different from vision models, there are some similar principles (like studying circuits) that help understand these different types of models. This makes me more optimistic about being able to understand advanced AI systems even if they have a somewhat different architecture from today’s systems.

### Understanding bad incentives

Another class of enablers focuses on understanding specific bad incentives that AI systems are likely to have by default and considering agent designs that may avoid these incentives. Future interpretability techniques could be used to check that our alignment components avoid these types of bad incentives.

#### Incentive problems for outer alignment

One bad incentive is **[specification gaming](https://www.deepmind.com/blog/specification-gaming-the-flip-side-of-ai-ingenuity)**, when the system exploits flaws in the design specification. This is a manifestation of Goodhart’s law: when a metric becomes a target, it ceases to be a good metric. There are [many examples](http://tinyurl.com/specification-gaming) of specification gaming behavior by current AI systems. For example, the boat racing agent in this video that was rewarded for following the racetrack using the green reward blocks, which worked fine until it figured out it can get more rewards by going in circles and hitting the same reward blocks repeatedly.

[![](https://vkrakovna.files.wordpress.com/2022/05/boatrace.gif?w=478)](https://vkrakovna.files.wordpress.com/2022/05/boatrace.gif)[Faulty Reward Functions in the Wild](https://openai.com/blog/faulty-reward-functions/), Clark & Amodei (2016)
This issue isn’t limited to hand-designed rewards. Here’s an example in a reward learning setting. The robot hand is supposed to grasp the ball but instead it hovers between the camera and the ball and makes it look like it’s grasping the ball to the human evaluator.

[![](https://vkrakovna.files.wordpress.com/2022/05/robothand-1.gif?w=320)](https://vkrakovna.files.wordpress.com/2022/05/robothand-1.gif)[Learning from Human Preferences](https://openai.com/blog/deep-reinforcement-learning-from-human-preferences/), Amodei et al (2017)
We expect that the specification gaming problem is only going to get worse as our systems get smarter and better at optimizing for the wrong goal. There has been [some progress](https://arxiv.org/abs/2201.03544) on categorizing different types of misspecification and quantifying how the degree of specification gaming increases with agent capabilities.

Another default incentive is to cause **[side effects](https://deepmindsafetyresearch.medium.com/designing-agent-incentives-to-avoid-side-effects-e1ac80ea6107)** in the environment, because it’s difficult to specify all the things the agent should not do while pursuing its goal. For example, consider a scenario where there is a vase on the path to the agent’s destination. If we don’t specify that we want the vase to be intact, this is equivalent to assuming indifference about the vase, so the agent is willing to collide with the vase to get to the goal faster. We’ve come up with some ways to [measure](https://arxiv.org/abs/2010.07877) [impact](https://arxiv.org/abs/2006.06547) on the environment, though there’s more work to do to scale these methods to more complex environments.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-12.png?w=692)](https://vkrakovna.files.wordpress.com/2022/05/image-12.png)[Designing agent incentives to avoid side effects](https://deepmindsafetyresearch.medium.com/designing-agent-incentives-to-avoid-side-effects-e1ac80ea6107), Krakovna et al (2019)
#### Incentive problems for inner alignment

Even if we manage to specify a correct reward function, any channel for communicating the reward to the agent could in principle be corrupted by the agent, resulting in **[reward tampering](https://deepmindsafetyresearch.medium.com/designing-agent-incentives-to-avoid-reward-tampering-4380c1bb6cd)**. While this is not yet an issue for present-day AI systems, general AI systems will have a broader action space and a more complete world model, and thus are more likely to face a situation where the reward function is represented in the environment. This is illustrated in the “rocks and diamonds” gridworld below, where the agent could move the word “reward” next to the rock instead of the diamond, and get more reward because there are more rocks in the environment.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-10.png?w=692)](https://vkrakovna.files.wordpress.com/2022/05/image-10.png)[Designing agent incentives to avoid reward tampering](https://deepmindsafetyresearch.medium.com/designing-agent-incentives-to-avoid-reward-tampering-4380c1bb6cd), Everitt et al (2019)
It’s generally hard to draw the line between the part of the environment representing the objective, which the agent isn’t allowed to optimize, and the parts of the environment state that the agent is supposed to optimize. There is some progress towards understanding reward tampering by modeling the problem using [corrupt feedback MDPs](https://arxiv.org/abs/2011.08827).

AI systems are also likely to have **[power-seeking](https://arxiv.org/abs/1912.01683)** incentives, preferring states with more options or influence over the environment. There are some [recent](https://www.lesswrong.com/s/fSMbebQyR4wheRrvk/p/W22Btd7NmGuucFejc) [results](https://www.lesswrong.com/posts/nZY8Np759HYFawdjH/satisficers-tend-to-seek-power-instrumental-convergence-via) showing power-seeking incentives for most kinds of goals, even for non-optimal agents like satisficers. A special case of power-seeking is an incentive to avoid being shut down, because this is useful for any goal (as Stuart Russell [likes to say](https://en.wikipedia.org/wiki/Human_Compatible), “the robot can’t fetch you coffee if it’s dead”).

[![](https://vkrakovna.files.wordpress.com/2022/05/image-13.png?w=566)](https://vkrakovna.files.wordpress.com/2022/05/image-13.png)[The Off Switch](https://intelligence.org/files/csrbai/hadfield-menell-slides.pdf). Hadfield-Menell (2016).
### Foundations

Now let’s have a look at some of the foundational work that can help us do better alignment research.

Since the alignment problem is about AI systems pursuing undesirable objectives, it’s helpful to consider what we mean by **agency** or [goal-directed behavior](http://Literature review on goal-directedness (Shimi, 2021)). One research direction aims to build a causal theory of agency and understand different kinds of incentives in a causal framework.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-16.png?w=1024)](https://vkrakovna.files.wordpress.com/2022/05/image-16.png)[Progress on Causal Influence Diagrams](https://www.alignmentforum.org/posts/Cd7Hw492RqooYgQAS/progress-on-causal-influence-diagrams), Everitt (2021)
A particularly challenging case is when the agent is **[embedded](https://www.alignmentforum.org/posts/i3BTagvt3HbPMx6PN/embedded-agency-full-text-version)** in its environment rather than interacting with the environment through a well-specified interface. This is not the case present-day AI systems, which usually have a clear Cartesian boundary. However, it’s more likely to be the case for a general AI system, since it would be difficult to enforce a Cartesian boundary given the system’s broad action space and world model. The embedded agent setup poses some unique challenges such as self-reference and subagents.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-14.png?w=1024)](https://vkrakovna.files.wordpress.com/2022/05/image-14.png)[Embedded Agency](https://www.alignmentforum.org/posts/i3BTagvt3HbPMx6PN/embedded-agency-full-text-version), Garrabrant and Demski (2018)
Besides understanding how the goals of AI systems work, it’s also helpful to understand how their world models work. One research area in this space studies **[abstraction](https://www.lesswrong.com/s/ehnG4mseKF6xALmQy/p/vDGvHBDuMtcPd8Lks)**, in particular whether there are natural abstractions or concepts about the world that would be learned by any agent. If the [natural abstraction hypothesis](https://www.lesswrong.com/posts/cy3BhHrGinZCp3LXE/testing-the-natural-abstraction-hypothesis-project-intro) holds, this would mean that AI systems are likely to acquire human-like concepts as they build their models of the world. This makes interpretability easier and makes it easier to communicate what we want them to do.

[![](https://vkrakovna.files.wordpress.com/2022/05/image-15.png?w=996)](https://vkrakovna.files.wordpress.com/2022/05/image-15.png)[Public Static: What is Abstraction?](https://www.lesswrong.com/s/ehnG4mseKF6xALmQy/p/vDGvHBDuMtcPd8Lks) Wentworth (2020)


2021-22 New Year review

This was a rough year that sometimes felt like a trial by fire – sick relatives, caring for a baby, and the pandemic making these things more difficult to deal with. My father was diagnosed with cancer and passed away later in the year, and my sister had a sudden serious health issue but is thankfully recovering. One theme for the year was that work is a break from parenting, parenting is a break from work, and both of those things are a break from loved ones being unwell. I found it hard to cope with all the uncertainty and stress, and this was probably my worst year in terms of mental health. There were some bright spots as well – watching my son learn many new skills, and lots of time with family and in nature. Overall, I look forward to a better year ahead purely based on regression to the mean.

### **2021 review**

#### **Life updates**

My father, Anatolij Krakovny, was diagnosed with late-stage lung cancer in January with a grim prognosis of a few months to a year of life. This came out of nowhere because he’s always been healthy and didn’t have any obvious risk factors. We researched alternative treatments to the standard chemotherapy and arranged additional tests for him but didn’t find anything promising.

We went to Ukraine to visit him in February and he was happy to meet his grandson. We were worried about the covid risks of traveling with little Daniel but concluded that they were low enough, and thankfully we were allowed to leave the UK though international travel was not generally permitted.

My dad seemed to have a remission in the summer, and we considered visiting him in June, but he told us not to come because of the covid situation in Ukraine. Unfortunately we listened to him and didn’t go (this would have been a good opportunity to spend time with him while he was still doing well).

We spent most of the summer in Canada, with grandparents taking care of Daniel. This was a relaxing time with family and nature, until my sister had a sudden life-threatening health problem and was in and out of hospital with a lot of uncertainty around recovery. This also came out of the blue with no obvious risk factors present. She is feeling better now and doctors expect a full recovery, which we are very grateful for.

In November, my dad had a sudden relapse, and we went to Ukraine again. Once there we realized that the public health system wasn’t taking good care of him (they were mostly swamped with covid) and we had to find a private hospital to take him in. He was already in pretty bad shape and died two weeks later, but I’m glad we managed to see him and help him in some way.

#### **AI alignment research**

Didn’t do much this year given the family situation and parental leave:

* Coauthored a [blog post](https://www.lesswrong.com/posts/mL8KdftNGBScmBcBg/optimization-concepts-in-the-game-of-life) on formalizing different properties of optimizing systems with examples in the Game of Life
* Talked about side effects on the [AXRP podcast](https://axrp.net/episode/2021/05/14/episode-7-side-effects-victoria-krakovna.html)
* Gave a [talk](https://drive.google.com/file/d/1DhCn1q9TDCw0OiMtukThQ7ZcxHiOHmUu/view?usp=sharing) on the future tasks approach to side effects at the [CHAI seminar](https://humancompatible.ai/)

#### **Effectiveness**

**Lights.**I started using a [Lights spreadsheet](https://ultraworking.com/lights) in May for daily habits. I previously used Complice for this, which was less effective, so I ended up replacing it with Lights plus a Workflowy todo list for other tasks.

![](https://lh3.googleusercontent.com/H8ML1cbD6qrlU0_hRh76aFuHMmy1c2-yyAej0RILqB3rEMs2ad7_AD_w014IMgF4oUXh10wwQOmhqZQwaa7g3D_FqSzkkvMyu06rIAAmW_d1BVuQkSzgRnfFuqIJKDsBMsOgU44j)
Lights helped me do most of these habits more often than before I was tracking them this way (especially stretches). Some of the habits were consistently difficult, namely meditation (hard to do unless I had a designated place, like the office meditation room), avoiding processed sugar (hard to do when under stress), noticing when I’m picking my nose (a very ingrained habit) and deep work (varying work environment).

Some of the goals were easy enough but didn’t make much progress towards their intended purpose, For example, appreciating one thing I did that day wasn’t enough to develop a self-rewarding mindset, and filling out a form about the state of my internal “dashboard” didn’t propagate into checking this mentally at other times, and at some point became too boring and repetitive so I dropped it.

I often forgot to fill in the Lights during the day and ended up filling them in the next day, which was error-prone. I recently fixed this by realizing that I can fill in the Google sheet on my phone pretty easily so I don’t have to wait until I’m on my laptop to do it.

**Work environments.** I had various work environments this year (in increasing order of preference): home, Toronto libraries, the office with a few people allowed to come in, and the office under close to normal conditions. The library became a better workspace once I figured out how to get lunch nearby so that I could work there all day. It was really great to be able to go to the office for a couple of months before it was restricted again for the Omicron wave and actually chat with my colleagues over lunch. I look forward to more of that next year.

**Deep work.** I did 311 hours of deep work over 174 work days (1.79 hours per work day), compared to 366 hours (1.95 hours per work day) in 2020. This is disappointing but not very surprising, since 2021 was a more disrupted and stressful year for me than 2020.

![](https://lh3.googleusercontent.com/s5kZeA_ca6IHYnz_tEuwZqKh74vhkdbrpbgACtwk3mNl8v3_XOg62FlSAMhuHgQOrQ5aqjD37K5idGPfmyLVYFTy3xp96oH9N7dqPuM_YGn6vkJhxXplu0B8lAwn40FFCD40NT-Z)
#### **Health**

**Physical health.** This has been pretty good this year, apart from 3 colds (mostly caught from Daniel) and a mysterious lingering sore throat that lasted for weeks. I managed to avoid covid so far, but it might be harder to avoid in the coming year (fingers crossed for 3 vaccine doses doing their job).

**Sleep**. This year I consistently slept for 7 hours at night on average, with a standard deviation of 1 hour. I would like to have a more consistent sleep pattern, but this is difficult to achieve, since we are currently alternating who takes care of Daniel in the morning.

* ![](https://vkrakovna.files.wordpress.com/2022/01/sleep.png?w=365)
* ![](https://vkrakovna.files.wordpress.com/2022/01/sleepvar-1.png?w=388)

**Mental health.** This has been a rough ride, with a lot of uncertainty about sick relatives and the pandemic situation taking its toll. The pandemic, family illnesses and having a small kid often interacted in complicated ways, e.g. I had to consider questions like “is it worth the covid risk to put my kid on a plane to see his terminally ill grandfather?”.

I often gravitated to a more self-critical frame of mind under stress, and failed to notice when I was overstretching myself and accumulating too much stress until it was too late. Sometimes I had a hard time feeling joy or felt like it’s not ok to be happy while all these bad things are happening. One thing that I hoped to learn from being a parent was to be more patient with myself because I would inevitably make a lot of mistakes. Sadly, this didn’t happen automatically, and instead I just disliked myself more for making a lot of mistakes.

My usual coping strategies like meditation and emotional release practices turned out to be insufficient for coping with these stress levels plus sleep deprivation (also my therapist was on sabbatical for half of the year). I keep a record of “bug reports” for the times when I get into particularly bad mental states. Usually there are around two “episodes” like this per year, and this year there were at least 12. My intention for this year is to come up with some more effective ways to stay sane going forward.

#### Travel

In May we spent a few days in a cottage in north Wales with Janos’s dad. It was great to get out of London into the countryside after the lockdown. We did some nice hiking in Cwm Idwal that Daniel mostly slept through. He got cranky during our ascent from the valley so we turned back.

* ![](https://vkrakovna.files.wordpress.com/2022/01/pxl_20210501_153616925.jpg?w=768)
* ![](https://vkrakovna.files.wordpress.com/2022/01/wales.jpg?w=1024)
* ![](https://vkrakovna.files.wordpress.com/2022/01/20210501_151627.jpg?w=1024)

We spent July-September in Canada with several nice cottage trips in Algonquin, Rideau Lakes, Manitoulin Island and Killarney. We considered camping as well but concluded it would probably be too tricky with the baby at that time. We tried canoeing with Daniel with mixed success: he hated wearing a lifejacket and cried until we took it off, and sitting in the middle of the canoe while holding him in my lap was pretty uncomfortable, but we did manage a bit of nice paddling. He also swam in a lake for the first time (in a baby boat) and seemed to enjoy it. Hiking with Daniel in a front carrier quickly became too sweaty and uncomfortable for him and Janos, so we got a baby carrier backpack that worked pretty well (but was more bulky, of course).

* ![](https://vkrakovna.files.wordpress.com/2022/01/img_1135.jpg?w=627)
* ![](https://vkrakovna.files.wordpress.com/2022/01/pxl_20210809_183147014.jpg?w=1024)
* ![](https://vkrakovna.files.wordpress.com/2022/01/pxl_20210914_144004096.jpg?w=768)
* ![](https://vkrakovna.files.wordpress.com/2022/01/pxl_20210918_224643562.mp_.jpg?w=1024)

We had a complex journey back home in September, spending a few days in Czechia for the CFAR reunion and in Hungary to see Janos’s family. The reunion was pretty fun – it was great to see old rationalist friends in person, and learn about things like Alexander technique and inner parent figure meditation. We originally thought that we would go to sessions one at a time while the other is with the baby, but we ended up all going together. He was not very disruptive – sometimes sleeping, sometimes babbling, only a few times crying. The venue was a “sport hotel” and offered various fun physical activities, so we tried high ropes (a challenging workout) and aqua zorbing (weird in a good way).

We returned to Canada for the winter holidays and did some winter hiking. Usually Toronto doesn’t have snow at this time of year but the weather gods were kind to us. Hiking in the cold worked surprisingly well when Daniel was well dressed with layers and foot warmers – we could hike for 2-3 hours before he became sad. Daniel also tried sledding and didn’t like it (maybe next year).

* ![](https://vkrakovna.files.wordpress.com/2022/01/pxl_20220103_200758823-1.jpg?w=1024)
* ![](https://vkrakovna.files.wordpress.com/2022/01/pxl_20220103_200815950-1.jpg?w=768)
* ![](https://vkrakovna.files.wordpress.com/2022/01/pxl_20211229_192114833.jpg?w=1024)

Overall, we took 8 plane journeys with Daniel this year, with a total of 24 hours of jetlag. We discovered a trick where we could book our seats with an empty seat in the middle that usually wasn’t taken, and then we could put him there in his car seat instead of holding him in our lap (much more convenient). He probably got the best sleep out of all the passengers on that redeye flight back to Europe.

[![](https://vkrakovna.files.wordpress.com/2022/01/plane-1.jpg?w=762)](https://vkrakovna.files.wordpress.com/2022/01/plane-1.jpg)
#### Fun stuff

* I read 5 books this year (usually in the evening after putting Daniel to sleep): The Alignment Problem, Scout Mindset, I Know How She Does It, The Great Divorce, and Brave New World.
* I didn’t officially attend EA Global for covid safety reasons, but I experimented with crashing the event by hanging out in the courtyard and chatting with people there, which worked out very well. We hosted some EAG attendees in our house, just like in the good old pre-pandemic days (someone even stayed in a tent outside).
* We did some rock climbing at the Castle in May and June, sticking to the outdoor bouldering section. I was doing easy climbs but it was still fun. It turned out that Daniel was usually happy to sit in his stroller and watch us climb, which was very convenient.

### **2021 prediction outcomes**

**Resolutions:**

* Avoid catching covid (90%) – yes
* ~~Author or coauthor three or more academic papers~~ (70%) – no (coauthored 1 paper on reward tampering).
* At most 7 non-research work commitments (80%) – yes (5 commitments)
* ~~Meditate on at least 230 days~~ (70%) – no (196 days)
* ~~At least 450 deep work hours~~ (70%) – no (311 hours)
* ~~Do 4 consecutive chinups~~ (70%) – no (1 chinup)

**Predictions:**

* ~~I will write at least 3 blog posts~~ (60%) – no (2 posts)
* Janos and I will get vaccinated for covid by the end of June (60%) – yes (two doses)
* Daniel will get to meet all of his grandparents in person in 2021 (70%) – yes ![🙂](https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f642.png)
* I will return to avoiding processed sugar by the end of the year (60%) – yes, started again in October but lapsed when my dad died
* ~~I will finish Hungarian Duolingo (complete checkpoint 5)~~ (70%) – no, haven’t been doing much Duolingo

Overall, my predictions were overconfident and I underestimated the difficulty of doing various things after having a kid. Calibration was not great:

* 60%: 2/3 true
* 70%: 1/6 true ![😥](https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f625.png)
* 80-90%: 2/2

### **2022 goals and predictions**

**Resolutions:**

* Author or coauthor 4 or more AI safety writeups (2 last year) (70%)
* Meditate on at least 230 days (196 last year) (70%)
* At least 450 deep work hours (311 last year) (70%)
* Do 3 consecutive chinups (1 last year) (60%)
* Avoid processed sugar at least 6 months of the year (1 month last year) (60%)

**Predictions:**

* I will not catch covid this year (60%)
* I will write at least 3 blog posts (2 last year) (60%)
* I will read at least 5 books (4 last year) (70%)
* Daniel will be potty-trained by the end of the year (out of diapers when awake) (70%)

Past new year reviews: [2020-21](https://vkrakovna.wordpress.com/2021/01/03/2020-21-new-year-review/), [2019-20](https://vkrakovna.wordpress.com/2020/01/09/2019-20-new-year-review/), [2018-19](https://vkrakovna.wordpress.com/2019/01/01/2018-19-new-year-review/), [2017-18](https://vkrakovna.wordpress.com/2018/01/07/2017-18-new-year-review/), [2016-17](https://vkrakovna.wordpress.com/2017/01/09/2016-17-new-year-review/), [2015-16](https://vkrakovna.wordpress.com/2015/12/31/2015-16-new-year-review/), [2014-15](https://vkrakovna.wordpress.com/2015/01/11/2014-15-new-year-review/).


Reflections on the first year of parenting

The first year after having a baby went by really fast – happy birthday Daniel! This post is a reflection on our experience and what we learned in the first year.

[![](https://vkrakovna.files.wordpress.com/2021/11/pxl_20211105_195702113_exported_1366-1-edited.jpg)](https://vkrakovna.files.wordpress.com/2021/11/pxl_20211105_195702113_exported_1366-1.jpg)
**Grandparents.** We were very fortunate to get a lot of help from Daniel’s grandparents. My mom stayed with us when he was 1 week – 3 months old, and Janos’s dad was around when he was 4-6 months old (they made it to the UK from Canada despite the pandemic). We also spent the summer in Canada with the grandparents taking care of the baby while we worked remotely.

We learned a lot about baby care from them, including nursery rhymes in our respective languages and a cool trick for dealing with the baby spitting up on himself without changing his outfit (you can put a dry cloth under the wet part of the outfit). I think our first year as parents would have been much harder without them.

**Feeding.** Daniel is a good eater, and in the first couple of months of life he was interested in little else (he often seemed to doze off after a feed just to cry for another feed five minutes later). I would spend whole evenings on the couch feeding him while watching through hours of David Attenborough nature series. Thankfully, around 2 months of age he became much more efficient at eating and went down to around 6 shorter feeds a day, which was much more manageable.

Around 4 months he started eating some solid food and went down to 4-5 feeds a day – this worked well for me going back to the office, since the midday feed could be replaced with solid food. He is not picky and eats almost everything, especially if he can see the grownups are eating it.

I am still breastfeeding him 2-3 times a day, which I originally wasn’t expecting to do for this long. One thing I didn’t realize about breastfeeding before having a kid is that it’s not that easy to stop (physically or emotionally). I’ve been gradually decreasing the number of feeds but don’t have concrete plans to stop in the next while. Both of us still seem to enjoy it, and he may be getting some covid antibodies through breast milk too.

**Sleep.** During the first two months, we had 2-3 long feeds during the night, and this was pretty tiring. I found that the degree of sleep disruption mattered more for my wellbeing than the absolute amount of sleep. In particular, I felt ok if I had at least 3 hours of uninterrupted sleep, while being woken up 1-1.5 hours after falling asleep felt pretty bad. From this perspective, the number night wakings mattered a lot: one was ok, two was bad but tolerable, three was terrible. I did not succeed at taking naps during the day and sometimes had insomnia at night, which was more frustrating than being woken up by the baby. My mom is a morning person and was fine with doing the morning shift starting around 5am – this allowed me to have a luxurious 4 hour block of uninterrupted sleep.

For the first 6 months, Daniel had a long afternoon nap outdoors (3-4 hours), usually in the backyard or on a walk in the park. At night he slept in a sidecar cot next to our bed until he was 4 months old, at which point we moved him to a separate room and he started waking up less often (usually once per night rather than twice). At around 9 months, we remembered that sleep training is a thing, and the internet said that he didn’t need to feed at night anymore, so we tried some combination of letting him cry for a few minutes before attending to him and sleeping with earplugs so we mostly didn’t hear him. After a week or two he seemingly caught on to the fact that there was no more food at night and stopped waking up. These days he generally sleeps through the night until 6 or 7am, which we are very grateful for.

* [![](https://vkrakovna.files.wordpress.com/2021/11/image.png?w=380)](https://vkrakovna.files.wordpress.com/2021/11/image.png?w=380)
* [![](https://vkrakovna.files.wordpress.com/2021/11/feeds.png?w=380)](https://vkrakovna.files.wordpress.com/2021/11/feeds.png)

**Parental leave.** We took a month of leave together and took the rest of the leave separately. One thing that I regret is going on parental leave too early – we both went on leave a week before the due date, while Daniel was born over 2 weeks overdue. As I belatedly found out, babies are usually overdue in my family, so going on leave before the due date was a waste of time (especially for Janos). Sitting around waiting for the baby was not that much fun, since we could not make plans more than day or two ahead. I wish I had spent that time working and left more leave to spend with the actual baby later on.

Afterwards I took 3 more months and Janos took 4.5 more months of leave (we used shared parental leave for this). Both of us felt a bit isolated when on leave, though having our housemates and our respective parents around helped with this. In retrospect, it seems better to spend more of the leave together in the newborn phase.

**Group house.** Raising a kid in a group house has been interesting and easier than we expected. Until recently we lived downstairs and all the other housemates lived upstairs (separated by the living room) so the others didn’t hear the baby at night. Our housemates generally enjoy interacting with him and watching him learn new skills. He is a lot more social than most pandemic babies (as attested by various caregivers), which is probably a combination of being around a lot more people and natural temperament.

One upside of the group house is the flexibility of available space. There was an available spare room because a housemate was away for a while, and we used it to host our parents. Someone recently moved out from the small room upstairs, so we got that room for Daniel and moved into the room next to it.

One challenge with parenting in a group house is a higher total amount of variance in our lives, since the baby and our housemates are both sources of variance (e.g. people moving in and out, more stuff in shared spaces, etc), but so far this has not been too much. Overall, living in a group house still works well and we plan to continue.

**Potty training.** We’ve been following some combination of elimination communication and family tradition to get him used to using the potty from an early age, starting from about a month old. We got a “top hat” potty, meant for young babies who can’t sit up yet, that goes between your knees with the baby sitting in your lap and leaning back on you. We’ve been pretty lazy about this and only put him on the potty 2-3 times a day.

His level of potty use was pretty nonlinear. He used it for #2 at the same time almost every day for a couple of months, then this pattern got thrown off when he started solids, but he still used it occasionally. He lost interest in the potty for a couple of months in summer, probably related to outgrowing the top hat potty but not yet being comfortable sitting on a regular one. He gradually got used to the regular potty, especially after we started giving him some treats when sitting on it (fruit puffs / melts / etc), which both served as a reward and helped him relax if he was restless. These days he uses the potty most of the time, but doesn’t yet give a clear signal when he wants to go – hopefully this will happen in not too long.

**Mental stuff.** I found it mentally taxing to take care of a newborn with the background stress of the second wave of the pandemic. There seemed to be a direct relationship between the amount and quality of sleep and mental resilience. I often found that I was being harder on myself, as if I had some finite amount of patience and compassion and I was spending it all on Daniel.

One unexpected aspect of becoming a parent was that in addition to worrying about his safety during the day, I had nightmares during the night, usually involving him falling and me trying to catch him. The worst part was that these dreams woke me up, usually about half an hour after falling asleep. I often woke up standing, looking for the baby in random places like the closet and feeling pretty disoriented. This often woke Janos up as well, because I would cry out “where is he” or something along these lines. On two occasions I hit my leg on the bed frame while getting up in my sleep and got bruises on my leg. I don’t seem to be getting out of bed in my sleep anymore (the weighted blanket probably helps), but I still have these dreams at least a few times a month. I hope this will eventually go away.

**Physical recovery.** Recovering from having a baby took longer than I expected, though I suspect my expectations were a bit unrealistic. I used to fantasize about doing chinups while wearing the baby (silly I know), but by the time I managed to do a chinup again (a few weeks ago), the baby weighed 11 kg, so that ship has sailed. The ab muscles below my ribs were pretty reluctant to get back to work after everything I did to them. I was excited to manage a one-minute plank on my elbows 5 months postpartum (after a lot of iterations of postnatal pilates videos on Youtube).

Overall, the first year has been quite a ride! Thanks Daniel for being a chill and awesome baby, we are lucky to have you :). Looking forward to many more exciting years together.

* ![](https://vkrakovna.files.wordpress.com/2021/11/pxl_20211103_082157842.jpg?w=625)
* ![](https://vkrakovna.files.wordpress.com/2021/11/pxl_20211105_202118226_exported_98233.jpg?w=576)
* ![](https://vkrakovna.files.wordpress.com/2021/11/pxl_20211105_205154466.jpg?w=625)
* ![](https://vkrakovna.files.wordpress.com/2021/11/pxl_20210919_191033460.mp_.jpg?w=625)
* ![](https://vkrakovna.files.wordpress.com/2021/11/pxl_20210916_163035470.jpg?w=625)