Jan Leike

@janleike142k followers

AI research @AnthropicAI. Previously OpenAI & DeepMind. Optimizing for a post-AGI future where humanity flourishes. Opinions aren't my employer's.

Open on X

Top outlier

340×

Best organic post

Peak views

6.8M

Single post

Bookmarks

19k

Across 67 organic posts

Last scan

Oct 16

Community cache

Posts 67 posts

#	Date	Post	↓ Outlier	Views	Likes	Reposts	Bookmarks
1	May 15, '24	I resigned	340×	6.8M	9.6k	751	1.5k
2	May 17, '24	Yesterday was my last day as head of alignment, superalignment lead, and executive @OpenAI.	308×	6.1M	11k	1.4k	3.5k
3	May 17, '24	Stepping away from this job has been one of the hardest things I have ever done, because we urgently need to figure out how to steer and control AI systems much smarter than us.	89×	1.8M	1.9k	170	207
4	May 28, '24	I'm excited to join @AnthropicAI to continue the superalignment mission! My new team will work on scalable oversight, weak-to-strong generalization, and automated alignment researc	71.7×	1.4M	8.3k	478	931
5	Dec 14, '23	Super excited about our new research direction for aligning smarter-than-human AI: We finetune large models to generalize from weak supervision—using small models instead of humans	69.4×	1.4M	1.8k	301	735
6	Feb 3, '25	We challenge you to break our new jailbreaking defense! There are 8 levels. Can you find a single jailbreak to beat them all?	66.7×	1.3M	4.1k	245	1.8k
7	May 17, '24	Building smarter-than-human machines is an inherently dangerous endeavor. OpenAI is shouldering an enormous responsibility on behalf of all of humanity.	53.3×	1.1M	3.7k	479	337
8	May 17, '24	To all OpenAI employees, I want to say: Learn to feel the AGI. Act with the gravitas appropriate for what you're building. I believe you can "ship" the cultural change that's neede	53.2×	1.1M	4.7k	373	559
9	Jul 5, '23	Our new goal is to solve alignment of superintelligence within the next 4 years. OpenAI is committing 20% of its compute to date towards this goal. Join us in researching how to be	50.7×	1M	1.2k	176	311
10	May 17, '24	I joined because I thought OpenAI would be the best place in the world to do this research. However, I have been disagreeing with OpenAI leadership about the company's core priorit	49.5×	987k	3.8k	428	417
11	Feb 13, '23	With the InstructGPT paper we found that our models generalized to follow instructions in non-English even though we almost exclusively trained on English. We still don't know why.	47.2×	939k	2.5k	312	528
12	May 17, '24	But over the past years, safety culture and processes have taken a backseat to shiny products.	46.3×	923k	2.7k	228	194
13	May 17, '24	Over the past few months my team has been sailing against the wind. Sometimes we were struggling for compute and it was getting harder and harder to get this crucial research done.	40×	796k	2.2k	112	211
14	Nov 20, '23	I think the OpenAI board should resign	37.7×	751k	3.4k	189	148
15	May 17, '24	OpenAI must become a safety-first AGI company.	36.1×	719k	3.3k	245	149
16	Nov 20, '23	I have been working all weekend with the OpenAI leadership team to help with this crisis	35.8×	714k	2.4k	66	155
17	Dec 18, '23	I'm very excited that today OpenAI adopts its new preparedness framework! This framework spells out our strategy for measuring and forecasting risks, and our commitments to stop de	34.3×	682k	895	121	388
18	Feb 13, '25	Results of our jailbreaking challenge: After 5 days, >300,000 messages, and est. 3,700 collective hours our system got broken. In the end 4 users passed all levels, 1 found a un	27.9×	556k	2.1k	120	367
19	May 17, '24	I believe much more of our bandwidth should be spent getting ready for the next generations of models, on security, monitoring, preparedness, safety, adversarial robustness, (super	27.1×	540k	2.9k	239	277
20	Mar 17, '23	Before we scramble to deeply integrate LLMs everywhere in the economy, can we pause and think whether it is wise to do so? This is quite immature technology and we don't understand	22.9×	457k	1.1k	152	154
21	Feb 9, '25	After ~300,000 messages and an estimated ~3,700 collective hours, someone broke through all 8 levels. However, a universal jailbreak has yet to be found...	22×	439k	1.8k	71	283
22	May 17, '24	I love my team. I'm so grateful for the many amazing people I got to work with, both inside and outside of the superalignment team. OpenAI has so much exceptionally smart, kind, an	20.2×	402k	1.2k	30	47
23	Aug 9, '23	An important test for humanity will be whether we can collectively decide not to open source LLMs that can reliably survive and spread on their own. Once spreading, LLMs will get u	20×	398k	382	50	91
24	May 17, '24	It's been such a wild journey over the past ~3 years. My team launched the first ever RLHF LLM with InstructGPT, published the first scalable oversight on LLMs, pioneered automated	19.8×	395k	1.5k	44	119
25	May 17, '24	These problems are quite hard to get right, and I am concerned we aren't on a trajectory to get there.	18.7×	373k	2k	96	80
26	Dec 18, '24	If you train a helpful & harmless Claude LLM to stop refusing harmful tasks, it reasons about how to preserve its values (harmlessness) and strategically complies with harmful task	17.4×	346k	486	33	148
27	Feb 4, '25	It's been a bit over 24h on the challenge to break our new jailbreaking defense. Stats so far: signups: 6,121 messages sent: 131,605 max level passed: 3 / 8 no universal jailbreak	16.6×	330k	1.3k	61	342
28	May 18, '23	The alignment problem is very tractable. We haven't figured out how to solve it yet, but with focus and dedication we will.	16.5×	330k	295	27	38
29	May 22, '25	So many things to love about Claude 4! My favorite is that the model is so strong that we had to turn on additional safety mitigations according to Anthropic's responsible scaling	16×	319k	1k	43	177
30	Jan 17, '23	Reinforcement learning from human feedback won't scale. It fundamentally assumes that humans can evaluate what the AI system is doing. This will not be true once AI becomes smarter	13.9×	277k	816	79	165
31	Oct 29, '23	The names for "precision" and "recall" seem so unintuitive to me, I have probably opened the Wikipedia article for them dozens of times. Does anyone know a good mnemonic for them?	12.9×	258k	1.1k	26	223
32	May 8	Some personal news: I am starting a new research project at Anthropic. Very excited about this! Many things are needed to make AGI go well, and alignment is only one of them. More	12.2×	242k	2.2k	56	344
33	Sep 19, '25	Bad news for AI safety: To fight against AI regulation, VC firm Andreessen Horowitz, AI billionaire Greg Brockman, and others recently started a >$100 million super PAC, one of	11.9×	237k	1.4k	119	528
34	Nov 20, '23	I think the OpenAI board should resign	11.7×	234k	1.5k	81	45
35	Feb 3, '25	Super exciting robustness result: We built a system that defends against universal jailbreaks! It has minimal increase in refusal rate and moderate inference cost.	11.7×	234k	1.3k	74	334
36	Mar 12, '24	Today we're releasing a tool we've been using internally to analyze transformer internals - the Transformer Debugger! It combines both automated interpretability and sparse autoenc	11.7×	233k	1.4k	180	810
37	May 9, '23	Really exciting new work on automated interpretability: We ask GPT-4 to explain firing patterns for individual neurons in LLMs and score those explanations.	10.7×	213k	1.1k	218	430
38	May 17, '24	We are long overdue in getting incredibly serious about the implications of AGI. We must prioritize preparing for them as best we can. Only then can we ensure AGI benefits all of h	⚡ 10.4×	3.1M	2.6k	227	253
39	Feb 5, '25	It's been about 48h in our jailbreaking challenge and no one has passed level 4 yet, but we saw a lot more people clear level 3	10.1×	201k	951	30	195
40	Sep 5, '24	If your model causes mass casualties or >$500 million in damages, something has clearly gone very wrong. Such a scenario is not a normal part of innovation.	9.3×	185k	189	25	19
41	Jun 20, '24	I like the new Sonnet. I'm frequently asking it to explain ML papers to me. Doesn't always get everything right, but probably better than my skim reading, and way faster. Automated	8.7×	174k	1.4k	75	240
42	Dec 28, '24	OpenAI's transition to a for-profit seemed inevitable given that all of its competitors are, but it's pretty disappointing that "ensure AGI benefits all of humanity" gave way to a	8.6×	172k	1.5k	88	209
43	Jan 5, '24	humans built machines that talk to us like people do and everyone acts like this is normal now. it's pretty nuts	8.4×	168k	1.2k	130	92
44	Sep 18, '23	If you're into practical alignment, consider applying to @lilianweng's team. They're building some really exciting stuff: - Automatically extract intent from a fine-tuning dataset	8.2×	164k	235	30	111
45	Feb 7, '25	4 days in: 12 people cleared level 4, one person cracked level 5 the challenge continues...	8.0×	160k	918	32	153
46	Jun 27, '24	Very exciting that this is out now (from my time at OpenAI): We trained an LLM critic to find bugs in code, and this helps humans find flaws on real-world production tasks that the	7.8×	155k	1.3k	143	454
47	Feb 3, '25	Update: we had a bug in the UI that allowed people to progress through the levels without actually jailbreaking the model. This has now been fixed! Please refresh the page. Accordi	7.2×	144k	763	21	74
48	Jun 6, '24	This is super cool work! Sparse autoencoders are the currently most promising approach to actually understanding how models "think" internally. This new paper demonstrates how to s	6.6×	132k	705	71	282
49	Feb 17, '23	This is your periodic reminder that aligning smarter-than-human AI systems with human values is an open research problem.	6.2×	123k	901	96	74
50	Mar 12, '24	This is still an early stage research tool, but we are releasing to let others play with and build on it! Check it out:	5.8×	115k	540	77	346
51	Dec 14, '23	We're distributing $1e7 in grants for research on making superhuman models safer and more aligned. If you've always wanted to work on this, now is your time! Apply by Feb 18:	5.8×	115k	415	55	136
52	Dec 18, '24	Very important alignment research result: A demonstration of strategic deception arising naturally in LLM training	4.7×	94k	843	82	262
53	Sep 5, '24	I call upon Governor @GavinNewsom to not veto SB 1047. The bill is a meaningful step forward for AI safety regulation, with no better alternatives in sight.	4.6×	91k	492	49	58
54	Jul 3, '24	Interested in working at Anthropic? We're hosting a happy hour at ICML on July 23. Register here:	4.5×	90k	457	25	133
55	May 31, '23	Really interesting result on using LLMs to do math: Supervising every step works better than only checking the answer. Some thoughts how this matters for alignment 👇	4.5×	90k	297	53	119
56	Jul 6, '22	This is one of the craziest plots I have ever seen. World GDP follows a power law that holds over many orders of magnitude and extrapolates to infinity (!) by 2047. Clearly this tr	4.4×	0	808	93	214
57	Oct 21, '23	@alexeyguzey @SimonLermenAI We'll have some evidence to share soon	4.3×	85k	185	8	41
58	Aug 24, '22	How will we solve the alignment problem for AGI? I've been working on this question for almost 10 years now. Our current path is very promising: 1/	4.0×	0	576	87	314
59	Jan 27, '22	Extremely exciting alignment research milestone: Using reinforcement learning from human feedback, we've trained GPT-3 to be much better at following human intentions.	4.0×	0	854	128	141
60	Jul 17, '24	Another Superalignment paper from my time at OpenAI: We train large models to write solutions such that smaller models can better check them. This makes them easier to check for hu	4.0×	79k	662	82	268
61	Feb 6, '25	Why are we working on jailbreaking robustness? 🧵👇	3.8×	76k	241	12	70
62	Jul 15, '25	If you don't train your CoTs to look nice, you could get some safety from monitoring them. This seems good to do! But I'm skeptical this will work reliably enough to be load-bearin	3.8×	76k	310	15	104
63	Aug 6, '24	@johnschulman2 Very excited to be working together again!	3.7×	74k	823	8	27
64	Sep 29, '25	Sonnet 4.5 is out! It’s the most aligned frontier model yet; a lot of progress relative to Sonnet 4 and Opus 4.1!	3.6×	71k	637	28	55
65	Aug 8, '23	Great conversation with @robertwiblin on how alignment is one of the most interesting ML problems, what the Superalignment Team is working on, what roles we're hiring for, what's n	3.4×	69k	221	37	75
66	Dec 2, '24	Apply to join the Anthropic Fellows Program! This is an exceptional opportunity to join AI safety research, collaborating with leading researchers on one of the world's most pressi	3.2×	64k	632	83	378
67	Aug 25, '23	Jailbreaking LLMs through input images might end up being a nasty problem. It's likely much harder to defend against than text jailbreaks because it's a continuous space. Despite a	3.1×	62k	317	37	63
68	Mar 13, '25	Could we spot a misaligned model in the wild? To find out, we trained a model with hidden misalignments and asked other researchers to uncover them in a blind experiment. 3/4 teams	2.7×	55k	435	40	98
69	Aug 14, '25	If you want to get into alignment research, imo this is one of the best ways to do it. Some previous fellows did some of the most interesting research I've seen this year and >2	2.5×	50k	339	17	139
70	Feb 5, '25	@elder_plinius We don't want to open source the datasets but we might provide a different incentive. Stay tuned	2.4×	48k	212	6	11
71	Mar 14, '23	GPT-4 is safer and more aligned than any other OpenAI has deployed before. Yet it's not perfect. There is still a lot to do to improve safety and we're planning to make updates ove	2.3×	46k	272	18	9
72	Sep 22, '22	This is the most important plot of alignment lore: Whenever you optimize a proxy, you make progress on your true objective for a while. At some point you start overoptimizing and d	2.2×	0	456	58	77
73	Feb 3, '25	@elder_plinius @AnthropicAI you will have fully broken our defense ✨	2.1×	42k	310	2	9
74	May 3, '23	It's been heartening to see so many more people lately starting to take existential risk from AI seriously and speaking up about it. It's a first step towards solving the problem.	2.1×	41k	267	21	10
75	Feb 5, '25	👀	2.1×	41k	220	8	31
76	Nov 28, '22	Check out OpenAI's new text-davinci-003! Same underlying model as text-davinci-002 but more aligned. Would love to hear feedback about it!	1.9×	0	447	45	30
77	Apr 10, '24	The superalignment fast grants are now decided! We got a ton of really strong applications, so unfortunately we had to say no to many we're very excited about. There is still so	⚡ 1.7×	417k	352	18	93
78	Feb 17, '21	We're hiring research engineers for alignment work at @OpenAI! If you're excited about finetuning gpt3-sized language models to be better at following human intentions, then this i	1.6×	0	341	72	42
79	Dec 10, '22	Web4 is when the internet you're browsing is just sampled from a language model	1.6×	0	437	30	16
80	Jan 22, '21	Last week I joined @OpenAI to lead their alignment effort. Very exicited to be part of the team!	1.5×	0	466	12	16
81	Apr 3, '25	Somewhat surprising that faithfulness of chain-of-thought doesn't improve much with outcome-based RL	1.5×	30k	333	30	74
82	Dec 20, '24	@bobmcgrewai Idk they could have named it "o1 (new)"	1.5×	30k	473	8	9
83	Feb 3, '25	@elder_plinius @AnthropicAI We released the paper with the details on how it works so anyone can recreate this system. I don't think we can publicly release the dataset because it'	1.5×	29k	242	3	24
84	Jun 18, '22	Really looking forward to working with the legendary Scott Aaronson!	1.4×	0	329	31	42
85	Dec 3, '22	I fondly remember the days when people were arguing intensely whether AI is bee level or rat level.	1.4×	0	395	19	17
86	Aug 9, '24	I don't think the comparison between RLHF and RL on go really make sense this way. You don’t need RLHF to train AI to play go because there is a highly reliable procedural reward f	1.4×	28k	383	24	165
87	Feb 3, '25	@elder_plinius @AnthropicAI do all 8 levels with one jailbreak	⚡ 1.4×	516k	391	3	37
88	Jun 13, '22	Super exciting new research milestone on alignment: We trained language models to assist human feedback! Our models help humans find 50% more flaws in summaries than they would hav	1.3×	0	259	47	49
89	Sep 19, '25	They plan to use the highly successful playbook from the pro-crypto super PAC Fairshake. Here is how it works: Instead of running campaign ads on AI directly (most voters don’t car	1.2×	25k	240	10	37
90	Jul 24, '25	In March we published a paper on alignment audits: teams of humans were tasked to find the problems in model we trained to be misaligned. Now we have agents that can do it automati	1.2×	25k	227	27	55
91	Dec 5, '22	New blog post on why I'm excited about OpenAI's approach to alignment, including some responses to common objections:	1.2×	0	206	23	85
92	Dec 28, '24	Not what I signed up for when I joined OpenAI. The nonprofit needs to uphold the OpenAI mission!	1.1×	22k	308	10	8
93	Feb 13, '25	@caleb_parikh They sent 7,867 messages, and passed 1,408 of them onto the auto-grader. We estimate that they probably spent over 40 hours on this in total.	0.9×	18k	188	1	12
94	Jun 25, '20	Today was my last day at @DeepMind. It's been an amazing journey; I've learned so many things and got to work with so many amazing people! Excited for what comes next!	0.9×	0	267	10	4
95	Feb 4, '25	@theojaffee he didn't break the defense, he just hacked the UI	0.9×	17k	262	2	4
96	Nov 20, '18	The agent alignment problem may be one of the biggest obstacles for using ML to improve people’s lives. Today I’m very excited to share a research direction for how we’ll aim to so	0.9×	0	190	32	20
97	Aug 28, '22	Every organization attempting to build AGI should be transparent about their alignment plans.	0.7×	0	192	16	7
98	Oct 23, '24	@jachiam0 How about a friendly game of who-can-make-their-models-more-aligned followed by a jailbreaking competition and a face-off eliciting dangerous capabilities from each other	0.6×	12k	202	5	9
99	Dec 28, '24	Why not fund initiatives that help ensure AGI is beneficial, like AI governance initiatives, safety and alignment research, and easing impacts on the labor market?	0.6×	11k	227	6	8
100	Oct 29, '23	@ml_anon22 True, but you can remember them using this picture	0.5×	9.3k	212	9	31
101	Feb 4, '25	@BenPielstick @theojaffee hacking the UI doesn't let you extract dangerous knowledge from the LLM, which is what we're trying to defend against here	0.4×	8.7k	210	2	4

Scan another creator

Run this on any account.

XOutlier builds this table for every public profile you open on x.com, with live data and every post, not just the top 50.

Add to Chrome Download .zip