Lilian Weng

@lilianweng302k followers

MTS @OpenAI; Author of Lil'Log

Open on X

Top outlier

124×

Best organic post

Peak views

5.2M

Single post

Bookmarks

29k

Across 99 organic posts

Last scan

Oct 16

Community cache

Posts 99 posts

#	Date	Post	↓ Outlier	Views	Likes	Reposts	Bookmarks
1	Sep 26, '23	Just had a quite emotional, personal conversation w/ ChatGPT in voice mode, talking about stress, work-life balance. Interestingly I felt heard & warm. Never tried therapy befo	124×	5.2M	2.8k	207	1k
2	Nov 20, '23	About 650 / 770 signed at this moment. As people start waking up, more will come. All the efforts started after 1:30 AM, 500+ within two hours and all of this after 2 crazy days wi	61.9×	2.6M	4.8k	534	452
3	Nov 8, '24	After working at OpenAI for almost 7 years, I decide to leave. I learned so much and now I'm ready for a reset and something new. Here is the note I just shared with the team. 🩵	23.1×	975k	6.3k	326	1.5k
4	Jul 7	new post on harness engineering for AI self-improvement: It is hard to forecast how much the future of RSI will rely on harnesses. Likely harness engineering will evolve in the dir	21.6×	913k	5.5k	795	6.6k
5	Nov 11, '19	😊Self-supervised learning opens up a huge opportunity for better utilizing unlabelled data while learning in a supervised learning manner. My latest post covers many interesting i	17.5×	0	1.6k	423	383
6	Apr 9, '20	Since the paper "Attention Is All You Need", so many new things have happened to improve the Transformer model, e.g. to make the attention span longer, to reduce memory & compu	16.5×	0	1.6k	356	385
7	Jun 9, '20	Exploration strategies in deep RL are such a critical topic. I almost immediately regretted it when I started writing on this big subject because it has so much more content than I	15.5×	0	1.5k	312	335
8	Jul 12, '21	Diffusion models are another type of generative models, besides GAN, VAE, and flow models. The idea is quite smart and clean. It is flexible enough to model any complex distributio	14.9×	0	1.4k	262	422
9	Jun 27, '23	Agent = LLM + memory + planning skills + tool use This is probably just a start of a new era :)	14.5×	612k	3.5k	739	2.4k
10	Jun 4, '21	Contrastive learning aims to learn representation such that similar samples stay close, while dissimilar ones are far apart. It can be applied to supervised / unsupervised data and	14.1×	0	1.3k	270	363
11	Mar 22, '19	If you are, like me, often ponder over the question of why DNN with so many parameters can generalize without severe overfitting, check this out:	13.6×	0	1.3k	364	213
12	Sep 12, '22	🧮 I finally spent some time learning what exactly Neural Tangent Kernel (NTK) is and went through some mathematical proof. Hopefully after reading this, you will not feel all the	11.7×	0	1k	170	408
13	Oct 15, '19	It has been a long journey for us. There were moments when I felt disappointed or almost hopeless, but the progress we have made, together as a team, is credible. We made it throug	11.6×	0	1.6k	241	33
14	Nov 10, '23	Feeling a bit intimidating to write about it but work on attacks can lead to good insights for mitigation. Plan to write about mitigation work separately later. Also want to thank	10×	424k	1.2k	197	568
15	May 17, '25	Giving your models more time to think before prediction, like via smart decoding, chain-of-thoughts reasoning, latent thoughts, etc, turns out to be quite effective for unblocking	9.6×	407k	3.6k	479	2.5k
16	Sep 26, '21	Training large models demands a lot of GPU memory and a long training time. With several training parallelism strategies and a variety of memory saving designs, it is possible to t	9.2×	0	863	158	258
17	Aug 31, '22	Updated this 1-year old post on diffusion models with some new content based on recent progresses - including classifier-free guidance, GLIDE, unCLIP, Imagen and latent diffusion m	9.1×	0	947	116	252
18	Jul 7, '23	(1/3) Alongside Superalignment team, my team is working on the practical side of alignment: Building systems to enable safe AI deployment. We are looking for strong research engine	9.1×	383k	397	48	75
19	Mar 9, '20	As a person who is working in Robotics but has no academic training in Robotics, I found this "Modern Robotics" course is super interesting: You also need the book:	9.0×	0	845	136	267
20	Nov 19, '23	❤️	8.5×	360k	993	21	42
21	Jan 11, '23	🦖Large Transformers are powerful but expensive to train & use. The extremely high inference cost is a big bottleneck for adopting them for solving real-world tasks at scale. C	7.8×	330k	1.3k	250	541
22	Feb 25, '21	I've read so many papers with small incremental changes that can be well summarized in one sentence. I wish there is a better way to share incremental improvements & correspond	7.4×	0	1k	69	50
23	Jan 30, '20	Humans learn from curriculum since birth. We can learn complicated math problems because we have accumulated enough prior knowledge. This could be true for training a ML/RL model a	7.3×	0	743	149	151
24	Aug 7, '20	Although popular and successful model architectures are mostly designed by human experts, it doesn't mean we have settled down with the best option. Neural Architecture Search (NAS	7.1×	0	679	153	152
25	Feb 7, '21	If you are interested in developing innovative solutions for real-world machine learning problems and deploying cutting-edge deep learning techniques via our API product to benefit	6.8×	0	717	132	131
26	Dec 2, '24	🦃 At the end of Thanksgiving holidays, I finally finished the piece on reward hacking. Not an easy one to write, phew. Reward hacking occurs when an RL agent exploits flaws in the	6.7×	284k	1.9k	222	1k
27	Mar 19, '23	🛠 New posts on Prompt Engineering: Steer a large pretrained language model to do what you want wo/ updating the model weights. Most importantly this just introduces general ideas,	6.6×	280k	1.6k	336	947
28	Nov 19, '23	🤷‍♀️	6.6×	278k	510	11	26
29	Feb 18, '25	This is something we have been cooking together for a few months and I'm very excited to announce it today. Thinking Machines Lab is my next adventure and I'm feeling very proud an	6.3×	267k	1.4k	64	232
30	Aug 22, '18	If you are interested in learning RL, especially Deep RL, but not sure where to start, check out this post I wrote earlier this year:	5.6×	0	526	172	83
31	Sep 6, '19	Gradient descent is not the only option when optimizing model parameters. Evolution strategies can help too. Check out my new post if you are interested in how CMA-ES works or the	5.6×	0	586	133	92
32	Oct 14, '24	📢 We are hiring Research Scientists and Engineers for safety research at @OpenAI, ranging from safe model behavior training, adversarial robustness, AI in healthcare, frontier ris	5.6×	237k	751	67	420
33	Feb 3, '19	My new post on language models examined how word embedding evolved from context-agnostic to context-dependent, as well as the new trend in large unsupervised pre-trained language m	5.5×	0	551	147	85
34	Nov 20, '23	OpenAI is nothing without its people	5.5×	233k	1.3k	60	29
35	Jun 26, '19	Meta RL is a great idea 💡: After trained over a distribution of tasks, an agent is able to solve a new task by developing a new RL algorithm with its internal dynamics. Check my l	5.4×	0	599	114	81
36	Feb 22, '22	Part 2 of “what if you don’t have enough training data” series on active learning. When the labeling budget is limited or labeling cost is very high, active learning comes handy to	5.3×	0	443	77	189
37	Jan 3, '21	The first post to start 2021💡: How to steer a powerful unconditioned language model to output what we want? It is still a challenging open research question. There are some ways a	5.2×	0	549	94	112
38	Oct 1, '25	GPUs are expensive and setting up the infrastructure to make GPUs work for you properly is complex, making experimentation on cutting-edge models challenging for researchers and ML	5.2×	220k	2.1k	128	713
39	Dec 6, '21	The performance of supervised learning tasks improves with more high-quality labels. However it is expensive to collect many such labels. Semi-supervised learning is one of the par	5.1×	0	434	89	166
40	Apr 18, '22	Part 3 of “what if you don’t have enough training data” series - touch base on creating more synthetic data by data augmentation or model generation, as well as some ideas on how t	5.0×	0	413	59	199
41	Jul 10, '24	Wrote about extrinsic hallucinations during the July 4th break. Here is what ChatGPT suggested as a fun tweet for the blog: 🚀 Dive into the wild world of AI hallucinations! 🤖 Dis	4.9×	206k	1.1k	194	614
42	Jun 14, '22	My new post looks into various methods on how to extend a pre-trained foundation language model to be capable of consuming visual signals; in other words, transform a pretrained LM	4.7×	0	456	84	122
43	Oct 14, '18	Here is my new blog post on flow-based deep generative models. Different from GAN or VAE, these model explicitly learn the probability density function of the real data using norma	4.5×	0	472	131	48
44	Sep 10, '25	Besides the fun fact that Connectionism is connected with the early days of the AI field and highlights similarities between neural networks and human brains, the flagship product	4.5×	189k	1.3k	70	402
45	Sep 26, '23	How people interact with AI models differs. Statements are just my personal take. 🙏	4.3×	181k	304	4	9
46	Oct 27, '25	On-policy distillation provides an elegant way to use the teacher model as a process reward model to provide dense reward while preventing SFT style "OOD shock" during rollout.	4.0×	171k	780	51	302
47	Apr 7, '23	Coding calms me down when I’m depressed or anxious, so coding with copilot is like doing a therapy.	3.8×	160k	985	55	63
48	May 14, '24	I’ve started using the similar function during my Japan trip like translating my conversation with a sushi chef or teaching different types of rocks in a souvenir store. The utilit	3.7×	157k	497	35	68
49	Nov 22, '23	seriously considering writing about corporate governance structure in my next blog post.	3.6×	151k	518	10	49
50	Jul 24, '24	Rule-based rewards (RBRs) use model to provide RL signals based on a set of safety rubrics, making it easier to adapt to changing safety policies wo/ heavy dependency on human data	3.4×	145k	302	42	119
51	Nov 28, '23	We have various teams working on AI safety at OpenAI. Let us know if you are interested!	3.4×	145k	314	20	59
52	Sep 26, '25	Looking through those little hidden gem stories in the footnote, you will find it so inspiring that researchers with interests on the same topic are able to work together to advanc	3.2×	137k	768	41	200
53	Sep 25, '24	🩵🩵🩵	3.2×	135k	329	8	37
54	May 25, '25	Probably the first product Thinky will build is a full panel of dials that researchers can use to physically adjust all the hparams during training. We gonna do hardware one day an	3.2×	135k	488	18	135
55	Nov 20, '23	💛	3.2×	135k	301	13	8
56	Mar 23, '21	Toxicity prevents us from safely deploying powerful pretrained language models for real-world applications. Check out for some work on toxic content detection and model detoxificat	3.2×	0	322	64	64
57	Dec 14, '21	You can fine-tune a GPT model with your own dataset on our API now. It opens up all the new possibilities ;)	3.1×	0	364	40	57
58	Jan 14, '21	Our paper on training a single goal-conditioned policy 100% with asymmetric self-play to generalize to many unseen objects and tasks: and more cool videos are available at (The att	3.0×	0	341	57	47
59	Apr 15, '24	🎨Spent some time refactoring the 2021 post on diffusion model with new content: ⬇️ ⬇️ ⬇️ 🎬Then another short piece on diffusion video models: (Yes, I had an intensive weekend🥹)	3.0×	126k	1k	163	518
60	Feb 1, '23	🚜 Cannot believe it is almost 3 years since my 2020 post on variations of Transformer. I spent some time and did a big refactoring of that old post with new section structure and	2.9×	123k	729	108	288
61	Dec 7, '21	In case you not knowing this already, OpenAI API () is open for immediate access now! No wait list any more! 🥳 🥳 🥳	2.9×	0	323	53	48
62	Nov 22, '23	Rebirth	2.9×	121k	586	11	14
63	Oct 27, '23	Preparedness team, led by @aleks_madry, will focus on evaluation of and protection for catastrophic risks that might be triggered by AGI-level capability, including cybersecurity,	2.8×	116k	152	19	26
64	Jul 13, '25	I still find it mysterious whether and how intelligence and capabilities transfer between domains and skills - from meta learning during early days to more recent question like whe	2.7×	115k	770	41	463
65	Jan 15, '24	Very interesting read. If we apply similar idea to build in a safe mode trigger, it can probably stay robust even after custom fine-tuning.	2.5×	104k	383	36	141
66	Jun 9, '22	Check out this post with @gdb, if you are curious about how to train large deep learning models and you may find it is easier than you expected :) Also we are hiring!	2.5×	0	240	27	74
67	Apr 25, '24	More work coming up & we are hiring:	2.4×	101k	264	14	97
68	May 9, '19	One of the hardest problems in robotics is that models trained in simulator normally do not work on real robots. Domain randomization is a simple but powerful idea to close this si	2.3×	0	262	60	22
69	Jul 15, '25	Yes - 🥳 Thinky starts hiring again:	2.3×	97k	571	23	179
70	Jul 13, '19	Einstein Field Equations - for beginners! —- such a brilliant video, cannot stop watching.	2.3×	0	241	37	52
71	Aug 18, '25	Welcome @_kevinlu to the team! Exciting to work together again.	2.2×	94k	303	5	27
72	Nov 3, '20	A system capable of answering any factual questions can enable many useful applications. My new post delves into how we can build an open-domain question answering model, with neur	2.2×	0	238	40	42
73	May 9, '19	Two most interesting papers I’ve found recently: “the lottery ticket hypothesis” (probably already very famous) and “adversarial examples are not bugs but features”	2.2×	0	247	43	35
74	May 12, '25	When a new dataset comes out, I get excited and check it out and then only realize that this is another meta-mixed dataset combining a collections of other existing datasets. My br	2.1×	89k	608	34	104
75	Apr 29, '25	Nope what’s that?	2.1×	87k	239	9	19
76	Dec 7, '21	Together with @_jongwook_kim we will present a tutorial on self-supervised learning. See you soon 🙌	2.1×	0	230	20	54
77	Feb 2, '22	Procrastinating on the next blog post...ended up with hacking a emoji search tool, totally not optimized for perf/latency though: As an emoji lover, I hope you find it as fun I do	2.0×	0	236	25	31
78	Sep 12, '24	🍓 Finally o1 is out - our first model with general reasoning capabilities. Not only it achieves impressive results on hard, scientific tasks, but also it gets significantly improv	2.0×	86k	555	51	108
79	Dec 17, '20	We are glad to share a rich collection of Mujoco simulation environments for training robotic tasks, including the hand for solving Rubik's cube and a variety of object manipulatio	2.0×	0	241	30	31
80	Jan 9, '19	This is the Part 4 of my "Object Detection for Dummies" series, focusing on fast detection models. Well, I explicitly dropped "for dummies" in the name. For whoever reads up to par	1.9×	0	203	44	35
81	Feb 9, '24	🗣️I've been thinking about data quality & human factor in the process a lot lately, so write a short post on the topic: More: If you are into the topic, my team is hiring Rese	1.9×	81k	746	101	397
82	Nov 7, '22	Looking for suggestions on my next post topic. A bit stuck right now 🤖🕵️🧚‍♀️👂	1.9×	0	190	6	13
83	Mar 9, '24	We are lucky to have you Mira and we are with you 💙	1.9×	80k	125	4	12
84	Nov 22, '23	🧡	1.9×	79k	127	4	9
85	Dec 15, '22	40 cents per million tokens + cutting edge performance. Why not give our embeddings a try? 😌🐳	1.8×	75k	214	11	23
86	Jan 25, '22	So proud of the team! This new series of embedding models have amazing performance on clustering and search tasks. And they are accessible via OpenAI API.	1.7×	0	225	16	27
87	Apr 14, '25	See you at #ICLR2025 soon. Excited about chatting with many of you about Thinking Machines and what we have been up to!	1.7×	73k	242	10	28
88	Apr 5, '25	👩‍🍳Actively cooking the next blog post. Tiny teaser: It is spiritually related to our new company.	1.6×	70k	607	14	61
89	Dec 24, '23	Finally finished the book “The Righteous Minds” on Xmas day. An old one but classic. Moral systems feel magic since they suppress self-interest to make cooperative societies possib	1.6×	68k	244	7	83
90	Dec 7, '20	Given that the author has decided to withdraw the paper and the institute has started investigation, I decided to delete my last tweet but still want to emphasize: "We all know how	1.5×	0	254	6	5
91	Aug 10, '20	In special times like now, I find it easy to overwork and not get motivated to take PTO...But I finally did it, no travel, no plan and it was simply amazing. Finished a big paintin	1.5×	0	236	8	6
92	Mar 14, '23	GPT-4 is here! Our best model so far with strong steerability and safety improvement. API waitlist available. A true team effort - with extraordinary talents, strong belief & s	1.5×	61k	306	24	20
93	Jan 22, '19	An interesting read on self-driving cars - - the complexity and engineering efforts involved are amazing.	1.3×	0	148	25	16
94	Mar 15, '22	Now you can do more!	1.2×	0	170	9	16
95	Apr 26, '19	“... suffering is a perfectly natural part of getting a neural network to work ...” - only half way through but I’ve already laughed many times, so true to life. Thanks for reading	1.1×	0	158	15	11
96	Feb 18, '25	If you’re interested in working with us and helping build artificial intelligence that can think, beep and boop, consider applying here: 🩵🩵🩵	1.1×	45k	139	2	30
97	Jun 30, '21	Try it out, my friends! :D	1.0×	0	145	14	9
98	Feb 29, '20	❤️	1.0×	0	180	1	1
99	Apr 1, '21	Just finished the first episode. Very high quality interview and strongly recommend it :) Now stepping into the second episode woohoo	1.0×	0	133	11	13
100	Apr 7, '22	I need to hang this art on my wall. Who can resist those sad puppy eyes with sparks of curiosity 😳	1.0×	0	151	9	6
101	Aug 11, '21	The future is now.	1.0×	0	136	14	6

Scan another creator

Run this on any account.

XOutlier builds this table for every public profile you open on x.com, with live data and every post, not just the top 50.

Add to Chrome Download .zip