XOutlier
Download .zipAdd to Chrome

Kevin Patrick Murphy

@sirbayes76k followers

Research Scientist at Google DeepMind. Interested in Bayesian Machine Learning.

Open on X
Top outlier
58.2×
Best organic post
Peak views
621k
Single post
Bookmarks
38k
Across 90 organic posts
Last scan
Oct 16
Community cache
Posts 90 posts
#DatePost↓ OutlierViewsLikesRepostsBookmarks
1Feb 16, '24While GenAI is fun, I think its economic value is grossly over estimated, because it’s unreliable, risky and expensive to make and serve. It’s fine for creative tasks, but not (yet58.2×621k1.5k231378
2Aug 15, '23The trilogy is complete! My "Advanced Topics" book is officially released today. Buy it on Amazon, or get it for free at .41.8×446k4.7k7262.1k
3Jan 3, '23I am delighted to announce that the "real" camera-ready version of my new book, "Probabilistic Machine Learning: Advanced Topics", is now available. It will appear in print this su30.8×328k2.1k410784
4Dec 9, '24I am happy to announce that the first draft of my RL tutorial is now available.30.1×321k4.4k7134k
5Jun 16, '23I like this paper. They prove that transformers are guaranteed to suffer from compounding errors when doing long reasoning chains (as @ylecun has argued), and much apparent "succes25.9×277k1.3k209781
6Feb 28, '22I am delighted to announce that a draft of my latest book, “Probabilistic Machine Learning: Advanced Topics”, is now available online at . It covers #DeepGenerativeModels, #Bayesia25.6×04.6k9551.2k
7Jul 29, '22I am delighted to announce that the camera-ready version of my new book, "Machine Learning: Advanced Topics", is finally available online for free at (@mitpress will publish the ha22.9×03.6k8051.4k
8Jan 29, '25Wait till they start reasoning in continuous latent space (why restrict the model to human legible discrete tokens? )21.4×228k1.1k64388
9Aug 30, '21I am pleased to announce that the camera ready version of my new textbook, "Probabilistic Machine Learning: An Introduction", is finally available from . Hardcopies will be availab19.7×03.5k734917
10Nov 5, '23I cover Kalman filtering in ch 8 of my book . We also have Jax code and show how to use it to fit a DNN to streaming data at18.5×197k65782516
11Mar 6, '22I just got a hardcopy of my own book - it finally feels real :) I'm very happy with the print quality (see attached photo) - but you can still get the pdf for free from if you pref18.1×03.8k435794
12Sep 14, '23Planning with LLMs fails if you change the variable names to be 'meaningless'. Same thing for causal inference (see ). LLMs are just retrieving pieces of the correct computational 16.5×176k28246234
13Dec 29, '23How can LLMs memorize text chunks exactly (eg from NYT) if they only do one pass SGD training with small learning rate? Does it require multiple copies of the same article to be on15.7×167k43111142
14Oct 3, '25I agree with @karpathy 's take here. The interview between @RichardSSutton and @dwarkesh_sp was interesting, but I think at times there was a communication gap due to some misunder15.5×165k76591806
15Feb 15, '22I am delighted to announce that my new book, “Probabilistic Machine Learning: An Introduction”, is finally available in print format! You can order it from , or from Amazon. Also a15×02.9k453663
16Feb 11, '23Great article on LLMs - MLE training is exactly lossy data compression! I know RLHF is changing the loss to maximize predicted human ratings, but it still feels like a bit like lip14.6×156k35349122
17Aug 26, '25I just finished reading this interesting book by @druv_pai, @_sdbuchanan and colleagues. It's fairly "heavy" but provides a very satisfying theoretical explanation for many differe13.6×145k778901k
18Nov 23, '24When did the deep learning crowd first start callling prediction (i.e., forward pass) "inference"? And what should we now call traditional inference (i.e., *inverting* a forwards p12.6×135k74350251
20Oct 22, '25Hi @karpathy . I loved your interview! However, you said there is no work on LLM self-play. Not true. See eg "Spiral" from @natashajaques et al (agent-v-agent) and "Absolute Zero R11.9×127k44112257
21Nov 28, '23Wow! Prof Ali Sayed at EPFL has just published a 3 volume series (totalling 3000 pages) on ML/ signal processing/ data science. It's a bit like my book in terms of topics, but with11.9×126k723101713
22May 20, '25I am pleased to announce a new version of my RL tutorial. Major update to the LLM chapter (eg DPO, GRPO, thinking), minor updates to the MARL and MBRL chapters and various sections11.5×123k2.4k4312.3k
23Mar 25, '25I'm happy to announce that v2 of my RL tutorial is now online. I added a new chapter on multi-agent RL, and improved the sections on 'RL as inference' and 'RL+LLMs' (although latte10.8×115k1.8k2811.8k
24Feb 16, '24Sora from OpenAI and Gemini 1.5 from Google are both super impressive. Both rely on long context transformers. I think the lesson is that semi parametric models are great - conditi10.5×111k2632068
25Feb 3, '24Yann is advocating Model predictive control in a latent space , which is learned without a reconstruction loss, as a way to solve planning, and get truly controllable behavior. I a10.4×111k36238299
26Apr 14, '23Good paper. training massive models on massive datasets using massive compute is so inefficient, we need new and better ideas… Paucity of compute in academia may fuel more methodol9.9×106k41058211
27Jan 25, '24To all who like this - what if I told you this book is not about the linear deterministic kind of SSM used by Mamba, S4, etc. but instead is about Bayesian inference in stochastic 9.9×105k51834419
28Nov 20, '23There is a lot of excitement about AI and LLMs at Google DeepMind, but at least they don't make us chant "Feel the AGI" at company parties... (Ilya always was a bit mystical about 9.8×105k2071443
29Jan 26, '25Is it feasible to do a true tabula rasa version of deepseek R1 zero, starting from an LLM with random weights, similar to alpha zero? Or is starting with an LLM which is pre traine9.7×104k2351497
31Feb 18, '24I’m not sure that maximizing likelihood of pixels is enough to force the model to reliably learn precise physics, as opposed to plausible dynamic visual texture - there is no press9.6×102k2071077
32Mar 29, '24Stanley's probability textbook () is fantastic, so I imagine this tutorial on diffusion is good as well. However, I am no longer of a fan of presenting the discrete time version - 9.4×100k60695751
33Jun 9, '24I read this on the plane from London. It’s probably the best (non fiction) book I have ever read! A full stack approach to intelligent agents, from DNA to brains to society, from R9.2×99k72662799
34Sep 12, '23Here's my theory of why LLMs work so well at predicting stuff. 1. The world is a set of complex hierarchical generative model that gives us (xi, yi) pairs, where xi is an input con9.1×97k40678498
35Mar 14, '24It’s interesting that the math behind diffusion and flow models is so rich and deep - ODEs, SDEs, OT, VI, etc - but autoregresive transformers seems to be so shallow - just chain r8.8×94k51236232
36Sep 16, '25I'm pleased to announce another (fairly minor) update to my RL tutorial (I fixed some typos, cited more papers, and added a wee bit more stuff on multi-agent RL and RL for LLMs). I8.7×93k1.5k1931.7k
37Apr 17, '23I am happy to announce that MIT Press will soon make a second printing of my book "Probabilistic ML: An Intro". I have uploaded the latest version to . This fixes over 100 github i8.7×93k57965180
38Aug 30, '21A draft of the sequel, "Probabilistic Machine Learning: Advanced Topics", will be released by late Fall 2021. Together these two new books amount to about ~2000 pages of content. S8.0×01.7k280279
39Jul 27, '25Finally, a good modern book on causality for ML: by @eliasbareinboim. This looks like a worthy successor to the ground breaking book by @yudapearl which I read in grad school. (h/t7.6×81k1k1541.2k
40Dec 26, '24He sounds like every Tiger Mom / Dad type that I’ve seen in in Palo Alto :) “More math tutoring, fewer sleepovers. More weekend science competitions, fewer Saturday morning cartoon7.5×80k3161280
41Oct 8, '25I am pleased to announce our new paper, which provides an extremely sample-efficient way to create an agent that can perform well in multi-agent, partially-observed, symbolic envir7.1×76k80496737
42Jun 22, '24This is obviously true if you think for just a minute about animal intelligence and evolution. Language enables cultural ratchet of human society, but builds on embodied cognition.7.1×76k2133276
43Dec 22, '22I am pleased to announce the release of , which is a library for structural time series forecasting in JAX, built on top of dynamax. Thanks to @mxinglongli for doing most of the he7.0×75k49061148
44Feb 13, '24I’ve never understood why modern nonconvex line search did not become more popular, since it’s clearly hard to find a single step size which is good throughout the loss landscape, 6.9×73k25217129
45Apr 18, '25I am pleased to announce that I have updated the online versions of my 2 textbooks (see ): I fixed all issues listed on github, added some new references (esp on LLMs), and made a 6.4×68k1.3k167836
46Nov 6, '24Looks like I’m 3 for 3 .6.3×67k2541348
47Dec 11, '23If you are attending NeurIPS this year, please come to the #MITPress booth (number 10) to sign up for a free signed copy of my books (volumes 1 and 2). Raffle will be Thursday 9.306.2×66k62839144
48Oct 9, '24Yes! If you want to know more about Hopfield networks and Boltzmann machines (Hinton's extension of Hopfield), and how they relate to Ising models, PGMs, and EBMs, see sec 4.3 of m6.2×66k47170286
49Oct 20, '24This looks like an excerpt from my book (p253). I intentionally assigned numbers to every equation so people can refer to them (eg student: "what is the const term in 5.170?" or gi6.1×65k46015102
50Nov 19, '24The achiles heel of Bayes is that it is only optimal if the model is the true DGP, which it never is. (That, plus all the computational issues). Generalized Bayes to the rescue! (A5.8×62k31232244
51Oct 28, '25I am pleased to share our new NeurIPS paper for online Bayesian inference for neural networks. Instead of focusing on updating the parameter posterior, we work with the predictive 5.8×62k52341326
52Jan 28, '25I read the R1 zero paper and the method is very simple , just a tweak to PPO to fine tune deepseek v3 base using a verifiable sparse binary reward. The fact that they got it to wor5.6×60k44738239
53May 7, '24I spent the day playing with @LightningAI studio, to speed up my NeurIPS experiments, and it is totally awesome! It’s just like developing in VSCode on your laptop, except it runs 5.6×60k30921158
54May 6, '22I am pleased to announce some significant improvements to the code that accompanies my book, "Probabilistic Machine Learning: An Introduction". 1/55.6×01.1k104323
55Mar 5, '24I think Apple is playing the long game here to actually profit from AGI - people will buy more iPhones so they can fine tune models on their private data locally on device - Person5.3×56k2131767
56Sep 22, '23I find claims that AGI will "solve" poverty, climate change, cancer etc to be so naive, because most of these problems have major non-technical bottlenecks. This blog post is an aw5.2×56k37047213
57Apr 13, '24Totally agree. Unfortunately for me, the golden hours, where Deepmind work hours in California overlap with work hours in Europe, are during my morning, so my day usually starts wi4.9×52k34426122
58Jan 19, '24This is a good visualization of the (obscene) size of the data and compute for LLMs. This feels like a very inefficient information extraction machine. Scale is not enough…4.5×48k33035108
59Jun 15, '22Some random musings on how the whole "AI scaling" debate. It is clear that neural nets can model any predictive distribution p(x_future|x_past) given enough parameters and data, si4.3×0805130208
60May 10, '24I am happy to share our new ICML paper on WOLF, the "Weighted Observation Likelihood Filter", which is a simple way to robustify many existing Bayesian filtering techniques (such a4.3×45k22521140
61Nov 16, '25This is a much more intuitive version of the prisoners dilemma from Jon Kleinberg’s excellent book, , that covers the holy trifecta that Michael Jordan keeps discussing, namely Eco4.2×45k34941412
62Jan 28, '23“Choosing a distribution is not a statistical question; it is a modeling decision. No statistical test can tell you whether a particular distribution is a good model for your data”4.1×44k2112454
63Feb 10, '22Finally got my hard copy! It was an honor to write the foreword for this very useful book.4.0×085082191
64Jun 27, '24I am pleased to announce that I have (slightly) updated the online version of my textbooks () - I fixed over 75 github issues (mostly trivial typos) filed since August 2023 (thanks3.9×42k755108373
65Feb 13, '25Yes, I'm a big fan of Kalman filtering and fancier non-linear, non-Gaussian SSMs. Fun fact: you can also use this machinery to train neural networks from streaming, non-stationary 3.9×42k52861416
66Aug 16, '23Several people pointed out that I missed an opportunity to use a different digit to distinguish "book 0" (2012). "book 1" (PML: Intro 2022) and "book 2" (PML: Advanced, 2023). I as3.9×42k2881770
67Nov 14, '22I am delighted to announce the release of “dynamax”, an open source library for dynamic state space models in JAX. It supports inference and learning in HMMs, Linear Gaussian SSMs,3.8×0719131166
68Mar 14, '25Gemma 3 is best in class for a VLM that runs on 1 GPU. Should make RL fine tuning feasible. Also Academic researchers can apply for Google Cloud credits (worth $10,000 per award) t3.8×40k30442122
69May 31, '24I am delighted to share our recent paper: . It can be thought of as a version of the Bayesian Learning Rule, extended to the fully online setting. This was a super fun project with3.8×40k33639238
70Oct 18, '24Excited to share our new paper on "Diffusion Model Predictive Control" (D-MPC). Key idea: leverage diffusion models to learn a trajectory-level (not just single-step) world model t3.8×40k54973408
71Oct 8, '22I finally got my copy. Should be a fun read!3.5×067041229
72Nov 30, '24100%. The idea that AI will solve intractable social problems - whether optimizing government or “solving” poverty - is so naive. These are values problems, not technical problems 3.4×36k2823249
73Jul 27, '24This is a very interesting talk about LLMs, doing proper science (controlled experiments with synthetic data, etc). It is great that FAIR encourages this kind of open basic researc3.2×34k26626187
74Nov 18, '24I'm happy to announce our new review paper on methods for online prediction (and segmentation) in the presence of non-stationarity, covering both gradual and sudden changes. Led by3.2×34k34043258
75Sep 6, '21For those that think that all we need to "solve AGI" is to "train bigger models" (eg GPT4?), I recommend this paper on causal inference in / for NLP as an antidote to such naive th3.1×054677190
76Nov 3, '24I am delighted to share our NeurIPS'24 paper "Bayesian Online Natural Gradient" . BONG can be thought of as a version of the Bayesian Learning Rule (BLR), extended to the fully onl2.8×30k35641215
77Mar 7, '25Very cool. Non-amortized variational EM for fitting a latent variable generative model to a small set of examples with a shared hidden common cause is indeed a promising approach f2.8×30k27140196
78Oct 13, '25Good article on AI / data center boom / bubble. The costs will outweigh returns for many years, until we make models much more efficient to train and serve (thinking tokens, I’m lo2.8×30k20914195
80Mar 29, '24I'm delighted to present that our AutoBNN library is out. We show how to define a meaningful prior over functions, based on sums and products of base GP kernels, and convert it to 2.7×29k24431137
81Mar 11, '22Interested in converting the theory of "Probabilistic Machine Learning: Advanced Topics" into beautiful #JAX code? Available to work this summer? Consider applying to #GSoC. Detail2.6×048880129
82Nov 7, '24Check out our NeurIPS paper on planning in (discrete) MDPs as (a new kind of) inference. We extend to factored models using loopy BP. Joint with @lazarox8, @dileeplearning and Li Y2.5×27k29246242
83Apr 4, '22Google Summer of Code applications open today. Apply to work with me on making open source advanced probabilistic ML demos in JAX. Details:2.5×046286116
84Jul 3, '24I guess the Twitter censorship algorithm is Bayesian?2.4×26k21477
86Jun 12, '23UBC refugees reunite at the Google Deepmind office in London! w/ @NandoDF and @ArnaudDoucet12.4×25k23566
87Feb 7, '23My recent talk at UBC+SFU on "The 4 pillars of ML: prediction, decision, discovery, generation" is now available at . (This is slightly more polished than the UBC version at )2.3×25k2033393
88Jul 15, '25Don’t just predict the mean of your clean data given your noisy data, predict the full distribution.2.3×25k25430165
89Apr 3, '25I am pleased to announce that the paper on our Dynamax Jax library for SSMs is now available at . Code is at . Joint work with @scott_linderman @grrddm @petergchang @karalleyna @Gi2.2×23k31449149
90Mar 6, '22BTW, if you are teaching from my 2022 book, and want to know how it differs from my 2012 book, you can look at this handy "transition guide" prepared by @mitpress .2.0×04284599
91Jan 19, '22I'm happy to announce that our paper on online Bayesian inference for "neural bandits" was accepted to AISTATS'22. Paper: . Code: . Thanks to @grrddm and @karalleyna for their hard1.7×03294090
94Jun 25, '22This is a good article. Lay people don’t think generative image models are intelligent, but get easily fooled by the verbal sophistry of generative language models- for evolutionar1.2×02043168
97Jan 16, '22I finally got covid, like so many other people. Luckily my symptoms are mild. My son got it 2 weeks ago but still tests +ve using NAAT test but -ve for antigen. So I made this litt1.0×02291230
98May 20, '22I agree with Yann on this - we need more new ideas, not just scaling current methods, which are woefully data inefficient and fragile.1.0×02462125

Scan another creator

@
@karpathy@levelsio@paulg@naval@marclou@illyism

Run this on any account.

XOutlier builds this table for every public profile you open on x.com, with live data and every post, not just the top 50.

Add to Chrome Download .zip