XOutlier
Download .zipAdd to Chrome

Jack Morris

@jxmnop54k followers

research @engramlab // language models, information theory, science of AI

Open on X
Top outlier
131×
Best organic post
Peak views
5.4M
Single post
Bookmarks
150k
Across 95 organic posts
Last scan
Oct 16
Community cache
Posts 95 posts
#DatePost↓ OutlierViewsLikesRepostsBookmarks
1Aug 19, '25*taps the sign*131×5.4M197k7.1k6k
2Apr 10, '25this guy invented VLLM. he's basically the john wick of CUDA kernels61.8×2.6M14k8865.5k
3Jul 8, '24observations from my first two weeks as a Meta research intern - research jobs are the same everywhere: no one ever asks me what I’m doing or how I’m spending my time; there’s an i53.4×2.2M9.9k1462.9k
4Apr 20, '24yearly reminder everything looks exponential from the middle of a sigmoid33.1×1.4M5.4k384557
5Sep 30, '25i'm about to be so rich33×1.4M18k6661.1k
6Feb 7, '25my understanding is that both AMD and qualcomm make chips that have ~equivalent performance to nvidia but neither can write the software tooling that N provides, like CUDA i get th29.9×1.2M6.7k2712.2k
7Apr 6, '25Recent AI model progress feels mostly like bullshit (2025)28.8×1.2M4.8k2343.1k
8Mar 29, '24what software was this made with? i don't think you can draw arrows that curve like that w/ Google Drawings28×1.2M2.4k572k
9Jan 3, '25no AI here, just the coolest paper i've seen in a while26.7×1.1M24k1.3k7.8k
10Aug 8, '25curious about the training data of OpenAI's new gpt-oss models? i was too. so i generated 10M examples from gpt-oss-20b, ran some analysis, and the results were... pretty bizarre t24.4×1M5.5k4844k
11Nov 18, '23now seems as good a time as ever to remind people that the biggest breakthroughs at OpenAI came from a previously unknown researcher with a bachelors degree from olin college of en23.2×958k5.4k5061.7k
12Aug 13, '25OpenAI hasn’t open-sourced a base model since GPT-2 in 2019. they recently released GPT-OSS, which is reasoning-only... or is it? turns out that underneath the surface, there is st22.5×930k6k4244k
13May 21, '25excited to finally share on arxiv what we've known for a while now: All Embedding Models Learn The Same Thing embeddings from different models are SO similar that we can map betwee22×910k6.2k5824.8k
14Feb 1, '25the first evidence i ever saw of superintelligence was in medicine deep learning can tell male from female eyeballs from just a picture with 70–90% accuracy doctors still can't do 20.4×842k6.2k5072k
15Feb 28, '242022: - oh nooo!!! you can't run language models on cpu! you need an expensive nvidia GPU and special CUDA kernels and– - *one bulgarian alpha chad sits down and writes some c++ co20.3×840k4k2991.9k
16Jan 29, '25another incredible thing about deepseek: all the american AI labs compete to hire the top PhD researchers - but deepseek didn’t compete deepseek researchers aren’t top PhDs. most a19.6×808k5.8k262737
17Aug 3, '25i wonder if this is what it will be like when we get 10-trillion parameter LLMs18.3×757k13k4771.6k
18Nov 11, '25there are dozens or perhaps a couple hundred ex-{OpenAI, xAI, Google DeepMind} researchers founding companies in the current climate there are, as far as i know, zero people leavin17.7×732k2.2k44355
19Jan 9, '24how I got my first job at google > be me > go to college > great state school, smallish CS program > google does not recruit here > that’s ok > no worries > build personal website 15.5×641k3.7k901.2k
20Jun 3, '25new paper from our work at Meta! **GPT-style language models memorize 3.6 bits per param** we compute capacity by measuring total bits memorized, using some theory from Shannon (1915.4×637k3.6k3952.8k
21Jun 10, '24*breathes deeply* ahhhh yes... the boundary of all human knowledge14.9×615k3.4k2461.7k
22May 19, '25heard a funny story about a friend who worked at a French LLM startup a couple yrs ago > their plan. to be the first to market with a certain type of multilingual model > early in 13×536k8.6k207961
23Apr 28, '24one of the most important things I know about deep learning I learned from this paper: "Pretraining Without Attention" this what I found so surprising: these people developed an ar11.8×490k3.1k4013.1k
24Jul 10, '24someone finally made leetcode for machine learning, and it's everything we hoped it would be just solved the first exercise: computing a matrix-vector product without any tensor op11.8×488k5.2k4687.3k
25May 4, '25it's interesting to see the big AI labs (at least OpenAI, anthropic, google, xai?) converge on EXACTLY the same extremely specific list of products: - a multimodal chatbot - with a11.2×463k3.6k97825
26May 6, '25take a year out of your life and read these two textbooks cover-to-cover. you will already know more than 90% of people in AI10.9×451k3.3k2274.7k
27Dec 17, '24watched the Attention Is All You Need re-release on IMAX 70mm.. so epic10.9×449k6k464707
28Nov 4, '25defending today 🥲10.8×447k3.3k102834
29Apr 22, '25the great irony here is that we aren't even close to having the tech required to build this, so their customers are actually the ones being cheated10.6×440k6.1k124324
30Feb 13, '24today i found out that this one australian guy has been toiling away making incredibly detailed Neural Circuit Diagrams with the vibe of a 1950s issue of Popular Mechanics, but con10.5×435k4.8k5684k
31Oct 4, '24We spent a year developing cde-small-v1, the best BERT-sized text embedding model in the world. today, we're releasing the model on HuggingFace, along with the paper on ArXiv. I th10.5×435k3.2k4233.5k
32Feb 15, '25my roadmap to learning LLMs - electrons - circuits - logic - transistors - comp arch - CPUs - GPUs - linear algebra - probability - machine learning - optimization - optimizers - t10.3×427k6.1k4035.4k
33Feb 26, '25people underestimate the mental cost of outsourcing code to Copilot/Cursor it's a mortgage: quick progress now at the expense of not understanding your own codebase it may be that 10.3×425k4.2k215590
34Jul 16, '25s3 does vectors now vector databases are officially dead10.2×422k4.2k2792.2k
35Jul 31, '24the self-driving battle is more interesting than LLMs rn and for some reason people aren’t talking about it cars that depend on LIDAR and hard-coded heuristics (waymo, zoox) can dr10.2×421k3k78492
36Jul 29, '25you can't make this stuff up10×415k5.2k186556
37Nov 3, '25dumb question, how do you cool a datacenter in a vacuum? also H100s are already nearly obsolete oops gotta send a datacenter sysadmin astronaut out there to swap in B200s10×414k2k38160
38Mar 31, '25always remember everything looks exponential from the middle of a sigmoid9.9×410k6.1k284618
39Apr 9, '25new blog post "There Are No New Ideas In AI.... Only New Datasets" in which i summarize LLMs in exactly four breakthroughs and explain why it was really *data* all along that matte9.8×404k2.8k3092.6k
40Jan 22, '25so DeepSeek R1 seems to be an incredible engineering effort. i'm trying to understand the recipe seems it's pretty much "run PPO (GRPO) on the model with verifiable math & coding p9.5×394k2.4k99953
41Oct 29, '25this post is complete misinformation LLMs are lossy compressors! of *training data*. LLMs losslessly compress *prompts*, internally. that’s what this paper shows. source: i am the 9.2×381k4k205868
42Mar 26, '24okay what 99.99% chance there's a bug in my code, but 0.01% chance i just solved text retrieval9.0×373k1.9k33294
43Jan 10, '25it's a baffling fact about deep learning that model distillation works method 1 - train small model M1 on dataset D method 2 (distillation) - train large model L on D - train small9.0×373k3.3k1631.9k
44Aug 2, '24remember the GPT Store? a few months ago there was some excitement around "prompt app stores", where gifted prompt-writers could make money by writing magical system prompts and pa8.9×369k2k1772k
45Jan 19, '25i think the “TikTok algorithm” is probably just an embedding model that matches video embeddings to user embeddings to recommend content this is how most state-of-the-art recommend8.9×368k2.5k70891
46Oct 13, '25underrated second-order effect of Claude Code is the proliferation of beautiful CLIs what tools do people use to build these interfaces? we should make many more of them8.7×358k2.1k63802
47Apr 18, '25heard from a friend of a friend that back in March a google researcher fixed a bug in Gemini training code and the model instantly got several points better on every benchmark that8.7×358k3.4k71394
48Jul 26, '24learned something super interesting this week if you train GPT-2 to multiply, you can't even train it to multiply 4-digit numbers (30% accuracy) but if you use a really clever (and8.1×335k2.4k2011.8k
49Feb 17, '25a short bear case for nvidia > NVIDIA’s main advantage is a software one (CUDA), not a hardware one (AMD has ~equally good GPU hardware) > thanks to the rise of RL, LLM-based reaso8.0×330k2k89903
50Jan 27, '25a second DeepSeek paper has hit the internet7.7×320k5.5k466739
51Aug 29, '24TIL that the volume of the unit sphere goes up until a dimensionality of exactly 5, and then tends towards 0 with infinite dimensions (as a human who experiences the world in three7.5×310k2.6k141995
52Mar 4, '25training on only ten images gets 94% accuracy on MNIST dataset distillation is wild7.4×306k3.5k1541.7k
53Sep 25, '25the largest tech company in europe finally released a “sovereign LLM”. it’s just chatGPT running on german servers meanwhile, Chinese DoorDash pretrains 560B LLMs from scratch7.4×306k5k2661k
54Jun 26, '24recently read one of the most interesting LLM papers i've ever read, the story goes something like this > dutch PhD student/researcher Eline Visser lives on remote island in Indone7.0×291k1.8k1981.1k
55Aug 30, '25my grandmother’s lifetime - color film - talking movies - television - cars - airplanes - plastics - penicillin - transistors - computers - the Internet - the Concorde my lifetime 7.0×289k13k476601
56Oct 28, '25me when i come across an exceptionally well-crafted personal website6.9×284k13k5211.2k
57Jan 27, '25openAI: we will build AGI and use it to rewrite the social contract between computer and man DeepSeek: we will build AGI for 3% the cost. and give it away for free xAI: we have mor6.7×278k6.8k323400
58Nov 4, '25i passed!6.7×277k4.5k62208
59Aug 27, '24i have a dictionary of ~350M key-value integers that i want to save to disk - written as ints to lines of a text file: 3.2 GB - stored as torch.Tensors in a collections.defaultdict6.6×272k1.9k43314
60Jun 1, '25> 2020: gpt2 can't write code > 2021: gpt3 can't reliably write python > 2022: instructgpt can't write blocks of code without syntax errors > 2023: chatgpt can't do lee6.6×271k5.5k186634
61Nov 6, '24still the most compelling Figure 1 i've ever seen - from "Visualizing the Loss Landscape of Neural Nets" (2017)6.5×267k3.6k2451.7k
62Jan 24, '25i guess DeepSeek broke the proverbial four-minute-mile barrier. people used to think this was impossible. and suddenly, RL on language models just works and it reproduces on a smal6.3×260k2.3k1871.1k
63Jun 11, '25observations from three months as a Meta research intern in NYC - research jobs are the same everywhere: no one ever asks me what I’m doing or how I’m spending my time - in particu6.1×254k2.1k38506
64Nov 14, '24shannon invented digital circuit design as a Masters student in 1937. invented information theory ~10 years later a typical phd student in 2024 ends up writing an esoteric thesis a6.1×253k3k118589
65Feb 11, '25the simplest explanation for why NNs work so well6.1×251k1.9k1251k
66Jan 22, '25View post on X5.9×243k4.9k122163
67May 27, '25> be me, first year PhD student > get obsessed with polyphonic music transcription during pandemic > for some reason, advisor lets me work on music > brilliant idea: explicitly enc5.8×238k4.2k158887
68Sep 10, '24holy cow guy who writes (very helpful!) blogs about logistic regression, activation functions, validation sets, et cetera has an h-index of 59 that's higher than almost all the res5.7×237k2k1291.4k
69Mar 30, '25when we finish building a skyscraper, the construction crew leaves to go build another building when we build software, the engineers never leave. they stay, and keep building more5.7×235k3.4k87284
70Mar 24, '25# A new type of information theory this paper is not super well-known but has changed my opinion of how deep learning works more than almost anything else it says that we should me5.7×234k2.7k3483.4k
71Jun 15Zen and the Art of AI Research So you want to do AI research? It's true that no one really teaches you how. Not directly, anyway. But it turns out that the way to get started is pr5.4×222k810911.3k
72Mar 17, '25don't waste your time studying CUDA or calculus. pytorch abstracts away both of these instead, if i were you, i would train hundreds of tiny little models train PixelCNN on MNIST. 5.2×215k2.4k1261.9k
73Jun 14, '25google simply does not get enough credit for the TPU the amount of conviction & foresight it took to fund and build hardware specifically for for AI… in 2015… ten years later, 5.0×208k4k228556
74Dec 15, '24LLMs run on a surprisingly old tech stack: gradient descent ‣ method for finding the minimum of a function ‣ Cauchy, 1847 (177 years ago) next-token prediction ‣ core learning task5.0×207k2.7k2411.8k
75Aug 1, '25i haven't heard it dicussed yet but AI basically killed hackathons. pretty much anything you could possibly make at a hackathon in 2019 can be built better and faster by AI in 20255.0×207k2.6k74239
76Aug 28, '24back to school this week and back to reminding people that one of the coolest things my advisor did was write a course on how to build PyTorch from scratch if you want to become an5.0×206k2.6k1993.7k
77Aug 12, '25figured out how to "undo" the RL and turn gpt-oss back into a base model will drop the weights tomorrow gn4.8×198k1.8k48586
78Mar 17, '25Deep Learning is Not So Mysterious or Different (2025)4.7×195k2k1752.4k
80May 7, '25i was surprised to learn how much of modern AI was developed by physicists: - scaling laws - alphafold - RLHF (dario got his phd in biophysics) - diffusion models - normalizing flo4.4×181k2.5k2161.4k
82Jan 26, '25the right takeaway from DeepSeek supremacy is that our median AI engineer is just not really that skilled most-popular libraries are piles of glued-spaghetti python code. rare to s4.1×170k2.8k157422
83Oct 8, '24BREAKING: The Nobel Prize in Physics has been awarded to ptrblock for “fundamental contributions to physics”3.7×153k2.3k100117
84Jun 12, '25if you want to read a really underrated paper:3.7×151k1.8k1742.5k
85Dec 28, '24the yacine guy becoming popular is strong evidence that the X algorithm rewards *volume* over anything else the app has RLHF’d him and many others into the same loop - spend all da3.4×142k2.1k37229
86Apr 21, '25neural networks are funny because they’re really just 17th century math (leibniz), but run on modern silicon with all the engineering tricks we know3.4×139k2.3k143364
87Nov 20, '24chinese ai researchers are incredibly cracked, and driving a lot of open-source progress > best coding model (Deepseek) > best open-source Multimodal language model (Qwen2-VL3.3×138k2k170942
88May 25, '25the scale of data collection in the AI labs pales in comparison to 2010s google it’s mostly web scraping and data-labeling. compare that to diligently photographing streets of ever3.3×136k2.9k129300
89Sep 8, '25apparently you make $16,000 for a semester of AI research in a PhD program. but $2.5M if you clone doordash in next.js and sell it to an AI lab something isn't adding up...3.1×129k2.3k51282
90Dec 5, '24from 2017–2020 i was learning ML. i didn't publish any research (and hadn't yet); i just trained a lot of tiny models and read a paper every single day i maintained a giant spreads3.0×125k1.8k1251.5k
92Mar 7, '25day in the life of an AI PhD in 2025 > wake up > new research idea (5 minutes) > kick off related work search w/ Deep Research (15 minutes) > set up 4 instances of Clau2.9×121k1.9k90575
93Jun 14, '25View post on X2.9×118k2k207537
94Mar 12, '22apparently deep learning models can tell male from female eyeballs with 87% accuracy but no one knows why: "Clinicians are currently unaware of distinct retinal feature variations 2.8×04.8k635602
95Jul 10, '25i am beginning to suspect that Humanity’s Last Exam may not in fact be humanity’s last exam2.5×103k1.8k5289
96May 9, '25just coded up a model that gets even higher fraud detection rate than stripe's transformer (97->100%). AMA2.5×101k2k4587
98Mar 22, '24navigated to a file to write a helper function, but the code was already there, written by me, exactly where i was about to write it2.0×84k2.8k110110
99Jan 18, '25when you’re a VP at google research you can start your research papers just like spongebob starts his essays1.9×77k2k86375

Scan another creator

@
@karpathy@levelsio@paulg@naval@marclou@illyism

Run this on any account.

XOutlier builds this table for every public profile you open on x.com, with live data and every post, not just the top 50.

Add to Chrome Download .zip