You May Also Enjoy
AI and labour
4 minute read
Published:
The rapid advancement of AI technologies will transform industries and labor markets at an unprecedented pace. Despite these anticipated changes, the relationship between AI and labor remains surprisingly understudied. Recent works, notably by Korinek & Suh (2024), Acemoglu (2025), and Epoch AI's GATE model (2025), illustrate the complexity of AI’s economic impacts, but also highlight significant gaps in understanding AI’s real-world implications for labor.
On Knowledge and Substrate
6 minute read
Published:
I’ve recently been thinking a lot about what the intrinsic space of all human knowledge looks like, what kind of topology and structure does the neural latent manifold have, how sparse is it, and how to think about all the space in between pockets of density. For instance, it is not clear to me what the dimensionality of the original space is and whether using tokens as the basic entities of this space even makes sense. Maybe tokens are too granular to be useful for this kind of a thought experiment and we need to think about this at a higher level, say sentences and concepts. The reason such a thought experiment is appealing to me is because I think it lies at the heart of a question I’m interested in - whether AI can discover truly new knowledge.
Enhancing Factual Accuracy in Large Language Models: Integrating Decoding Strategies and Model Steering
20 minute read
Published:
The emergence of open-source Large Language Models (LLMs) like Llama has revolutionized natural language generation (NLG), making advanced conversational AI accessible to a broader audience [1]. Despite their impressive capabilities, these models often grapple with a significant challenge: factual hallucinations. Factual hallucinations occur when an AI model generates content that is unfaithful to the source material or cannot be verified against reliable data [2]. This issue is particularly concerning in critical and information-dense fields such as health, law, finance, and education, where misinformation can have catastrophic consequences [3][4].
Perspectives on the Future of AI
13 minute read
Published:
How big are the models going to get and how much longer is the scaling hypothesis going to hold? It’s unclear, but according to current performance trends, which haven’t shown signs of plateauing (GPT-4o, Claude 3.5 Sonnet, Gemini-1.5-Pro, Llama-3.1-405B, Grok-2), and the power budget of announced data centres (5GW OpenAI/Microsoft Stargate campus), it is likely that there is an order of magnitude left (OOM) to climb in model size. This Epoch AI research covers these scenarios in depth and estimates training runs of the order of ~2e29 FLOPs being possible by 2030, which would be 4 OOMs larger than GPT-4 (2e25 FLOPs). These training runs will primarily be power constrained, followed by chips, data, and latency.