Uncover Sakana AI's Continuous Thought Machine (CTM), a groundbreaking new AI model inspired by biological neural networks. Unlike traditional artificial neural networks, the CTM uses precise timing information at the neuron level and the synchronization between neurons as its core reasoning mechanism. This innovation enables the model to "think" through problems step-by-step, making its reasoning process interpretable and human-like. Witness its enhanced problem-solving capabilities and efficiency across various tasks, such as tracing paths through mazes and analyzing images with human-like attention patterns. The CTM represents a meaningful step toward bridging the gap between artificial and biological neural networks, potentially unlocking new frontiers in AI capabilities.

Unlock the full potential of state-of-the-art multilingual LLMs with FineWeb2, a groundbreaking 20 terabyte (5 billion document) dataset. This new pre-training data is generated by a revolutionary curation pipeline that automatically adapts to support any language. Overcoming the inherent difficulty of tailoring filtering and deduplication for a large number of languages, FineWeb2 has been scaled to over 1000 languages using Common Crawl snapshots. It produces more performant models than prior datasets for non-English corpora and includes a principled approach to rebalance datasets for additional performance uplift. Access the released dataset, pipeline, training, and evaluation codebases today!

Learn what defines complex systems, such as being made of interacting entities, exhibiting emergent collective behaviour, and being capable of adaptation and self-organization. Understand why the classic Newtonian approach is often inadequate for such systems

Explore the potential of applying Machine Learning to "Big Code"—massive corpora of successful, widely used software systems. Learn how a new data-driven approach is emerging to influence the design of software development tools by leveraging the statistical distributional properties found in these large code bases.