Cosmology, Algorithms, Metaphysics Corey Hubbard Cosmology, Algorithms, Metaphysics Corey Hubbard

Telescopes Got Brains! How AI is Supercharging Your Cosmic View

Explore the transformative role of Artificial Intelligence (AI) in astronomical optical telescopes, revolutionizing both their design and operation. This comprehensive review details AI applications across key domains, including manufacturing planning observatory site selection and optical system intelligence  and operational maintenance

Read More
Creativity, Digital Age, Computer Science Corey Hubbard Creativity, Digital Age, Computer Science Corey Hubbard

The 20TB Multilingual LLM Data Revolution | Scale to 1000+ Languages with One Pipeline

Unlock the full potential of state-of-the-art multilingual LLMs with FineWeb2, a groundbreaking 20 terabyte (5 billion document) dataset. This new pre-training data is generated by a revolutionary curation pipeline that automatically adapts to support any language. Overcoming the inherent difficulty of tailoring filtering and deduplication for a large number of languages, FineWeb2 has been scaled to over 1000 languages using Common Crawl snapshots. It produces more performant models than prior datasets for non-English corpora and includes a principled approach to rebalance datasets for additional performance uplift. Access the released dataset, pipeline, training, and evaluation codebases today!

Read More