Apple Unveils Details of New AI Foundation Models | JQJO

Apple released a tech report detailing its new on-device and cloud-based foundation models. The on-device model, with 3 billion parameters, is split into two blocks for improved memory efficiency and speed. The cloud-based model uses a custom Parallel-Track Mixture-of-Experts architecture for faster, more accurate responses. Multilingual support significantly improved, increasing multilingual data from 8% to 30% of training data. Data sources included web crawling (respecting robots.txt), licensed content, and synthetic data. The report highlights Apple's focus on privacy and efficiency.