Apple has revealed updates to the AI models that support its Apple Intelligence features across devices like iOS and macOS. However, according to the company’s own tests, these new models don’t perform as well as older models from competitors like OpenAI.
In a blog post on Monday, Apple shared that human testers found the text produced by its latest “Apple On-Device” model, designed to work offline on devices such as the iPhone, to be “comparably” good, but not superior to the text generated by similar models from Google and Alibaba. Additionally, testers rated Apple’s more advanced model, known as “Apple Server,” lower than OpenAI’s year-old GPT-4o.
In another assessment focused on image analysis, human raters preferred Meta’s Llama 4 Scout model over Apple Server, which is unexpected. In various tests, Llama 4 Scout has performed worse than leading models from AI companies like Google, Anthropic, and OpenAI.
These benchmark findings support reports that suggest Apple’s AI research team is having difficulty keeping pace with rivals in the competitive AI landscape. Recently, Apple’s AI offerings have not impressed many, and a much-anticipated upgrade for Siri has been postponed indefinitely. Some customers have even filed lawsuits against Apple, claiming the company has promoted AI features that are not yet available.
The Apple On-Device model, which has around 3 billion parameters, is responsible for tasks like summarization and text analysis. (Parameters are indicators of a model’s problem-solving ability; generally, more parameters lead to better performance.) As of Monday, third-party developers can access this model through Apple’s Foundation Models framework.
Apple claims that both the Apple On-Device and Apple Server models offer improved efficiency and tool usage compared to earlier versions and can understand about 15 different languages. This improvement is partly due to a broader training dataset that now includes images, PDFs, documents, manuscripts, infographics, tables, and charts.
