This Week in AI: Apple won’t say how the sausage gets made

Hiya, folks, and welcome to TechCrunch’s regular AI newsletter.

This week in AI, Apple stole the spotlight.

At the company’s Worldwide Developers Conference (WWDC) in Cupertino, Apple unveiled Apple Intelligence, its long-awaited, ecosystem-wide push into generative AI. Apple Intelligence powers a whole host of features, from an upgraded Siri to AI-generated emoji to photo-editing tools that remove unwanted people and objects from photos.

The company promised Apple Intelligence is being built with safety at its core, along with highly personalized experiences.

“It has to understand you and be grounded in your personal context, like your routine, your relationships, your communications and more,” CEO Tim Cook noted during the keynote on Monday. “All of this goes beyond artificial intelligence. It’s personal intelligence, and it’s the next big step for Apple.”

Apple Intelligence is classically Apple: It conceals the nitty-gritty tech behind obviously, intuitively useful features. (Not once did Cook utter the phrase “large language model.”) But as someone who writes about the underbelly of AI for a living, I wish Apple were more transparent — just this once — about how the sausage was made.

Take, for example, Apple’s model training practices. Apple revealed in a blog post that it trains the AI models that power Apple Intelligence on a combination of licensed datasets and the public web. Publishers have the option of opting out of future training. But what if you’re an artist curious about whether your work was swept up in Apple’s initial training? Tough luck — mum’s the word.

The secrecy could be for competitive reasons. But I suspect it’s also to shield Apple from legal challenges — specifically challenges pertaining to copyright. The courts have yet to decide whether vendors like Apple have a right to train on public data without compensating or crediting the creators of that data — in other words, whether fair use doctrine applies to generative AI.

It’s a bit disappointing to see Apple, which often paints itself as a champion of commonsensical tech policy, implicitly embrace the fair use argument. Shrouded behind the veil of marketing, Apple can claim to be taking a responsible and measured approach to AI while it may very well have trained on creators’ works without permission.

A little explanation would go a long way. It’s a shame we haven’t gotten one — and I’m not hopeful we will anytime soon, barring a lawsuit (or two).

News

Apple’s top AI features: Yours truly rounded up the top AI features Apple announced during the WWDC keynote this week, from the upgraded Siri to deep integrations with OpenAI’s ChatGPT.

OpenAI hires execs: OpenAI this week hired Sarah Friar, the former CEO of hyperlocal social network Nextdoor, to serve as its chief financial officer, and Kevin Weil, who previously led product development at Instagram and Twitter, as its chief product officer.

Mail, now with more AI: This week, Yahoo (TechCrunch’s parent company) updated Yahoo Mail with new AI capabilities, including AI-generated summaries of emails. Google introduced a similar generative summarization feature recently — but it’s behind a paywall.

Controversial views: A recent study from Carnegie Mellon finds that not all generative AI models are created equal — particularly when it comes to how they treat polarizing subject matter.

Sound generator: Stability AI, the startup behind the AI-powered art generator Stable Diffusion, has released an open AI model for generating sounds and songs that it claims was trained exclusively on royalty-free recordings.

Research paper of the week

Google thinks it can build a generative AI model for personal health — or at least take preliminary steps in that direction.

In a new paper featured on the official Google AI blog, researchers at Google pull back the curtain on Personal Health Large Language Model, or PH-LLM for short — a fine-tuned version of one of Google’s Gemini models. PH-LLM is designed to give recommendations to improve sleep and fitness, in part by reading heart and breathing rate data from wearables like smartwatches.

To test PH-LLM’s ability to give useful health suggestions, the researchers created close to 900 case studies of sleep and fitness involving U.S.-based subjects. They found that PH-LLM gave sleep recommendations that were close to — but not quite as good as — recommendations given by human sleep experts.

The researchers say that PH-LLM could help to contextualize physiological data for “personal health applications.” Google Fit comes to mind; I wouldn’t be surprised to see PH-LLM eventually power some new feature in a fitness-focused Google app, Fit or otherwise.

Model of the week

Apple devoted quite a bit of blog copy detailing its new on-device and cloud-bound generative AI models that make up its Apple Intelligence suite. Yet despite how long this post is, it reveals precious little about the models’ capabilities. Here’s our best attempt at parsing it:

The nameless on-device model Apple highlights is small in size, no doubt so it can run offline on Apple devices like the iPhone 15 Pro and Pro Max. It contains 3 billion parameters — “parameters” being the parts of the model that essentially define its skill on a problem, like generating text — making it comparable to Google’s on-device Gemini model Gemini Nano, which comes in 1.8-billion-parameter and 3.25-billion-parameter sizes.

The server model, meanwhile, is larger (how much larger, Apple won’t say precisely). What we do know is that it’s more capable than the on-device model. While the on-device model performs on par with models like Microsoft’s Phi-3-mini, Mistral’s Mistral 7B and Google’s Gemma 7B on the benchmarks Apple lists, the server model “compares favorably” to OpenAI’s older flagship model GPT-3.5 Turbo, Apple claims.

Apple also says that both the on-device model and server model are less likely to go off the rails (i.e., spout toxicity) than models of similar sizes. That may be so — but this writer is reserving judgment until we get a chance to put Apple Intelligence to the test.

Grab bag

This week marked the sixth anniversary of the release of GPT-1, the progenitor of GPT-4o, OpenAI’s latest flagship generative AI model. And while deep learning might be hitting a wall, it’s incredible how far the field’s come.

Consider that it took a month to train GPT-1 on a dataset of 4.5 gigabytes of text (the BookCorpus, containing ~7,000 unpublished fiction books). GPT-3, which is nearly 1,500x the size of GPT-1 by parameter count and significantly more sophisticated in the prose that it can generate and analyze, took 34 days to train. How’s that for scaling?

What made GPT-1 groundbreaking was its approach to training. Previous techniques relied on vast amounts of manually labeled data, limiting their usefulness. (Manually labeling data is time-consuming — and laborious.) But GPT-1 didn’t; it trained primarily on unlabeled data to “learn” how to perform a range of tasks (e.g., writing essays).

Many experts believe that we won’t see a paradigm shift as meaningful as GPT-1’s anytime soon. But then again, the world didn’t see GPT-1’s coming, either.