For reward systems created by either users or companies to be possible, the devices would have to know what you’re doing at all times. The language of these patents makes it clear that Google is acutely aware of the powers of inference it has already, even without cameras, by augmenting speakers to recognize the noises you make as you move around the house. The auditory inferences are startling: Google’s smart-home system can infer “if a household member is working” from “an audio signature of keyboard clicking, a desk chair moving, and/or papers shuffling.” Google can make inferences on your mood based on whether it hears raised voices or crying, on when you’re in the kitchen based on the sound of the fridge door opening, on your dental hygiene based on “the sounds and/or images of teeth brushing.”
