Amazon Health AI
An organization that turned phone-camera images into health measures: body fat, waist‑to‑hip ratio, and mobility, stability and posture.
- Role
- First product lead of Amazon’s Health Computer Vision and AI organization
- Scale
- 100+ scientists and engineers; a $100M+ a year investment portfolio I directed
- Validated
- Three studies in npj Digital Medicine: Body and Movement in 2022, waist-to-hip in 2023
I was the first product lead of Amazon’s Health Computer Vision and AI organization: 100+ scientists and engineers and a $100M+ a year portfolio. I set the rule that more money waited for two proofs; three of its products were validated in npj Digital Medicine.
What I decided
Brought the leaders together on one direction
I brought the organization’s leaders together for three days to agree on one direction: using the camera in everyone’s phone to help people live healthier lives.
Required two proofs before more money
Each new capability needed two proofs before more money: a customer problem stated without the technology, and experiments showing the capability worked. If either was missing, we paused further investment.
Chose each measure’s clinical reference first
For each product, I chose the clinical reference by starting from the claim we needed to publish. All three studies appeared in npj Digital Medicine, a Nature Portfolio journal.
How it worked
Every measure needed a clinic and an instrument
Body fat came from a clinical dual-energy X-ray absorptiometry (DXA) scan, a movement assessment from a motion-capture suit and a physical therapist, and a waist-to-hip ratio from trained staff with a tape. Each needed a clinic and trained staff.
One organization, from research to products
In 2021 I became the first product lead of Amazon’s Health Computer Vision and AI organization, which ran from research and development to commercial products.
What the camera asked, and what came back
Halo Body estimated body fat from phone photos, and from August 2021 its scan worked from the knees up, for members in smaller rooms. Halo Movement scored mobility, stability and posture from 0 to 100, from five exercises filmed by a phone on the floor. A third model estimated waist-to-hip ratio, a body-shape measure linked to disease risk, from three photos taken to voice prompts.
Each measure against its own reference
Body was tested against clinical DXA scans at two clinical sites, Movement against a motion-capture suit and 13 clinical tests. The waist-to-hip model was tested on 1,200 people, 550 measured with tape by trained staff and 541 by laser body scan.
Three products, three clinical references
Each product was checked against its own clinical reference.
The data as a table
| Product | The phone saw | Checked against | The result | Published |
|---|---|---|---|---|
| Body | Front and back photos | DXA, a clinical X-ray scan, 134 adults | About 2.2 points, average error in body fat; the limit was 3 | npj Digital Medicine, June 2022 |
| Movement | Video of five exercises | A motion-capture suit and 13 clinical tests, 150 adults | Moderate to strong agreement with the suit and the 13 tests | npj Digital Medicine, September 2022 |
| Waist-to-hip | Front, side and back photos | Tape measures by trained staff, 1,200 people | About half the error of people measuring themselves | npj Digital Medicine, September 2023 |
Friday builds with a 1% chance of working
Some ideas started as Friday builds: each Friday a product manager, a designer and a group of engineers took on one charter, “build something that won’t work.” We chose builds with about a 1% chance of working and a tenfold gain on a business metric if they did.
Average error in waist-to-hip ratio, against trained staff with a tape
Launch
On 28 September 2022, at Amazon’s devices event, I introduced Halo Rise, a bedside sleep tracker built by Amazon’s Halo Rise team.
What came next
The waist-to-hip study was published in September 2023. In June 2024 I published the method we ran, in an essay for product managers building consumer AI products.
