Predicting body composition has many clinical and research applications, but generally requires expensive equipment to measure accurately. We propose an appearance based model for predicting body composition from using only 2D photographs, height, and weight as input.


As part of the Shape Up! Study, principal component analysis (PCA) was performed on 3D optical scans of 67 male subjects to form a statistical representation of human geometry. Subjects underwent whole body DXA scans to provide ground truth total and regional fat mass measurements. Using the silhouette of a subject extracted from a front-facing photograph, we optimize for the 3D body shape, as a function of PCA components, that best fits the silhouette using measured height and weight to inform a shape prior. The camera focal length is known, but we optimize for the camera pose. We map the resulting mesh to body composition with an affine mapping learned from the PCA-fitted training subjects and their percent fat measurements.


This method was benchmarked on 16 male subjects aged 19 to 71 from a reserved test set. Percent fat estimates from the 2D imaging method showed good agreement with DXA (validation RMSE = 2.9 percent units, R^2 = 0.65). Estimating percent fat from a 3D shape sampled from the PCA space using only height and weight priors yielded RMSE = 5.4 and R^2 = 0.098. Visceral fat mass was computed with RMSE = 0.225 kg and R^2 = 0.55.


Body shape from silhouette alone can provide a fast, non-invasive, and inexpensive estimate of body fat. Our method could be improved with more training data to better model the variation in the human population and with shape-from-shading algorithms from computer vision that exploit appearance variation across the surface of the skin. Low cost body analysis methods like this could increase access to medical information previously available only through advanced imaging equipment and could mediate the gap in healthcare access in low income areas.