An evaluation of ChatGPT for nutrient content estimation from meal photographs

Background/Objectives: Advances in artificial intelligence now allow combined use of large language and vision models; however, there has been limited evaluation of their potential in dietary assessment. This study aimed to evaluate the accuracy of ChatGPT-4 in estimating nutritional content of commonly consumed meals using meal photographs derived from national dietary survey data.

Methods: Meal photographs (n = 114) were uploaded to ChatGPT and it was asked to identify the foods in each meal, estimate their weight, and estimate the nutrient content of the meals for 16 nutrients for comparison with the known values using precision, paired t-tests, Wilcoxon signed rank test, percentage difference, and Spearman correlation (r_s). Seven dietitians also estimated energy, protein, and carbohydrate content of thirty-eight meal photographs for comparison with ChatGPT using intraclass correlation (ICC).

Results: Comparing ChatGPT and actual meals, ChatGPT showed good precision (93.0%) for correctly identifying the foods in the photographs. There was good agreement for meal weight (p = 0.221) for small meals, but poor agreement for medium (p < 0.001) and large (p < 0.001) meals. There was poor agreement for 10 of the 16 nutrients (p < 0.05). Percentage difference from actual values was >10% for 13 nutrients, with ChatGPT underestimating 11 nutrients. Correlations were adequate or good for all nutrients with rs ranging from 0.29 to 0.83. When comparing ChatGPT and dietitians, the ICC ranged from 0.31 to 0.67 across nutrients.

Conclusions: ChatGPT performed well for identifying foods, estimating weights of small portion sizes, and ranking meals according to nutrient content, but performed poorly for estimating weights of medium and large portion sizes and providing accurate estimates of nutrient content.