Exploring the potential of customized AI chatbots in food policy research: Capabilities and constraints in comparative perspective

Abstract

Artificial Intelligence (AI) tools are rapidly transforming the way people work, providing opportunities for improving efficiency, accelerating innovations, and supporting human decision making. The area of policymaking is no exception. To explore the advantages and disadvantages of customized chatbots for supporting food policy research, we created a virtual food policy expert using a customized AI system using training literature drawn from about two dozen food policy research reports or food policy documents. A range of questions seeking food policy advice were then posed to the chatbot to better understand the limits to its expertise, the accuracy of its responses, and the value it might offer users seeking specific advice to guide the formulation and implementation of food policy related strategies and programs. A relatively simple comparative assessment was made of the results obtained from asking the same food policy-related questions to different AI sources, including the customized AI virtual food policy expert. Overall, we found that AI -based chatbots can be useful tools for summarizing research and providing “first cut” information only. We also conclude that, while customized chatbots have the advantage of offering the possibility of uploading pre-selected papers, unless a large amount of information is uploaded or the chatbot is also allowed by design to draw on information from other sources including the internet, the responses the virtual coach provides may be limited to only a small and specific set of questions. To be more credible and useful for policymakers, chatbots need to be enabled to provide more specific information. For the foreseeable future, providing such detailed and often country-specific information will still be in the hands of human experts.