Government Digital Service (2024) Government website / press release

The findings of our first generative AI experiment: GOV.UK Chat

Inside GOV.UK Blog

Ref: SRC-004-GBR-002

https://insidegovuk.blog.gov.uk/2024/01/18/the-findings-of-our-first-generative-ai-experiment-gov-uk-chat/

Accessed: 3/24/2026

Summary

Details of the initial experiment: sentence-transformers/all-mpnet-base-v2 embedding model, OpenAI gpt-3.5-turbo-16k via API, LangChain framework, five phased experiments, 1,000-user pilot (157 survey completions), 70% usefulness and 65% satisfaction ratings, hallucination concerns, DPIA conducted, trust concern that users may underestimate inaccuracy risks due to GOV.UK brand credibility.

View Harvard reference

GDS (2024) 'The findings of our first generative AI experiment: GOV.UK Chat', Inside GOV.UK Blog, 18 January. Available at: https://insidegovuk.blog.gov.uk/2024/01/18/the-findings-of-our-first-generative-ai-experiment-gov-uk-chat/ (Accessed: 24 March 2026).

Attached File

FILE

SRC-004-GBR-002_The_findings_of_our_first_generative_AI_experiment_GOV.UK_Chat.html 92 KB

Download

Linked Cases (1)

GBR-002 GBR Pilot / Controlled Trial Phase

GOV.UK Chat

Social assistance Foundation model