I led two major projects to digitise the Medical Officer of Health (MOH) reports held at Wellcome Collection to make them easier to discover, search and use online. The work combined funding, digitisation, OCR, tabular data extraction and experimentation, rights management and public access at scale. This collection of 70,000 reports span 150 years of UK public health management, charting the entire duration of regional MOH departments.
The reports contain narrative text, numerical tables, charts, graphs and occasional illustrations and annotations. They are mostly printed, with a few typescript, and thus suitable for OCR. Over several years, Wellcome’s holdings and a further 500 reports from the London Archives were digitised, OCRed and made available online. The work was carried out in two stages: a pilot project on the London reports, followed by a larger project to digitise the remainder of the collection.
The first stage began in 2018 when I secured part-funding through a Jisc grant scheme for a digitisation pilot project focused on the Greater London reports. Around 5,500 reports were digitised and OCRed and around 270,000 tables were processed using a combination of automated layout analysis and manual correction. This data powers London’s Pulse, a public website with contextual information about the history of the MOH, a search interface, and access to images and downloadable tables.
After the pilot I worked on scaling up digitisation at Wellcome Collection as part of the UK Medical Heritage Library programme. Increased on-site digitisation capacity with the Internet Archive made it possible to digitise the remaining Wellcome holdings: a further 65,000 reports covering the UK and Ireland and a small number of reports from former British colonies. Digitisation and OCR was completed in late 2018. The MOH reports are available on the Wellcome Collection website, and the reports from this expanded digitisation project are also available on the Internet Archive website.
Due to the vast scale of the project – including potentially 2-3 million tables – it was not financially viable to extract tables using the semi-automated process used for the London reports. To explore practical options, I commissioned researchers from The University of Salford’s Pattern Recognition and Image Analysis Research Lab (PRImA) to carry out a pilot study and recommend future approaches. You can read more about how this study was developed here. A paper on the study was presented at DATeCH in 2019, and the full text is available online. Due to changing priorities within Wellcome Collection during and after Covid, this work did not proceed into operational development. Technology has changed significantly in the intervening years, with improved character and layout recognition and the growing development of AI-assisted tools to support analysis of complex documents such as these.
I also led the rights management aspects of both projects, learning many lessons along the way, and connecting with regional archive and record offices around the country. The project was well received by all who engaged with us, and this continued as I adjusted the rights clearance strategy to an opt-out approach more appropriate to the scale of the effort, the nature of the original materials, and the intentions of the project.
Header image credit: MOH reports, Wellcome Collection, CC BY.
