SignGPT – Project awarded £8.45m to build a sign language AI model for the Deaf community
A large-language model (LLM) built to meet the needs of the Deaf community, translating between signed and spoken language, is the aim of a new project led by the University of Surrey.
SignGPT: Building Generative Predictive Transformers for Sign Language has been awarded £8.45m from the UK Engineering & Physical Sciences Research Council. The five-year project will build tools to allow spoken language to be automatically translated into photo-realistic sign language and video of sign language to be translated into spoken language – a complex translation problem that is yet to be solved.
Surrey will work alongside the University of Oxford, the Deafness Cognition and Language Research Centre at University College London, key Deaf stakeholders, and the Deaf community.
Globally, there are around 70 million Deaf or hard-of-hearing individuals, many of whom use sign language as their primary form of communication. For many, written/spoken languages serve as a second or third language and proficiency in these languages can vary. There is no universal sign language: sign languages are natural human languages created over centuries by Deaf communities and are not derived from spoken languages. Their underlying rules and structures remain a rich area of linguistic study. Each sign language has its own unique grammar and lexicon, relying on both manual gestures (hands) and non-manual expressions (body and face), along with spatial elements, to convey meaning.
This project is a unique collaboration between vision scientists and sign linguists, with Deaf and hearing researchers working together towards our common goals.Professor Bencie Woll, sign linguist, co-investigator of the project, and founder of the Deafness Cognition and Language Research Centre at UCL
I am pleased that this important grant will empower the Deaf community to have further equal access by harnessing AI and large language models. We will ensure that the University of Surrey, Oxford University, and the Deafness Cognition and Language Research Centre at UCL, alongside Deaf-led stakeholders such as RAD, take a people-centred approach to ensuring ethical responsibility and the accuracy of translations so that we, the Deaf community, can use them for everyday purposes.Mark Wheatley, CEO of the Royal Association for Deaf People (RAD)
So much work in sign language technology is undertaken by researchers with no understanding of how sign languages work nor any lived experience of deafness themselves. This project will allow real co-creation/co-development of this technology with Deaf and hearing researchers in linguistics and deaf studies working alongside computer vision specialists – with each group learning from each other – and, importantly, building capacity amongst Deaf researchers so they may lead this field in the future.Professor Kearsy Cormier, one of the Co-Investigators on the project from University College London
SignGPT's research team will produce the largest sign language dataset in the world and use it to build a sign language LLM that can provide the breadth of application to the Deaf community that current LLMs provide for written/spoken languages. In doing so, the project will also generate tools for data annotation that will be released for use by the wider community. The project already has Deaf members within both the research team and wider partners, but it is hoping to recruit more staff for whom British Sign Language is their primary language.
The challenge of automatically translating between sign languages and spoken languages is highly complex and remains unsolved. SignGPT will produce open-source toolkits for linguistic use, web-based demonstrations for accessible knowledge exchange and run outreach programmes alongside collaborative workshops.
Related sustainable development goals
Media Contacts
External Communications and PR team
Phone: +44 (0)1483 684380 / 688914 / 684378
Email: mediarelations@surrey.ac.uk
Out of hours: +44 (0)7773 479911