Over the last several years, artificial intelligence (AI) has made huge strides forward in the areas of language modelling and translation. Recent improvements are exemplified by the accuracy of machine translation services such as Google Translate, the ability of devices like Amazon’s Alexa to interpret and respond to instructions given in conversational speech, and the automated production of summaries based on a source text.
However, traditional language-modelling AIs still require exposure to massive amounts of examples, often in the millions, to learn how to understand or produce language. Additionally, while these AIs do develop an understanding of language rules, exactly how this understanding works remains unclear.
“There’s been this huge and quite breathtaking improvement in the performance of things like language models […] but what they do internally, the kinds of implicit theoretical representations that they learn, are just not available for humans to read,” explained Timothy O’Donnell, assistant professor in McGill’s Department of Linguistics, in an interview with The McGill Tribune.
O’Donnell recently co-authored a research article published in Nature Communications describing an innovative approach to training AI to understand the rules of human language. O’Donnell and his team of researchers from McGill and the Massachusetts Institute of Technology focused on an area of linguistics problems in morphology—a branch of the discipline that seeks to understand the internal structure of words. The team looked more specifically at patterns of sound change within languages and they took morphology problem sets from standard introductory textbooks as their testing grounds for the AI.
To solve these simple linguistics problems, the researchers developed an AI that could turn them into complex logical expressions, which could then be understood and solved by a computer.
“We took a problem that we can read, we transformed it to an extremely complicated logic problem, which we can’t,” O’Donnell said. “If you looked at it, you would have no idea what was going on. But there’s specialized software that’s really good at solving these kinds of logic problems.”
By transforming linguistics problems into logic problems, O’Donnell and the researchers could train their model to develop morphological rules and patterns using significantly fewer examples than traditional AI techniques would usually require.
Another advancement is that its output is completely human-interpretable. In other words, the system creates rules that can be read and understood by a human researcher. Taken as a whole, the process is remarkably similar to the way that human linguists develop theories about language.
“We are very inspired by the way in which […] professional linguists proposed theories,” O’Donnell said. “If you go and you’re studying a new language somewhere, you collect some data, and then people immediately, even when the datasets are relatively small, start proposing theories of phonology and the morphology of these new languages.”
This software could even be considered a “theory builder,” or an AI that analyzes a set of data and is able to develop its own scientific theories to explain the data. Models like this, which are also being worked on in other scientific fields, have the potential to drastically change the way that researchers develop theories.
“One of my collaborators called it thinking prosthetics, so, a way to extend the ability of researchers to think by using automated tools to help them search for good theoretical proposals,” O’Donnell explained.
This particular model is still functioning at a fairly basic level and being tested mostly on pre-determined examples. However, given the speed of development in AI right now, one can imagine a near future where researchers are assisted by, or even surpassed by, this type of theory-building AI.
