The Problem of Circularity in Dictionary Curation, and its Implications for Dictionary-Reliant AI

We are designing a machine intelligence capable of understanding and interacting with natural language. Both to facilitate the ease with which it interprets language inputs and to increase its transparency to human users, its very knowledge structures are encoded in natural language. But it isn’t enough to simply map out relationships between words as current LLM-based technologies do—we want it to understand what these words mean. And where better to obtain the definitions it needs to understand language than from the dictionary itself?

For obvious reasons, the definitions in the dictionary were not written with machine-readability in mind. Extensive curation of dictionary entries is required before they can be of any use to a machine. A certain amount of background knowledge can be assumed when it comes to human users, but the knowledge required to make sense of a dictionary definition will often be lacking in the case of a machine. For example, details may be scarce in the definitions, or the hierarchical organization used may be too simple to be useful, as the dictionary writer assumes the human reader will create their own much more complex search structure, which would be too wearisome to describe on tens of thousands of definitions.

One common problem from the perspective of a dictionary-approach to AI is that of circularity. By this, we mean definitions making reference to one another such that the meaning of none of the words involved can be discerned without pre-existing knowledge of what at least one of the words means.

In its simpler forms, circularity will happen in a straightforward way with a few words pointing directly to one another for their definition, as in the following example:

Fix - Mend or repair
Mend - Repair
Repair - Fix or mend

This is a problem because unless the machine already has some notion of what one of those words means, it cannot make sense of any of them. It will get caught up looping back and forth between the definitions as it keeps searching for resolution in an entry that only redirects it back to its starting position.

Sometimes providing a more specific definition of the thing requiring repair may alleviate circularity. For example, defining the object as “a thing suffering from damage or a fault” provides a clue as to what the aim of “repair” is. Likewise, context-rich usage examples such as “faulty electrical appliances should be repaired by an electrician” can serve to further clarify an ambiguously-defined meaning. However, this approach only works for relatively simple concepts.

So how does one go about fixing this circularity issue? In some cases, care needs to be taken in avoiding specific words that will not simply link back to the initial entry.  In the bulk of cases, the problem can be solved by simply creating more inclusive definitions. Words should be explained in detail so as not to have the definition too dependent on any single word. More thorough definitions can compensate for any circularity that may occur, because it ensures that there are alternative keywords that can be relied upon to grasp the meaning of the definition. Additionally, it provides context, so that the meaning of an unknown word may potentially be pieced together from the circumstances of its occurrence.  

Before one can go about correcting circularity within the dictionary, it is first necessary to detect it. This is easier said than done, as it may not become apparent that the relevant definitions only fold back into themselves until one has gone multiple levels deep in following their path.

In the below example, an attempt is carried out to interpret the definition of “safe”. At an initial glance, the definition appears to contain a wide enough range of keywords to compensate for any instances of circularity that may be encountered, but when one pursues these definitions through multiple expansions, it becomes apparent that all roads lead to “harm”, which itself only loops back to previous words in the chain.

(As an aside, rushing through the sky in a metal tube at near the speed of sound is not obviously “safe”, but it can meet the definition of “not likely to be harmed” by the enormous effort that is made to make it safe.)

This example serves to illustrate that the problem of circularity in dictionary definitions is often buried multiple levels deep. In such cases, it is no less insidious, but not as simple to detect.

For a human to catch all such instances of circularity, it would require untold labour hours. But while we already have a machine combing through the dictionary to map out an understanding of its contents, why not use the machine itself to detect such instances? The process could be readily automated, sending up a flag to human specialists whenever it encounters a problem. This way, human efforts can be focused on manual restructuring rather than wasted on simple detection.

This post was prompted by the desire by some to impose concepts like “safe”, “trustworthy”, and “loyal”, on an LLM.

We will ignore the difficulty of connecting a concept to something which does not understand the meaning of words, and stick to how to define the words.

One may feel safe by erecting a large fence, but the large fence may alert criminals that there is something of value which may be stolen, so what seems obvious may be self-defeating – another concept to get across.

And when defining loyalty, “he was loyal to a fault” may give one pause.

Trying to define a concept will take you to many places you may not have expected. If the concept applies to the content of social media or the output of an LLM – many times so.