Semantic AI and LLMs

 

Why do people seem to think that the meaning of words is unnecessary for an LLM to understand English text. Is it because many people have no understanding of the work involved in parsing text and imagine that if they don't have to consciously think about it then an LLM doesn't have to think about it either.

A simple example – the word “bar” as a noun (deciding whether a word is acting as a noun or a verb can be tricky – this word can also be a preposition).

An iron bar, a wine bar, a colour bar, a bar to entry, a unit of pressure, a measure of music, the legal profession.

The path between speaker and listener

English, even with a 50,000 word vocabulary and about 10,000 wordgroups (combinations of words that have a different meaning to the apparent meaning of the individual words), is still a shorthand for describing the mental world. 

Isn't Semantic AI just words? No, it manages the meanings of words.

For nouns, there are several flavours:

    Simple Noun - an object - a dog.
    InfinitiveNoun - a noun linked to an infinitive - the need to avoid dehydration,
    Clausal Noun - a noun "owning" a clause - the idea that the world is flat.

For verbs, there are many flavours:

    Intransitive Verb - no object - the hamster died of old age
    Transitive Verb - a direct object - he fed the dog
    Ditransitive - he bought John a bicycle
    Infinitive 0-he wanted to know the truth
    BiTransClausal - I bet you any money your horse won't win
    and about a hundred other forms

Connecting a node in text to a verb and its parameters

The meanings for the intransitive form of "run"

It gets complicated very quickly - the particular meaning for a word may not be known until all the text is read - a good reason for humans doing it unconsciously so they don't get confused

The same kind of structure applies to:

    Adjectives - attributive, predicative, trailing (aforethought)
    Adverbs - relational, adjectival modifier, adverbial modifier, sentential
    Prepositions - a whole heap

LLMs don’t have any idea about meaning, and have no use for synonyms either.

A segregated dog park.

A park dedicated to dogs and their owners, separated into different areas for different sizes of dogs.

One will be picked up by searching, one won’t. They can be paired with a tool that expects a particular input, and then the combination is undone by an extra prepositional phrase. 

LLMs epitomise the "quick and dirty" approach, where we will throw away a natural language approach and rely on words being close to other words - fine if you are doing something trivial, not fine if you intend to rely on its output.

As a stopgap measure until Semantic AI arrives for the simple stuff (the current goal is to do the complex stuff - stuff that people struggle with and make multi-billion dollar errors), the error rate may be barely tolerable, but they need to be replaced as soon as Semantic AI for simple stuff becomes available.

Why has it taken so long for Semantic AI to arrive. A fast buck is always attractive. Calling LLMs AI was proven to be an easy way to make money, relying on a gullible public (and Semantic AI took a lot of work, but has greatly raised the complexity that people can handle (see Four Pieces Limit).

Orion

Semantic AI