Calculus of Language

I'm tentatively calling the new syntax and related logic a “calculus of language” as it has the properties of a predicate calculus and appears to represent natural language well. Syntactically, it's a superset of predicate calculus. I am writing up a draft of it, going over the formal logic and grammars, and heuristics for the sequential parsing of natural language and hope to publish it soon. I'll comment on any developments as they occur; this technique appears to be working on arbitrary natural language sentences. However, the output is strings, meaning that getting to the exact semantics (URI/integers) appears to be a second step algorithm.

Hypotheses

The hypotheses that I'm presently testing are

1) Representation Hypothesis

Letting P(X,Y) = ~P(Y,X)

A natural language sentence can be represented:

S → S₁ | S₁ S₂ | S₁ S₂ S₃ ...
S_i → P(A,A) | ~P(A,A)
A → P(A,A) | ~P(A,A) | N | _
N → n₁ | n₂ | n₃ ...
P → p₁ | p₂ | p₃ ...

2) Noun Order Hypothesis

We can view S as a sequence of trees. The leaf nodes of those trees, that are nouns, if viewed as a sequence, can be such that a noun can only be in that sequence if the previous noun from the natural language sentence exists previously in the sequence.

3) Hypothesis of Heuristic

It is heuristically possible to convert to and from this representation and natural language.


Optionally, for sentence structures resembling “X makes Y Z” and “X finds Y Z”, and should it help the heuristic, we can ammend the representation hypothesis:

S_i → P(A,A) | ~P(A,A) | P(A,P'(A,A)) | ~P(P'(A,A),A)
A → P(A,A) | ~P(A,A) | P(A,P'(A,A)) | ~P(P'(A,A),A) | N | _

Where P' can be determined from P.

Machine Reading, Paraphrases

1) {Bobby} [likes][drawing] {pictures} [with] {crayons}.
Likes(bobby,With(Draw(_,pictures),crayons))

2) {Bobby} [likes][using] {crayons} [to draw] {pictures}.
Likes(bobby,Using(crayons,Draw(_,pictures)))
Likes(bobby,~With(crayons,Draw(_,pictures)))

So, the nesting of predicates can be discerned by noun order paraphrases. This is theoretical at this point, but so far so good. I'm using the premise of semantics-preserving noun order paraphrases as an axiom in theorizing machine reading algorithms.

Representation Notes

This sentence suggests that gerunds should be in noun brackets.
{Bobby} [likes] {drawing}. B(A,C)

However,
{Bobby} [likes] %drawing% B(A,C) or B(A,C(_,_))

{Bobby} [likes drawing] {pictures}. B(A,C)
{Bobby} [likes][drawing] {pictures}. B(A,C(_,D))
{Bobby} [likes][to draw] {pictures}. B(A,C(_,D))

So the gerund can be P(_,_) and the infinitive can be the same or, if a direct object is specified, be P(_,Y).

{Bobby} [drew]. B(A,_)

An argument for structural reuse is in the sentence:
As Bobby likes to draw pictures with crayons, he went to the store and he purchased a box of them.
<As> [[1]], {he}[went to]{the store} and {he}[purchased] {a box}[of]{them}.
As([[1]],[[2]])

The <> brackets represent elements or operators that relate predicates or sequences of predicates, logic operations and other punctuation relevant to processing.

Wikipedia Example 2

Knowledge representation is an issue that arises in both cognitive science and artificial intelligence. In cognitive science it is concerned with how people store and process information. In artificial intelligence (AI) the primary aim is to store knowledge so that programs can process it and achieve the verisimilitude of human intelligence. AI researchers have borrowed representation theories from cognitive science. Thus there are representation techniques such as frames, rules and semantic networks which have originated from theories of human information processing. Since knowledge is used to achieve intelligent behavior, the fundamental goal of knowledge representation is to represent knowledge in a manner as to facilitate inferencing i.e. drawing conclusions from knowledge. [1]

{Knowledge representation|A} [is|B] {an issue|C} that [arises in|D] both {cognitive science and artificial intelligence|E}. [In|F] {cognitive science|G}, {it|H} [is concerned with|I] [how|J] {people|K} [store and process|L] {information|M}. [In|N] {artificial intelligence|O} {the primary aim|P} [is|Q] [to store|R] {knowledge|S} [so|T] that {programs|U} [can process|V] {it|W} and [achieve|X] {the verisimilitude|Y} [of|Z] {human intelligence|AA}. {AI researchers|AB} [have borrowed|AC] {representation theories|AD} [from|AE] {cognitive science|AF}. <Thus|AG> <there are|*> {representation techniques|AH} [such as|AI] {frames|AJ}, {rules|AK} and {semantic networks|AL} which [have originated from|AM] {theories|AN} [of|AO] {human information processing|AP}. <Since|AQ> {knowledge|AR} [is used to achieve|AS] {intelligent behavior|AT}, {the fundamental goal|AU} [of|AV] {knowledge representation|AW} [is|AX] [to represent|AY] {knowledge|AZ} [in|BA] {a manner|BB} as [to facilitate|BC] {inferencing|BD} <i.e.|BE> {drawing conclusions|BF} [from|BG] {knowledge|BH}.

B(A,C)
D(C,E₁)
D(C,E₂)
I(F(H,G),J(K,L₁(K,M)))
I(F(H,G),J(K,L₂(K,M)))
Q(N(P,O),T(R(_,S),V(U,W)&X(U,Z(Y,AA))))
AG(AC(AB,AE(AD,AF)),AM(AI(AH,AJ),AO(AN,AP)))
AG(AC(AB,AE(AD,AF)),AM(AI(AH,AK),AO(AN,AP)))
AG(AC(AB,AE(AD,AF)),AM(AI(AH,AL),AO(AN,AP)))
AQ(AS(AR,AT),AX(AV(AU,AW),BC(BA(AY(_,AZ),BB),BD))
BE(BD,BG(BF,BH))

[1] Knowledge Representation, Wikipedia

Machine Reading, Paraphrases

A rule system may allow semantic subtrees to be mapped to one another. For example, the paraphrase:

Tommy put the book on the shelf to be helpful.
InOrderTo(Did(tommy,PutOn(book,shelf)),Is(tommy,helpful))

The following rule Is(X,helpful) → Help(X,_) might resemble that if something is helpful then that something help(s/ed) some thing or things. Formulating rules in this manner might transcend, in part, hermeneutic circles that occur when relating lexical elements to one another.

Hermeneutic circle refers to the semantic interconnectedness of a set of entities— for example, the definitions in a dictionary refer to other words in that dictionary. Philosophers from Schleiermacher and Dilthey through Heidegger and Gadamer considered this phenomenon. Wittgenstein said regarding this that light dawns gradually over the whole.

A rule system lexicon might further assist the semantic equivalence of paraphrases allowing semantic substructures to map to one another based on the lexical hermeneutic circle.

Like most things in AI, machine reading is more easily described than programmed. Any system that can equate noun-order paraphrases to the same set of predicates (or permutable equivalents) would be a milestone in my opinion.

Reification, Ontological Consensus and Ergonomics

Ontological consensus is the goal of the WikiCorpus project and is, in theory, possible via the on screen predictive list of predicates. Users would rather select predicates (that work for the semantics of the sentence) from the list than create new ones, just as users prefer to find the results of a search on the first page of a search engine's results. This list, hopefully produced by an accurate predictive algorithm, will facilitate consistency in the dataset.

In the user interface, the ergonomics of nested predicates would be that a completed predicate can be dragged (moved or copied) into the argument slot of a predicate being formed. The problem is, firstly, that moving and copying the predicate are both done with the drag and drop motion. Secondly, the nesting of predicates might deter from semantic and ontological consensus as complicated constructions are possible from the elements in the predictive list.

Take for example:
1) The book was put on the shelf.
2) Tommy put the book on the shelf.
3) Tommy put the book on the shelf to help the library.

Each is a semantic superset of its previous sentence duplicating and nesting the previous predicate.

1) PutOn(book,shelf)
2) Did(tommy,PutOn(book,shelf))
3) InOrderTo(Did(tommy,PutOn(book,shelf)),Help(tommy,library))

That is, the first sentence is the first predicate, the second is both the first and second, and the third is all three. Two terms related to this method of representing sentences are that sentences have a semantic core and a semantic root (I'll try to find out if other terminology already exists). The core is the predicate constructed from, often the most nested predicate, and the root is the least and the root of the corresponding tree structure.

So, while recursive binary predicates appear to be able to capture natural language, the interface considerations and ergonomics are more complicated. Also, finding the semantic core of sentences appears to relate to placing the sentence into its semantic frame via PropBank or FrameNet corpora; the algorithm would then be to construct the entirety. These nested predicates can be represented in the matrix format with some notational conventions.

Interestingly, similar to the reording of nouns in the matrix format, the reordering of the predicate arguments or nouns can be done, resembling:

InOrderTo(DoneBy(PutOn(book,shelf),tommy),Help(tommy,library))
The book was put on the shelf by Tommy to help the library.

InOrderToInverse(Help(tommy,library),DoneBy(PutOn(book,shelf),tommy))
In order to help the library, the book was put on the shelf by Tommy.
In order for Tommy to help the library, the book was put on the shelf by him.

InOrderToInverse(Help(tommy,library),Did(tommy,PutOn(book,shelf))
In order for Tommy to help the library, he put the book on the shelf.
To help the library, Tommy put the book on the shelf.

So, each binary predicate may be related to another predicate with its arguments in the opposite order. Some predicates may not, limiting the possible noun orderings for paraphrases. This is just one approach to capturing semantics using nested predicates; I look forward to learning other approaches and designing a web interface for a collaborative corpus.

Knowledge Representation, Paraphrases

In machine reading, a goal is for all paraphrases to be processed into the same set of predicates or other form of representing semantics. One method I found to represent this feature is to represent a sentence as a block matrix representing the pairwise binary predicates between nouns. This utilizes the fact that n-ary predicates can be decomposed into a set of binary predicates.

The rows and columns are in the order of nouns occurring in a sentence and the entry in the i-th row and j-th column is meant as the predicate(s) that relate(s) the i-th and j-th noun (nouns on main diagonal). Using this, permutations can change noun ordering (one variety of paraphrases) and the semantics can be preserved.

Example:
1) Tommy went to the store to get a crescent wrench. <Tommy,store,wrench>
2) To get a crescent wrench, Tommy went to the store. <wrench,Tommy,store>

From this or another noun-order invariant representation format, the goal of NLG is then to compose grammatically correct sentences containing the semantics with the nouns in the order of the underlying matrix. A reason for robustness in this is that noun ordering is often context dependent, as sentences are composed in paragraphs and documents; both noun ordering and sentence aggregation [1] are overarching processes that are part of “good writing style”.

Using the phases of NLG from the article, the process with this knowledge representation may resemble:

Content determination: Knowledge is obtained from a knowledgebase or web.
Discourse planning: Knowledge is moved into a matrix format.
Sentence aggregation: Block diagonalization and utilization of other patterns discerned from a corpus of well-written articles.
Lexicalisation: Putting words to the concepts.
Referring expression generation: Linking words in the sentences by introducing pronouns and other types of means of reference.
Syntactic and morphological realisation: Permutations are applied as per patterns discerned from well-written articles; each sentence is realized with the nouns in the best order.
Orthographic realisation: Matters like casing, punctuation, and formatting are resolved.


[1] Natural Language Generation, Wikipedia article

Integer-Based Representation

I'm working with integer-based or numerical representations in triples and quadruples. Basically, this approach uses bitfields instead of URI. I designed it to be computationally faster than string-based formats and the bitfields allow differentiation between tense negation, logical negation, set complementing and other operations on entities and relation types. I'm working on some tools to convert XML-based data to and from this format.

Also, n-ary predicates can be converted to a set of triples (or quadruples) by combinatorically relating the n arguments (pairwise) using binary subpredicates. This could be of use in converting sentence predicates into a triples-based language.

A rule system will probably be required to capture overlap between different predicates' subpredicates — which is unfortunate as there are upwards of 4,527 frames in FrameNet and 3,635 in PropBank. This pairwise relating the syntactic elements in n-ary predicates or frames should result in more useful structured knowledge. This is a first attempt at a post-SRL/pre-NLG format.