Neural networks are all the thing because they are the core of all the chatbots that are around the place these days. Dominating the news of AI technology, billion-dollar startup companies, infighting between the not-for-profit and the aggressive profit versions, and seeping into almost everything you see and touch.
On my search engine, which used to always pull up Wikipedia first, now an AI system gives me a result too. When I compare the two though it is clear the AI has taken advice from Wikipedia, and other places.
That is a hint to an associated problem. That the Large Language Models or LLMs all get their training from everything available to it on the net, so there are copyright battles.
In the beginning Neural Nets were called Perceptrons. I liked that word better because neural nets are nothing like biological neurons.
The term was created by Frank Rosenblatt in 1957 and I published a paper called “A Trichotomy on Vision” when I was at Elizabeth Matriculation College in 1970, as an entry in the Junior Science Talent Quest. I talk about that more in my book “The Vandemonian” and how Seymour Papert who was the inventor of Logo, the software behind my turtle educational robots, (in 1979, the “Tasman Turtle”), and Marvin Minsky, both at MIT, published their controversial book “Perceptrons” in 1969 documenting the limitations of Perceptrons.
All of this was a primer for me, because I quickly discovered the work of Jerry Lettvin and others, about how frogs see, duplicating the research in my Trichotomy paper, then the work of David Hubel and Torsten Wiesel on receptive fields. From it I developed receptive field networks instead of neural networks in pattern recognition experiments.
Getting away from artificial vision, the next thing was the work both John Reid and I did to add speech recognition to the robots. John with his mathematical genius tried different techniques, Fourier based mostly, equations way beyond me, but settled eventually to signal processing methods using Comb Filters.
That was all fine for recognising sounds like formants in words, but the interesting thing for me was natural language understanding. I had researched the work of Terry Winograd on what he called “Primitives” and on Chomsky’s “Generative Grammars”. The first conceiving language as consisting of words for objects and their fundamental properties. The second on how they are strung together to make comprehendible representations. Combined those two approaches gave me ideas for understanding spoken sentences.
John took a different approach, using his bent for statistics, and preferred a Predictive way of forming sentence syntax.
I had no time for that just as I had no time for neural nets, but interestingly the modern work on LLMs, which have amazing results, use massive neural nets with revolutionary processing techniques within the network, feeding back making final decisions for instance in what is called transformer deep learning.
The mathematics is still beyond me, but the concept attempts to predict the meaning or relevance of each word in a sentence by what it calls focussing. Each word in a sentence is statistically associated with every other word in the sentence in a way that transforms the attention from the linear ordering of the sentence to the more useful association of thoughts or ideas or concepts being portrayed in the sentence. In other words the semantics or meaning of the sentence.
For example, in the sentence, “the cat which is on the mat is old and died in its sleep” the word “cat” is as closely associated with the word “sleep” as any other word, and more so than the words around it like “which” or “is” or “on”. Only then can the meaning and relationship between “cat” and “sleep” be made.
The point is that it is still a statistical prediction thing. I see it in my word processor all the time, trying to predict what I will say next after a word I just used. Never correct. Probably because I talk funny.
Back in the 60’s Pepper traversed 40 miles tucked into rather confined spot next to the battery and engine block, close to the cooling fan, of our station wagon. He had a quarter inch or so of an ear missing when he climbed out from under the car at my grandparents house. Pepper was a huge tom cat. Your cat was a different species all together. Yours looks like it spent a lot of time “running through the jungle” (CCR song comes to mind).
Thanks for the fascinating story! I have a business card from Brian Heil, Applications Consultant, Systronics, Inc., Norcross Georgia sitting on top of my vintage floor speaker next to my favorite reading spot on the sofa. Back in the day we used a Systronics vision system to identify anomalies in coatings on/into a membrane. Star wars data processing techniques led to finding those anomalies later for removal downstream of the drying sections of the production line.
Your reference to “Chomsky’s “Generative Grammars” sent me off to Duck Duck Go to see what it reported back on my- “jaspers on communications” query as it’s been a LONG time since I read “Reason and Existenz”. Not a single Wiki link was reported on the first page of results.
I think I’ll add these suggestions to my reading list-
https://www.academia.edu/115597550/Karl_Jasperss_Philosophy_of_Communication
https://www.academia.edu/100051555/The_Failed_Attempt_at_Existential_Communication_between_Martin_Heidegger_and_Karl_Jaspers_as_a_Genuine_Loss_to_Political_Philosophy_Part_I
Mark
Thanks for your comment Mark. Also an interesting story Mark. It took me a couple takes on your opening sentence, but I realised soon enough that you were talking about a stowaway before the identity of Pepper was made. Then I saw the connection with the tail end, no pun, of my post. “Generative Grammar” comes up on Wiki and is good enough. I also use DuckDuckGo and cannot find a reference to my original Chomsky material. It was a book I once owned but which has disappeared. It was his Class 4 syntax from memory that intrigued me and led to my using language as an example of another evolutionary process.
AI is a phenomenon hard to get a philosophical take on. Chatterbox? Maybe that sums it up, but AI has a significant influence and will undoubtedly have more as it becomes further entrenched in society. Do artificially generated neural pathways matter as much as its proponents would have the rest of us believe? The universe is all connected. It constitutes one hell of a mass of neural connections which dwarfs anything humanity may concoct. What of AI if we pull the plug on it? Ultimately it is the peoples’ choice. The current crisis facing the West is nothing to do with the advent of AI, global warming or a shortage of fossil
fuels, but an existential crisis in the face of a global challenge from growing competition for natural resources/room to live.
The most pressing problem in the West is a lack of effective political engagement. One can blame the welfare state, the pan-national corporates and the political elites who long lost the plot, but in fact, it is the demo, the people, who are sorely lacking, lacking in nous. There is a clear lack of awareness amongst those who should know better. The electorates in the West have been doped up by comfortable living and political indoctrination. Will they wake up in time to save themselves?
Once I read that the world economic system is the most complex thing outside the human brain. Your comment suggests that everything is connected in the universe. Connected or not, the universe must be the most complex thing since it contains all of our brains. It is impossible (so far) to know what the big bang was. Not what it was immediately afterwards, we know that, but immediately before or at the bang instant. If it was a burst of energy that evolved, devolved, turned into, all the matter and energy that exists now, then everything arose from that initial energy bubble, so is connected in that sense. So I agree with you in that way.
As for the current wave of AI, it is fascinating, impressive, but I have commented here and to anyone who listens to me, that I do not see it as a human-like intelligence despite its familiarity. I describe it as a very good search engine with an even better natural language interface. To me it is firstly not AI which is much more than a conversation machine (vision, touch taste, planning, interpreting, problem solving, and all the things a brain does to help its owner to survive in a Darwinian world). And secondly it is impressive to me in the sense of how a calculator does maths a million times quicker and better than a human brain. But we do not think of a calculator as intelligence. I’m impressed with calculators (which includes computers), as I am with AI chatbots, but do not believe neural nets can generate intelligence like our biological nets do.
That said, two of my heroes, Marvin Minsky and Seymour Papert, both pioneers of AI, once said a similar thing about early neural nets, which as I said in my blog used to be called Perceptrons. They were wrong. (There has been a small reinterpretation more recently, but they were wrong.) That thesis of theirs was responsible for putting the development of AI back a decade because their influence turned everyone off neural net research.