A Start to Artificial Conciousness - Making A Computer Worry With Machine Learning

Bring things spring from small seeds. This is the thought that keeps running through my mind when I think of Artificial Consciousness in computers. Ever since I saw the Imitation Game and the story of Alan Turing, I wondered how such an intelligent man could think that computers could think.  Of course he stipulated that the thought process was different, than in humans.

Then along came Dr. Stephen Thaler who artificially introduced the idea of perturbations in computer "thinking" and got a patent for it. A perturbation is essential to artificial consciousness.  Essentially, a computer is programmed to linearly follow an execution path of its program. Even in artificial neural networks, the output of one perpceptron is fed into another layer.  It is a linear defined path.  Thaler introduced perturbations by selectively killing off perceptrons in a layer and in what he describes as artificial neuronal near-death, and the machine becomes a creative design machine.  His landmark example was a neural network that identified coffee cups and when it was brain-damaged, it came up with creative coffee cup designs.

Perturbations can come from all things. They can come from random events. But in the state of consciousness of any sentient thing or being (notice I now have to add things, because computers have the ability to become sentient), perturbations can come from the state of consciousness itself.  A prime example is worry. We observe something within our conscious sphere, and we think about it, make a judgement about it, add the judgement to the thought process, and keep recycling the thought in an obsessive compulsive state, and you have what is known as worry.

Going back to the opening statements of big things spring from small seeds, the thought struck me that I could make my computer worry.  It would be a small worry program to start with, but then I could hop it up to another layer of abstraction and make a universal computer worry module that could become part of the Operating System.  It would be the Worry Service.

Here's how a simple version would work. The Worry Service runs a task manager, a memory monitor or a CPU usage monitor in the background.  The minute that it detects that memory or CPU is approaching 100% or saturation, it kicks the worry.exe module. The worry module essentially assumes the highest thread priority, prints out on any display saying "I am incredibly busy" and deallocates processing priority to the heavy task, slowing it down.  It then detects that the task is slowed down, and kicks another worry module about its lack of performance.  The worry modules are able to be queried, and their response is always "I am incredibly busy and it is affecting my performance".  The worry module also writes to every log that it can, and using a machine learning neural network, reinforces the worry parameters so they automatically fire at lower thresholds.  Once the busy task is completed, the worry abates and slows down, and the computer becomes efficient again.  Of course, if the machine-learning neural network is too effective and eager in kicking in the worry, it becomes a compulsive worry, and needs to see a programming computer psychiatrist to up the thresholds of its worry mechanism by running a few positive reinforcement training epochs.  All of this technology is available now.

But I can just see it. Some Goth programmer will chain all of the worry modules into the depression module, making the computer virtually worthless for sustained work.

The very first step to scary artificial intelligence, is making a computer with the ability to navel gaze. This is a start. I am convinced that human consciousness is merely an accident of an over-developed tropism, and the evolution of Artificial Consciousness can start with this simple step -- a computer worry wart. Windows machines will be the worst worry warts and the most depressive among the conscious computers.

How To Create Computational Creativity Without Killing Anything

In a previous blog post, I outlined some ideas on Computational Creativity, and the seminal work of Dr. Stephen Thaler. You can read it HERE.  What Dr. Thaler did, was create neural nets, trained them to do things like recognize coffee cups, and then created a layer of supervisory neural nets to watch the nets.  Then he would bring them to near death by killing a pile of the neurons in the layers.  In very anthropomorphic terms, the neural network in paroxysms of Near Death, would create unique designs of other coffee cups. He called this process the Creativity Machine and it was some of the first steps in Computational Creativity using Artificial Neural Nets.

What Thaler was doing by formulating a mechanism for the Eureka moment, was to create the impetus, spark and ignition of thoughts from a machine was was programmed not to think outside the box, but to slavish following a compiled set of instructions in its register stack.  His unique algorithm was to produce a perturbation in the execution of the neural network process to create a confabulation or false idea that would be new and unique.  For the time, (and it still may a valid algorithm), it was quite revolutionary.  The problem to solve, was to have to find some way to spark new information synthesis out of pre-programmed siliconized transistor pathways. After all, ideas just can't pop into a computer's circuits.

Our brains have massively parallel neural nets and just thinking about anything sparks new thoughts.  Our thinking processes undergo a perturbation of essentially interrupt vectors in staid ways of thinking. That was Thaler was looking for inside the computer, when he started the practice of committing neuroncide, and killing neurons.

In another blog article, where I try to link synaptic pruning as a method of creating perturbations in Artificial Neural Networks ( HERE ), I came up with the idea of crippling instead of killing the neurons by pruning some random inputs in the layers. I haven't tested it yet. I don't think that the resultant "ideas" or designs would be as far-out or as revolutionary as Thaler's killings, but it might prove useful.  That idea has yet to be tried.

Then it struck me, that perhaps brain damage isn't a viable algorithm in the long term.  Even though creativity can be brainlessly expressed when monkeys finger-paint and elephants do the Picasso thing with their trunks, one would want brains, even artificial ones, with all of their faculties for serious creative thought.  So there has to be a better way than Thaler's, without killing anything.

If you want to avoid killing, and near-death experience just to create something, you still need the perturbations in regularized logic activity of artificial neural networks.  Otherwise you would just get the outputs that the neural nets were trained for.  However to Thaler's credit, he did introduce another mechanism that can be useful in creating these perturbations in Artificial Neural Networks in producing unique thoughts, and that is the supervisory network atop the thinking network.

In a future blog post, I will outline how I think that supervisory networks can contribute to Machine Consciousness, but for now, they can be integrated for non-death perturbations and idea creation in a new breed of Creativity Machines.

First let's look at a simple artificial neuron:

(I stole this gif from Thaler's website:  http://imagination-engines.com/ )

By adjusting the weights and thresholds,the simple neuron is one or two of the Boolean Gates of Knowledge that computers are made of. It can be and AND Gate or an OR Gate. In this case the weights are decimals and the inputs and outputs are integers.

There is no activation function.  Usually an activation function is like a sigmoid function.  It takes the sum of the inputs multiplied by their weights and after calculating the function, the output values are usually between -2 and +2 and the activation threshold is when the curve of the function is some value > 0.

 If the threshold value for the neuron firing is set at say 0.9 or almost one, then then anything below that is ignored and the neuron doesn't fire.  But that doesn't mean that the activation function is quiescent.  It still calculates and spits out numbers, generally in a small range between -2 and +2.  So if the activation threshold is 0.9 and the result of the sigmoid function is say 0.6, it will not activated the neuron. But we could say that the neuron is in an "excited" state because the output value of the sigmoid function is near the firing threshold.  It is just on the cusp of firing. This excited state could be used as a perturbation to excite unique thoughts.  This is where the supervisory network comes in.

A supervisory circuit can be a lot more powerful than Thaler envisioned.  First of all, supervisory circuits overlaid on top of artificial neural networks placed in an n-tier of recursive monitoring are the first steps to machine consciousness.  More on that in future blog posts.

But suppose that an independently trained ANN is monitoring other circuits for semi-excited layers or neurons, and reached out creating a synaptic link to these excited neurons.  This may or may not cause the supervisory circuit to breach its firing thresholds, and get an output where none was expected.  And the discovery of the unique ideation,  is predicated on the model by Mother Nature where she plays dice and creates millions of redundant things in the event of one surviving and making something wonderful.  In a like fashion, the outputs of all networks could be ANDed or ORed with another supervisory network monitoring for unique things, and the stimulation and simultaneous firing would cause perturbations and new ideas from two unrelated neural networks.

  That would be the mechanism for a perturbation and confabulation of two fixed networks coming up with a new idea without having to kill anything like connections or any neurons.  There would be no near-death creativity, just a flash in the pan triggered by something that just might turn out to be useful.  A pseudo-schematic is shown below:

Our human brains operate on a massively parallel neural network.  This concept is a bit of bio-mimicry that extends that.

The concept of killing brain cells in the name of creativity is not exactly new in the biological world as well.  We apparently kill thousands of brain cells with an alcoholic drink or a few puffs on joint. Many people say that this is the key to creativity.  After all, Hemingway wrote his opus half-drunk and won the Nobel Prize for Literature.  However there are millions who drink and don't turn out anything creative except for a bum's life on the Nickel, sleeping between parked cars in the rain. But je digress.

So all in all, I think that this could be an alternative method for machines to dream up new ideas in the realm of Computational Creativity.  It may not be as much fun as watching things gasp out creative in their death throes, but it could be more reliable and ultimately less destructive to some perfect good artificial neural nets.

Concepts to Cogitate

(click on pic to get the whole nine yards)

I started working with graph databases this morning.  Life is sometimes a graph.  (http://en.wikipedia.org/wiki/Graph_(abstract_data_type)  It struck me that a graph database can describe complex and fuzzy relationships.  http://en.wikipedia.org/wiki/Graph_database  Here is the rub though -- it takes a human to interpret these fuzzy relationships.  A graph database can store them and retrieve them and make machine inferences using them, but it really can't understand them.

Scroll down to the blog entry below on Machine Consciousness and you will get the drift of what I am talking about.  I met my friend Al at the venue pictured above, and he asked a cogent, rhetorical question:

"At what point does logic become intelligence?"

It was an amazing insight into the foundations of intelligence and quite a cogent one.  Without logic, there is no intelligence.  It is almost a precursor.  To a sophisticated state machine, there is lots of logic. What is required to make the leap to intelligence?  Interesting, and to be continued.

Musings on Machine Consciousness or Artificial Consciousness

(click on photograph to enlarge)

I was walking along a country road and I came upon this scene picture above.  The picture really does no justice to the rolling hils, the lines, folds and textures as well as the light and dark areas.  As you eye takes in the panorama in real life, you realize that it is a magical place that makes you feel numinous.

And right in front of me, was a guy with a huge piece of paper, trying to capture the scene with a charcoal sketch in preparation for a painting.  The back of his VW stationwagon was lifted open and sketches were scattered all over the car.  I peeked at his sketches, and we started talking.

As it turned out, he was from Australia,  an ex-computer engineer, a mainframe guy who was now an artist and had been for over 20 years.  He earns his living with his art.

We got to talking about stuff, computers and such, and the magnificent scene in front of us, and the impossibility of reproducing and how it made us both feel numinous and from there we got on to consciousness.

I brought up the topic that Ray Kurzweil had predicted a machine with a soul with his book "The Age of A Spiritual Machine".  My new-friend begged to differ, citing that we really didn't know what consciousness was, so how could we emulate or even create it on doped silicon.

I wanted to beg to differ with the artist.  My own neural nets tell me that consciousness is an over-developed tropism.  Plants started it with phototropism for the leaves and geotropism for the roots, and after several hundred million years of evolution gone wild, the tropisms developed into senses, the brain evolved to process the sensory stimuli and it didn't stop.  We developed the ability to abstract, to think, and to integrate the sensory data to knowledge and preserve and pass it on to our species.

This got me to thinking about how a primitive artificial consciousness would be formulated in a computer.  We already have the prototypes with QoS circuits in servers monitored Quality of Service.  We have a heartbeat, plus all sorts of monitors that read and report on processes, cycles, peripherals, services and such.  Right now, these QoS circuits report to humans.  They would have to report to the machine itself.  And the machine would have to react to them.

How it reacts is the tricky part.  In humans, we react to our sensory input by abstracting it to a higher plane.  If for example, our heart skips a beat, we start worrying about a heart attack.  What it means for machine consciousness, is that the data integration into information and finally stringing information into knowledge has to be developed.  As a further step, knowledge must be abstracted into an ideal that can be applied to many other things.  For example, any child knows the following ideal:

Using this ideal, a child can match the above to this:

And with even a bit of cogitation, a child can match the abstract with this:

So, just to recap to this point, a computer would need sensory inputs from everything about itself, hardware and software, and then it would need some powerful inference tools to sort the Big Data from its sensors into information, and from there in knowledge and from there, into an abstract ideal, or an interface if you will, in programming language.

We haven't touched the world outside the computer yet.  But we are not done.  Once we have the knowledge bits, we need some emotions bits if we are to match the human experience.  I'm not saying that we really need that.  The computer conscious could be a Spock-on-steroids who has no emotions, but that wouldn't be fun, would it?

Emotions need qualitative judgements and reactions to those qualitative judgements.  In a human being, if we are worried, our performance is impaired.  If we were to match this in a machine, worry would spawn threads that would impair performance based on whether the machine was bummed out or not.  But how would a machine express happiness?  Would it be by innocuous things like playing mp3s when it should be doing floating point co-processing?

My Australian friend said that we would never develop Artificial Consciousness.  Moi, I say that what I have brought up are mere details.  There will be frameworks upon frameworks that will give the computer a good idea of what is going on, how the computer should react, and that there is a whole other world outside the binary bits and bytes of the doped Gallium Arsenide matrices and PNP and NPN junctions in its transistor neurons. It's a scary thought when a computer first comes to the realization of an outside world, but damn, it would be exciting.   More to come.