If you dono t have prior experience with neural nets, look in o additional informationo down below.







A while ago, I wanted to build a neural net in Minecraft. Unfortunately, I was unable to do so. The plan was to build a conventional neural net, which learns by adjusting connection strengths. This had many problems, because it was difficult to add the values of dozens of connections. Only one number could be added at a time, so it would have been painfully slow.



Io m now going to try to create a different type of neural net, which builds (or allows data to pass through) new connections to learn rather than adjusting weights. The threshold represents the number of active connections required for the output to be on.



The new redstone update makes this much easier. One new option is to use weighted pressure plates. For each neuron, a dispenser would shoot an item onto a pressure plate. Once the pressure plate has enough items on it, the signal strength would be strong enough for the neuron to activate.



To clear items from pressure plates, monsters would pick up items and then be killed.



Everything would be coordinated using a clock. Every cycle, neurons either fire (activate) or not.



Wireless connections will be used. Custom spawners send mobs to far away neuron, which activate pressure plates to represent a signal.



Now that it is much easier to create neural nets in Minecraft, I can focus on functionality. The rest of this article is about that. I might not use all of these ideas, depending on the lag it would generate and how long it would take to make.



I like answering questions. :D






A Few Definitions



These sections are copied from some posts I made on another web site. I dono t want to re-write them, so Io m giving a few definitions beforehand.



1. Neurotransmitter in the context I use it means a chemical which changes the properties of many neurons at once (usually the threshold).



2. Brain waves are sudden increases in activity throughout the brain. You can ignore them, although they are slightly important to pay attention to during sleep. Io m sure they have other functions, though.







The Basic Design of the Cortex, the Intelligence Part of the Brain



Designing neural networks is like making soup. You add various things to the mix, which mix together to make good soup. Obviously, there is some kind of base. For example, tomato soup could be the base. After weo ve got the base, we add whatever else we wanto onions, potatoes, salt, or whatever. (Io ve never made soup, so this is an inaccurate soup-making guide.)



The Baseo Hebbian Learning



Hebbian learning is surprisingly simple. Whenever a neuron fires, it connects to other neurons which are firing at the same time. To paraphrase a Wikipedia article, o neurons which fire together wire together.o



Ingredient #1o Distributed Attribute-Based Representation
Neurons have names. For example, if you were looking at a dog, a neuron which represents o livingo would activate, as well as a neuron which represents o browno . Together, all the neurons which fire when you see a dog would represent o dogo . I should note that neurons typically represent something much simpler. A neuron can represent anything from an input of the retina, to a corner, all the way up to pieces of perception and thought.



Ingredient #2o Hierarchy
The neurons are arranged in layers, stacked like pancakes. The lowest layer receives the input, and then each layer sends data to the next one up.
Each layer contains neurons which represent more complex attributes than the last. This is because neurons can only connect to neurons in the next highest layer (using the Hebbian algorithm.)



The detailed reasoning for this difficult to explain, so Io ll use an example. Leto s say neuron A is one layer lower than neuron B. Neuron A represents a head, and neuron B represents a dog. Whenever you see a dog, neuron B will fire because it represents a dog. Neuron A will also fire, because dogs have heads. They will both fire at the same time, so neuron A will connect to neuron B, and youo ll have learned that dogs have heads.



The end result of what I just explained is learning based on what is experienced. There are still several issues, such as naming neurons, and deciding what to do with the output (mental state) of the highest layer. To make decisions, basic goals are necessary. Everything we do is based on one basic goalo be happy (which is influenced by specific things such as eating, achieving, etc.)







Naming Neurons of the Cortex



Naming neurons is essential. I dono t know if the brain uses this method to name neurons, but the RedSlime will.



When the brain first develops, each neuron starts with some random connections. Those random connections define what each neuron means. The problem is that each neuron will activate at times inconsistent with what it actually means, because it could mean o meowo yet be associated with o dogo . Whenever you hear a meow, you would be more likely to think of a dog, not a cat.



This problem is soon remedied. Every time you hear a meow, you see a cat. The Hebbian aspect of learning means that you will begin to associate o meowo with o cat,o and not with o dogo . So neurons start with a slightly incorrect name (o dogo = o legso and o big black wet noseo and o meowo ), and are adjusted to have correct name. Any connections inconsistent with the overall meaning of the other connections are destroyed, and new connections are formed to create a more precise meaning.




Coordinating Names Between the Cortex and Other Parts of the Brain



Neurons must activate each-other at the correct time. That is, they must o knowo each-othero s names.
Coordinating Names in the Cortexo an Example of Name Coordination:



In this method, the actual names of neurons are not determined. Instead, their implications are determined. For example, o 4o and o +o and o 3o has the implication o 7,o which is learned because those attributes are associated with 7. (Recall Hebbian learning- associating things with other things, based on when they are an attribute of what is being perceived.)
This type of implication is entirely within the part of the brain which I talk about most- the cortex. This type of implication is pretty much what I explained in the Basic Design of the Cortex section. The point is that the neurons of the cortex dono t know what the names of other neurons mean. They simply connect based on Hebbian learning.
More precise methods for name coordination are used in the next section.






Behavior and Decision Making



Goals are required for behavior. Everything a person does is an attempt to achieve a set of basic goals encoded within their brain.



Simply encoding the goals isno t enough. The cortex must try to achieve those goals. There are several mechanisms which cause the brain to try to achieve its goals. The reward system is major, but I dono t know how rewards work. Rather than creating rewards and reinforcing behavior, you could cause something similar to cravings, and reinforce behavior using an indirect method which I will explain. This idea relies heavily on prediction.



Here are some vital mechanisms:



1. Whenever a goal is achieved, dopamine is produced. Dopamine is also produced when a goal is imagined to be achieved (which is more important in this article). This is easy to do, because neurons in their predictive state can be considered active, and treated the same as neurons in their normal active state. This is extremely important for dopamineo s role in this system.



2. Dopamine increases the strength of imagination signals, and decreases the strength of normal signals. If dopamine levels are high enough, your brain will keep imagining an action being considered, and probably execute the action.



3. Prediction causes the brain to try to achieve secondary goals (which will aid the actual, primary, goals in the future), because imagining executing a secondary goal causes the brain to imagine executing a primary goal, which essentially makes it motivated or unmotivated.



4. Norepinephrine has the opposite effect of dopamine. It causes you to think about something else, by inhibiting the imagination connections.







The diagram in the pictures section shows the connections between each part. Io ll analyze some parts and some connections here, to explain how this system works in more detail. It's written confusingly, so you might want to skip to the last bit, which is a generalized summary.



Norepinephrine + Dopamine Neurons



Each neuron of this part represents a reason to be motivated (dopaminergic neurons) or a reason to be unmotivated (norepinephrinergic neurons). It isno t necessary for neurons in this part to interact.



Goals Encoded



This part encodes the goals. It activates dopaminergic or norepinephrinergic neurons based on the input alone. It doesno t take into account data from the cortex, which is done through a different mechanism.



Norepinephrine + Dopamine Neurons to Cortex



This connection controls how powerful the imagination connections are. It doesno t send signals to specific neurons. Instead, it modulates the activity of all cortical neurons with dopamine and norepinephrine. The level of each neurotransmitter depends on how many neurons of that neurotransmitter are active.



Cortex to Norepinephrine + Dopamine Neurons



This connection is vital for learning secondary goals. Recall that the brain doesno t know what each output of the cortex means, but instead uses an indirect method.



This method is difficult to explain, but it is simple once understood. The N. and D. neurons will activate at the same time as certain neurons in the highest cortical level (which send outputs), because activity of both parts generally corresponds to certain inputs. This fact means that the highest-level neurons will send connections to the N. and D. neurons which should be excited by what the neuron represents, because of Hebbian learning.



Norepinephrine also reduces the thresholds of cortex neurons. Io ll explain why while I explain the next connection.



This method causes the brain to try to achieve primary goals, and therefore secondary goals. You can think of the o Goals Encodedo as representing the goodness of the current input, and the cortex output as being the future input, which is essentially analyzed by the o Goals Encodedo as well, but indirectly.



Norepinephrine + Dopamine Neurons to Output



This connection makes the brain less able to do something when dopamine levels are low. If there is enough cortical activity, then the brain still outputs a signal.



Norepinephrine doesno t make a difference here. When norepinephrine levels are high, cortical activity increases, so the brain can think of a way to solve the problem. If both dopamine and norepinephrine levels are high, the brain should be able to execute the action. Dopamine doesno t represent happiness; it represents a good idea. Norepinephrine represents both negativity of an idea and the need to find a good but experimental idea.



Cortex to Output



It probably seems strange to use Hebbian connecting between the cortex and output. This is effective because of other parts of the brain. When the brain is motivated, the output neurons are more likely to be active, and neurons of the mental state connect to active output neurons. This means cortical neurons will cause a beneficial output.



Note that starting with random connections would be useful, though unnecessary, because neurons connect to random neurons Hebbianly, not all of them.



To help speed up this process, output neurons could Hebbianly connect to each-other. There could also be random starting connections, which would be tested during early development.






Overall, this method means the brain tries to achieve certain goals, both through primary goals and secondary goals which it creates itself.



This section was confusing. All of the mechanisms should work well, except for the output method. It isno t very effective, so Io ll look for a different method.







Sleep



Sleep has many functions, such as restoring the immune system and healing. The function Io ll focus on here is brain activity during sleep, especially neurotransmitter levels (things which change the properties of many neurons at once). Brain activity during sleep is geared towards learning.
Sleep is divided into stages. During each stage, brain activity levels are in waves of differing frequency. This is a very rough generalization, and during each stage brain activity constantly changes:
1: 11-16 hertz (brain waves per seconds)
2: 0.5-2 hertz at least 1/5 of the time, becoming more frequent over time, and high amplitude (slower and more synchronized, synchronized meaning more neurons at once.)
3: 11-16 hertz
4: REM sleep (over 13 hertz, with more memorable dreams than the other stages)
5: Cycle repeats
So brain waves become slower, then faster, then slower. I think the reason for this is that during sleep the brain experiments with new ideaso decisions, connections between memories, etc.o and tests these new ideas.
But how does it generate these new ideas to test? I dono t know for sure, but it might have something to do with one of the earliest parts of the brain- the pons. During sleep, the pons sends random data to the cortex. Random date may sound pointless, but perhaps it is like looking at scribbles and trying to see faces. The cortex interprets each new scribble it sees based on the previous scribble (because the cortex warps data towards what it thought would come next.)
To make this process more effective, neurotransmitters are used. Although there are many neurotransmitters, Io ll focus on a couple key neurotransmitters.
Serotonin:
Serotonin levels are high during all stages, except for REM sleep. It is excitatory to neurons in the cortex, which is highly beneficial. Before I explain why high serotonin levels are beneficial, it is necessary to explain how the cortex learns.
The cortex learns using a Hebbian algorithm, which means that active neurons connect to some other active neurons (makes those neurons more likely to activate when the neuron is on.) This creates knowledge. Each neuron represents an attribute of what the brain is sensing, so connecting neurons which are directly activated by inputs to higher level neurons (represent higher abstractions) adds knowledge. This is somewhat hard to explain, so if you dono t understand this rambling either trust me or look up Jeff Hawkins/Numenta on Youtube.
Now that thato s been explained, leto s return to serotonin, which is excitatory to neurons in the cortex. Unlike some neurotransmitters, serotonin is distributed to large areas of the brain (whereas some neurotransmitters are exchanged between single neurons.) I like to think of this as changing the properties of many neurons at once. I think serotonin simply lowers the threshold of every neuron during sleep. This means that neurons which would be almost active without the serotonin will be active with the serotonin, and so connect to other neurons. This is like saying o Neurons which are almost active would probably be active if they knew more connections, so weo ll activate nearly-active neurons.o This is a form of exploratory learning, which could be a detriment rather than a benefit if used during wakefulness.
Acetylcholine:
Acetylcholine levels are high during REM sleep, when the exploratory changes of non-REM sleep are fine-tuned. Like serotonin, acetylcholine is a o wide-area neurotransmittero . However, it affects neurons in a more complex way than serotonin. Rather than affecting all neurons in one way, it affects different types of neurons differently.
The neurons which act as inputs to the neocortex are inhibited (their thresholds are increased) by acetylcholine. This is like filtering out weak inputs. Acetylcholine also increases the depolarization time of neurons, meaning that acetylcholine makes neurons collect data for a longer time before firing. The overall effect of high acetylcholine levels is the ignorance of details.
By ignoring the many details of random input from the pons, more realistic dreams are produced. This fine tunes neurons by testing the exploratory changes of non-REM sleep in more realistic dreams.
Reasons for different brain wave frequencies during different stages of sleep:
Brains waves during non-REM sleep are low frequency and highly synchronized. This means that neurons will be activated by more neurons at a time, when a brain wave occurs. This has a similar effect to serotonin. There are probably other purposes. For example, more synchronization means the random dreams of non-REM sleep will cause less problems. (Thato s sort of hard to explain without explaining how the cortex works.)
Brain waves during REM sleep are high frequency and less synchronized. Synchronization isno t necessary, because neurons shouldno t be activated more heavily than normal. Because synchronization isno t necessary, parts of the cortex can go at their own speed, increasing the frequency of brain waves. In addition, dreams will be more realistic, so synchronization isno t as important.
The overall result of sleep is faster learning.



I recently found out that serotonin levels are normal during NREM, and acetylcholine is slightly high during REM (I think). Some of the same ideas apply, though, and I'm going to use the same methods I talked about.







Applying These Ideas to Minecraft



To be created! Some things are explained in the first section.

Additional Notes

If you don't have prior experience with neural networks, I suggest looking up "artificial neural network". You'll learn about neural nets in general, which is easier than learning about HTM first (which this will be based on.)
The next step is to search "Numenta" or "Jeff Hawkins" on youtube. It's really confusing, but hopefully it will be understandable in combination with my articles about Hebbian learning and maybe neuron naming.
To me, the brain is amazing. Everything humans do is because of the brain. This revolution will let us create truly intelligent machines.