@Tim

Nice that you picked up on the aspect of the model that it accounts for non-linear scaling of sequence lengths.

I’m not too worried about the apparent contradiction in calling the way it which the model learns “unsupervised” and describing time as the teacher. I’m using “unsupervised” in the traditional machine learning sense, referring to paradigms in which the desired outputs of the system are known in advance and the difference between actual and desired output is used to generate an error signal which is used as an update rule for learning. Though in the future it’s probably not a bad idea to explicitly state that.

And thanks for the pointer to Reverse Hierarchy Theory. I hadn’t heard of it.

@Anne

Like Hawkins’, mine is also meant to be a very simple, abstract model of neocortex.

You bring up reinforcement learning…and yes, this model could easily be augmented with a reinforcement signal.

As for the type of patterns the particular network shown can store, it actually cannot store A-B-C or D-B-A. This toy example can only store sequences of length 2, 4, and 8, such as A-B-D-C. But yes, distinct nodes are recruited to represent the forward and backward sequences, e.g. a different nodes encode for A-B and B-A. I don’t know if this answers your question. If not, please let me know.

Really interesting piece of text! Jeff Hawkins’ description is supposed to be an abstract model of the cortex. Where can I position your model in the brain? As an engineer I can imagine that there are multiple implementations of hierarchical, sequential ordering possible. One more fit then the other for certain circumstances. For example, it can even be implemented on a single neuron level to detect the time difference between sounds arriving at the “ears”, it’s one of the Izhikevich neuron types . And for example at the basal ganglia side, which is often said to implement action selection, it makes sense to have their some “internal supervised/reinforcement” signal in the sense of dopamine. And it wouldn’t be wise of evolution not to speculate about the possible consequences of its actions. So, is your model supposed to govern storing of order in the cortex?

Moreover, order is something quite peculiar, it’s so easy to get combinatorial explosions, in the case it is not recognized that there is NO order involved somewhere. If A,B,C and D are received in different orders, one at a time, while their actual occurrence is random, 4! permutations have to be received (multiple times) to comprehend that fact. So, a real-world system as the brain (on the cortex level) has probably some neural circuitry involved that selects “candidates”. I consider in this context stability-plasticity as being able to store a sequence of events and dismiss this sequence later, depending on a sort of vigilance parameter. Do you foresee some stability-plasticity solution within your hierarchy by including recurrent connections?

And actually a very stupid question. Is your hierarchy supposed to store a pattern like: A,B,C and D,B,A? So, A and B in different order depending on the context?

However, I am a layman still regarding neuroscience… I hope that will end soon! Thanks for your article,

Anne

Thanks for sharing the interesting idea. I’m particularly intrigued that your model predicts that there is at least a theoretical capacity for representations to increase exponentially in complexity per sequencing layer. If the temporal sequence learning idea is similar to more conventional receptive fields, the idea may help highlight why ‘object representations’ can already be so complex/invariant in IT (speaking from the vision paradigm, sadly the only one I know well).

I think your title is clever, but I have to admit, however, I stopped in confusion for a minute by the idea of “…unsupervised learning, using time as a teacher, …” I figure you mean that the unsupervised learning, clustering, whatever you want to call it, occurs in the temporal domain, and that is the extent of it, but still, I just wanted to give you a heads up in case someone else calls you out on that apparent paradox.

Coming from the vision paradigm again, if you’re looking for some leads on how to incorporate feedback into your model, you might want to take a look at Reverse Hierarchy Theory if you haven’t done so already. The general idea is that feedback learning will alter lower layers to better respond to the pieces that originally built the complex representation at higher layers. This would, in a sense, specialize the network but also increase the rate at which ‘word length’ increases while traveling up the model layers. This would also push the top layers to reach their maximum capacity as well; I’m envisioning something like eliminating overlaps in sequences, e.g. {(a->c->b->d), (c->b->d->e), (b->d->e->f)} into {(a->c->b->d), (b->d->e->f), (room for more…)} Then again, I don’t think RHT is the most concrete theory in the world, so it might not be worth more than a quick look.

Thanks again!

you are right on the spot with Eugene’s. He did not seem to have an answer to the question on how you stably reactivate a representation over time if you have STDP that continuously sculpt your synapses. This is what my advisor, Stephen Grossberg, calls the “stability plasticity dilemma”. It is in fact a dilemma, and I have been working on these issues for quite some time. These sort of problems are also key in our new SyNAPSE grant, and indeed in any neural circuit that has online plasticity (namely, you do not constrain learning artifically). May be Eugene has some new results on the topic? Should we ask him to do a post here???

I can write him an email!

Max

Ethan: Thanks for the reference. I’ll definitely check it out. I do hope these similar ideas on on the right track.

thanks for the very interesting post.

Some of your ideas intersect in certain aspect with Eugene Izhikevich. He visited our department about 2 years ago, and stayed for a full day, where I had the chance to appreciate his very original ideas. One of them is known with the name of polychronization. I would check it out. Some of the concepts are similar to what you propose: synaptic delays are a feature, not a defect. They are used to help patterns of invariant activation to form in random networks of spiking neurons with STDP learning. In reality, polychornization networks are everywhere… where you do not constrain networks of spiking neurons by imposing fixed delays. Even in my work I had polychonizing networks, but I did not know it until Eugene pointed it out!

You seem to add an interesting twist to this polychronization story, something in between Jeff’s and Eugene’s ideas. Very intriguing. I would be curious to learn more of…what you are learning. One big issue, which you start alluding to, is the robustness of the network. Since timing is so important, changes in stimulus (or other cell’s output) intensity is reflected in changes in timing of spike, which in turns has a profound effect on what cell learn, namely what sequence is stored via STDP.

Max