On A Modern Self-Referential Weight Matrix That Learns to Modify Itself

This is an interesting and unusual research paper by Jürgen Schmidhuber.

Here the weights of the neural network are not frozen in time after training, but are self improved after training.

From the abstract:

"... The WM [weight matrix] of a self-referential NN, however, can keep rapidly modifying all of itself during runtime. In principle, such NNs can meta-learn to learn ... in the sense of recursive self-improvement. While NN architectures potentially capable of implementing such behavior have been proposed since the '90s, there have been few if any practical studies. Here we revisit such NNs, building upon recent successes of fast weight programmers and closely related linear Transformers. We propose a scalable self-referential WM (SRWM) that uses outer products and the delta update rule to modify itself. We evaluate our SRWM in supervised few-shot learning and in multi-task reinforcement learning with procedurally generated game environments. Our experiments demonstrate both practical applicability and competitive performance of the proposed SRWM."

[2202.05780] A Modern Self-Referential Weight Matrix That Learns to Modify Itself