Neural Random Access Machines

Deep Neural Networks (DNNs) have achieved great success in supervised learning

tasks mainly due to their “depth”, allowing DNNs to represent functions whose

implementation requires some sequential computation, which turned out to be

sufficient to solve many previously-intractable problems.

Given that depth was a key ingredient in the success of DNNs, it is plausible

that much deeper neural models — namely, models that are computationally uni-
versal — would be able to solve much harder problems using less training data.

The Neural Turing Machine is the first neural network model of this kind, which

has been able to learn to solve a number of algorithmic tasks from input-output

examples using backpropagation. Although the Neural Turing Machine is compu-
tationally universal, it used a memory addressing mechanism that does not allow

for pointers, which makes the implementation of a number of natural algorithms

cumbersome.

In this paper, we propose and investigate a computationally universal model that

can manipulate and dereference pointers. Pointer manipulation is a natural opera-
tion, so it is interesting to determine whether they can be learned with backprop-
agation as well. We evaluate the new model on a number of simple algorithmic

tasks whose solution requires pointer manipulation and dereferencing. Our results

show that the proposed model can learn to solve algorithmic tasks of such type,

provided that they are not too difficult.

• Machine Intelligence