A Growing Long-term Episodic & Semantic Memory

The long-term memory of most connectionist systems lies entirely in the weights
of the system. Since the number of weights is typically fixed, this bounds the
total amount of knowledge that can be learned and stored. Though this is not
normally a problem for a neural network designed for a specific task, such a bound
is undesirable for a system that continually learns over an open range of domains.
To address this, we describe a lifelong learning system that leverages a fast, though
non-differentiable, content-addressable memory which can be exploited to encode
both a long history of sequential episodic knowledge and semantic knowledge over
many episodes for an unbounded number of domains. This opens the door for
investigation into transfer learning, and leveraging prior knowledge that has been
learned over a lifetime of experiences to new domains.

• Machine Intelligence