|
Poirazi, Panayiota Effect of Morphology on the Memory Capacity of Neurons With Active Dendrites Conventional notions of neural learning hold that memory state in the brain is stored primarily in the pattern of synaptic weight values, which can change on a wide range of time scales under the control of multiple biophysical mechanisms. This equation of neural learning with synaptic weight modification has been supported by (1) a vast physiological record which indicates that changes in synaptic strength can result from both pre- and post-synaptic activity, singly or in combination, and (2) connectionist theory, which has laid mathematical foundations for synaptic modification rules, and has spawned numerous practical applications based on ``neural network'' learning systems. A potential conflict exists, however, between the low-level biophysical properties of synapses in the mammalian CNS and the synaptic weights of conventional connectionist learning schemes. In particular, synaptic weights in a typical neural network are represented as high resolution analog parameters. In contrast, learning-related experimental protocols in the biological realm commonly show that the efficacy of synaptic transmission can fluctuate by a factor of two or more in response to brief trains of pre-synaptic stimuli, which may be assumed to occur frequently in vivo. Such fluctuations in synaptic weight values seem inconsistent with the connectionist assumption of stable high-resolution weights. Furthermore, a recent experimental study suggests that hippocampal synapses may exist in only a few stable states, where the continuous grading of synaptic strengths seen in measures of LTP (e.g., slope of population spike) apparently results from the averaging of many such synapses with different thresholds for learning [Malenka et al. 1998]. In brief, the restriction that synaptic weights may exist in only a few stable levels could be difficult to reconcile with conventional connectionist notions, especially when these are invoked as models for the neurobiological basis of learning and memory. Our previous work has focused on the additional memory capacity available to a neuron with active dendrites, above and beyond that which may be contained in the synaptic weights themselves [Mel, 1992ab; Poirazi & Mel, 1998]. In these previous studies, we showed that the spatial arrangement of synaptic contacts can strongly impact on the response of an active dendritic neuron, and that a significant repository of information is contained in nonlinear interactions among two or more input variables-sometimes referred to as higher-order ``features''. In this work, we compare quantitatively the amount of trainable memory capacity in two kinds of neurons: (1) a ``conventional'' neuron-like unit (e.g. the Perceptron) which calculates a weighted sum of its inputs before applying a single global threshold, and (2) a generalized ``clusteron'', whose output is a sum over several independently thresholded subunits. To compare the memory capacity of these two models, we derive analytical expressions for the number of distinct parameter states available to both neuron types (fig. 1), where the total number of synaptic sites and levels-per-weight is made equivalent for the two models. We show that for large d (input dimension) and k (synaptic sites per branch), the memory states available to the clusteron grow much faster than those available to a Perceptron (fig. 2). For example, for input dimension d = 100, and s = 10000 synaptic sites distributed over m = 100 branches of size k = 10, the clusteron capacity exceeds that of its Perceptron counterpart by a factor of 23. Building upon previous empirical studies of clusteron capacity [Mel 1992ab], we have confirmed that much of the additional capacity which is theoretically available to the clusteron (indicated in fig. 2) is in fact expressible using biophysically plausible learning rules. We are currently exploring this issue in greater depth using randomized local Hebb-like learning rules which effect the sorting of correlated synaptic connections onto distinct dendritic subunits. Figure 1: Bit expressions for the clusteron (1) and the Perceptron (2) models, where d is the input space dimension, m is the number of clusteron branches, k is the number of synapses per branch and s is the number of total synaptic connections, s = mk. Figure 2: The generalized clusteron bit capacity grows much faster than Perceptron capacity when the number of branches (m) is kept constant, and the number of synaptic connections (s) increases.
|
