Learning as Bayesian inference. (360G-Wellcome-110114_Z_15_Z)

Organisms face a hard problem: based on noisy sensory input, they must set a large number of synaptic weights. However, they do not receive enough information in their lifetime to learn the optimal weights (i.e., the weights that ensure that the circuit, system, and ultimately organism, functions as effectively as possible). In this kind of high noise regime, it is advantageous to compute a probability distribution over the weights, rather than just a point estimate (as is done under standard le arning rules). This would allow synapses to efficiently use incoming information, greatly improving learning. In addition, it would allow organisms to match learning rates to incoming information -- speeding up learning when the rate of new information is high, and slowing it down when the rate is low. Here we explore the hypothesis that synapses do compute, approximately, probability distributions over their weights. This leads to three specific questions: 1. What are the learning rules u nder the assumption that synaptic weights keep track of probability distributions? 2. What is the effect of these learning rules on large networks of spiking neurons? 3. How can we test experimentally whether these learning rules are the ones used by the brain? To address these questions, we will derive update rules for the probability distribution over synaptic weights using rules of probabilistic inference, and analyze, both theoretically and through simulations, the effect of those l earning rules on network behavior. We will then generate specific hypotheses, with the goal of collaborating closely with experimentalists to test them; those tests should result in further refinement.