Display options
Cite
FitzGerald TH, Dolan RJ, Friston K. Dopamine, reward learning, and active inference. Front Comput Neurosci. 2015;9:136doi: 10.3389/fncom.2015.00136.
FitzGerald, T. H., Dolan, R. J., & Friston, K. (2015). Dopamine, reward learning, and active inference. Frontiers in computational neuroscience, 9136. https://doi.org/10.3389/fncom.2015.00136
FitzGerald, Thomas H B, et al. "Dopamine, reward learning, and active inference." Frontiers in computational neuroscience vol. 9 (2015): 136. doi: https://doi.org/10.3389/fncom.2015.00136
FitzGerald TH, Dolan RJ, Friston K. Dopamine, reward learning, and active inference. Front Comput Neurosci. 2015 Nov 04;9:136. doi: 10.3389/fncom.2015.00136. eCollection 2015. PMID: 26581305; PMCID: PMC4631836.
Copy Download .nbib Format: NLM
Share it on

Front Comput Neurosci. 2015 Nov 04;9:136. doi: 10.3389/fncom.2015.00136. eCollection 2015.

Dopamine, reward learning, and active inference.

Frontiers in computational neuroscience

Thomas H B FitzGerald, Raymond J Dolan, Karl Friston

Affiliations

  1. The Wellcome Trust Centre for Neuroimaging, University College London London, UK ; Max Planck - UCL Centre for Computational Psychiatry and Ageing Research London, UK.
  2. The Wellcome Trust Centre for Neuroimaging, University College London London, UK.

PMID: 26581305 PMCID: PMC4631836 DOI: 10.3389/fncom.2015.00136

Abstract

Temporal difference learning models propose phasic dopamine signaling encodes reward prediction errors that drive learning. This is supported by studies where optogenetic stimulation of dopamine neurons can stand in lieu of actual reward. Nevertheless, a large body of data also shows that dopamine is not necessary for learning, and that dopamine depletion primarily affects task performance. We offer a resolution to this paradox based on an hypothesis that dopamine encodes the precision of beliefs about alternative actions, and thus controls the outcome-sensitivity of behavior. We extend an active inference scheme for solving Markov decision processes to include learning, and show that simulated dopamine dynamics strongly resemble those actually observed during instrumental conditioning. Furthermore, simulated dopamine depletion impairs performance but spares learning, while simulated excitation of dopamine neurons drives reward learning, through aberrant inference about outcome states. Our formal approach provides a novel and parsimonious reconciliation of apparently divergent experimental findings.

Keywords: active inference; dopamine; incentive salience; instrumental conditioning; learning; reward; reward learning; variational inference

References

  1. Nat Neurosci. 2012 Mar 11;15(5):786-91 - PubMed
  2. Nat Neurosci. 2013 Sep;16(9):1170-8 - PubMed
  3. Science. 1997 Mar 14;275(5306):1593-9 - PubMed
  4. Psychopharmacology (Berl). 2007 Apr;191(3):461-82 - PubMed
  5. Behav Brain Sci. 2013 Jun;36(3):181-204 - PubMed
  6. Biol Cybern. 2012 Oct;106(8-9):523-41 - PubMed
  7. Front Psychiatry. 2013 May 30;4:47 - PubMed
  8. Philos Trans R Soc Lond B Biol Sci. 2014 Nov 5;369(1655):null - PubMed
  9. Psychol Rev. 2014 Jul;121(3):337-66 - PubMed
  10. Psychopharmacology (Berl). 2007 Apr;191(3):433-7 - PubMed
  11. J Neurosci. 2010 Jan 20;30(3):1158-65 - PubMed
  12. PLoS One. 2013 Jun 05;8(6):e65799 - PubMed
  13. J Neurosci. 2009 Dec 2;29(48):15104-14 - PubMed
  14. Front Psychol. 2013 Oct 07;4:710 - PubMed
  15. Cogn Neurosci. 2015;6(4):187-214 - PubMed
  16. Neuron. 2013 Oct 16;80(2):312-25 - PubMed
  17. Brain. 2012 Jun;135(Pt 6):1871-83 - PubMed
  18. Nat Rev Neurosci. 2010 Feb;11(2):127-38 - PubMed
  19. Neuron. 2014 Feb 5;81(3):687-99 - PubMed
  20. PLoS Comput Biol. 2012 Jan;8(1):e1002327 - PubMed
  21. J Neurophysiol. 1998 Jul;80(1):1-27 - PubMed
  22. Trends Cogn Sci. 2012 Jan;16(1):72-80 - PubMed
  23. Proc Biol Sci. 2011 Jul 7;278(1714):2053-9 - PubMed
  24. PLoS Comput Biol. 2013;9(12):e1003383 - PubMed
  25. Front Hum Neurosci. 2011 May 02;5:39 - PubMed
  26. Behav Neurosci. 2000 Jun;114(3):468-83 - PubMed
  27. Philos Trans R Soc Lond B Biol Sci. 2014 Nov 5;369(1655):null - PubMed
  28. Nature. 2001 Sep 6;413(6851):67-70 - PubMed
  29. Neuropsychopharmacology. 2013 Jul;38(8):1495-503 - PubMed
  30. Neuron. 2011 Dec 8;72(5):721-33 - PubMed
  31. J Neurosci. 2011 May 25;31(21):7867-75 - PubMed
  32. Eur J Neurosci. 2012 Apr;35(7):1144-51 - PubMed
  33. Nat Neurosci. 2013 May;16(5):648-53 - PubMed
  34. Psychopharmacology (Berl). 2007 Apr;191(3):391-431 - PubMed
  35. Biol Cybern. 1992;66(3):241-51 - PubMed
  36. Science. 2008 Feb 29;319(5867):1264-7 - PubMed
  37. Nature. 2012 Jan 18;482(7383):85-8 - PubMed
  38. Trends Neurosci. 2013 Jun;36(6):336-42 - PubMed
  39. PLoS One. 2012;7(10):e47502 - PubMed
  40. J Neurosci. 2003 Nov 26;23(34):10827-31 - PubMed
  41. J Neurosci. 2010 Oct 6;30(40):13525-36 - PubMed
  42. Nat Neurosci. 2013 Jul;16(7):966-73 - PubMed
  43. Cereb Cortex. 2009 Feb;19(2):483-95 - PubMed
  44. Med Hypotheses. 2015 Feb;84(2):109-17 - PubMed
  45. Cereb Cortex. 2015 Oct;25(10):3434-45 - PubMed
  46. Neuron. 2014 Oct 1;84(1):177-89 - PubMed
  47. Neuropsychopharmacology. 2012 Mar;37(4):950-8 - PubMed
  48. Science. 2003 Mar 21;299(5614):1898-902 - PubMed
  49. Nature. 1996 Feb 1;379(6564):449-51 - PubMed
  50. J Neurosci. 2005 Jun 29;25(26):6235-42 - PubMed
  51. Nat Neurosci. 2007 Sep;10(9):1214-21 - PubMed
  52. Neuron. 2012 Mar 22;73(6):1173-83 - PubMed
  53. Nature. 2011 Jan 6;469(7328):53-7 - PubMed
  54. Front Hum Neurosci. 2014 Mar 25;8:160 - PubMed
  55. Cogn Affect Behav Neurosci. 2014 Jun;14(2):473-92 - PubMed
  56. Front Hum Neurosci. 2013 Sep 25;7:598 - PubMed
  57. Eur J Neurosci. 2012 Apr;35(7):1124-43 - PubMed
  58. Neuropsychopharmacology. 2014 Sep;39(10):2366-75 - PubMed
  59. Neuron. 2011 Mar 24;69(6):1204-15 - PubMed
  60. Curr Opin Neurobiol. 2011 Jun;21(3):381-6 - PubMed
  61. Neural Comput. 1995 Sep;7(5):889-904 - PubMed
  62. Front Hum Neurosci. 2014 Jun 26;8:457 - PubMed
  63. Trends Cogn Sci. 2014 Apr;18(4):194-202 - PubMed
  64. Hum Brain Mapp. 2009 Jun;30(6):1866-76 - PubMed
  65. PLoS Comput Biol. 2014 Sep 04;10(9):e1003810 - PubMed
  66. Proc Natl Acad Sci U S A. 2012 May 8;109(19):7511-6 - PubMed
  67. Science. 2004 Dec 10;306(5703):1940-3 - PubMed
  68. Curr Biol. 2012 Aug 21;22(16):1477-81 - PubMed
  69. Neuron. 2010 Jan 14;65(1):135-42 - PubMed
  70. J Neurosci. 2008 Nov 19;28(47):12294-304 - PubMed
  71. Eur J Neurosci. 2012 Aug;36(4):2521-32 - PubMed
  72. Behav Neurosci. 2005 Feb;119(1):5-15 - PubMed
  73. Science. 2009 May 22;324(5930):1080-4 - PubMed
  74. Front Neurosci. 2011 Jun 21;5:81 - PubMed
  75. Front Psychol. 2014 Feb 21;5:124 - PubMed
  76. Nature. 2006 Aug 31;442(7106):1042-5 - PubMed
  77. J Neurosci. 2011 Jul 27;31(30):10829-35 - PubMed
  78. Proc Natl Acad Sci U S A. 2014 Apr 29;111(17):6455-60 - PubMed
  79. Brain Res Brain Res Rev. 1998 Dec;28(3):309-69 - PubMed
  80. Nat Neurosci. 2007 Aug;10(8):1020-8 - PubMed
  81. Nat Rev Neurosci. 2009 Jan;10(1):48-58 - PubMed
  82. Neural Comput. 2015 Feb;27(2):306-28 - PubMed
  83. Front Neurosci. 2012 Feb 06;6:9 - PubMed
  84. Curr Biol. 2013 Feb 18;23(4):282-9 - PubMed
  85. Nat Neurosci. 2000 Nov;3 Suppl:1178-83 - PubMed
  86. Nat Neurosci. 1998 Aug;1(4):304-9 - PubMed
  87. Trends Cogn Sci. 2006 Jul;10(7):309-18 - PubMed

Publication Types

Grant support