Display options
Cite
Baranes AF, Oudeyer PY, Gottlieb J. The effects of task difficulty, novelty and the size of the search space on intrinsically motivated exploration. Front Neurosci. 2014;8:317doi: 10.3389/fnins.2014.00317.
Baranes, A. F., Oudeyer, P. Y., & Gottlieb, J. (2014). The effects of task difficulty, novelty and the size of the search space on intrinsically motivated exploration. Frontiers in neuroscience, 8317. https://doi.org/10.3389/fnins.2014.00317
Baranes, Adrien F, et al. "The effects of task difficulty, novelty and the size of the search space on intrinsically motivated exploration." Frontiers in neuroscience vol. 8 (2014): 317. doi: https://doi.org/10.3389/fnins.2014.00317
Baranes AF, Oudeyer PY, Gottlieb J. The effects of task difficulty, novelty and the size of the search space on intrinsically motivated exploration. Front Neurosci. 2014 Oct 14;8:317. doi: 10.3389/fnins.2014.00317. eCollection 2014. PMID: 25352771; PMCID: PMC4196545.
Copy Download .nbib Format: NLM
Share it on

Front Neurosci. 2014 Oct 14;8:317. doi: 10.3389/fnins.2014.00317. eCollection 2014.

The effects of task difficulty, novelty and the size of the search space on intrinsically motivated exploration.

Frontiers in neuroscience

Adrien F Baranes, Pierre-Yves Oudeyer, Jacqueline Gottlieb

Affiliations

  1. Department of Neuroscience, Columbia University Medical Center New York, NY, USA.
  2. Inria Bordeaux, France ; Ensta ParisTech Paris, France.
  3. Department of Neuroscience, Columbia University Medical Center New York, NY, USA ; Kavli Institute for Brain Science, Columbia University New York, NY, USA.

PMID: 25352771 PMCID: PMC4196545 DOI: 10.3389/fnins.2014.00317

Abstract

Devising efficient strategies for exploration in large open-ended spaces is one of the most difficult computational problems of intelligent organisms. Because the available rewards are ambiguous or unknown during the exploratory phase, subjects must act in intrinsically motivated fashion. However, a vast majority of behavioral and neural studies to date have focused on decision making in reward-based tasks, and the rules guiding intrinsically motivated exploration remain largely unknown. To examine this question we developed a paradigm for systematically testing the choices of human observers in a free play context. Adult subjects played a series of short computer games of variable difficulty, and freely choose which game they wished to sample without external guidance or physical rewards. Subjects performed the task in three distinct conditions where they sampled from a small or a large choice set (7 vs. 64 possible levels of difficulty), and where they did or did not have the possibility to sample new games at a constant level of difficulty. We show that despite the absence of external constraints, the subjects spontaneously adopted a structured exploration strategy whereby they (1) started with easier games and progressed to more difficult games, (2) sampled the entire choice set including extremely difficult games that could not be learnt, (3) repeated moderately and high difficulty games much more frequently than was predicted by chance, and (4) had higher repetition rates and chose higher speeds if they could generate new sequences at a constant level of difficulty. The results suggest that intrinsically motivated exploration is shaped by several factors including task difficulty, novelty and the size of the choice set, and these come into play to serve two internal goals-maximize the subjects' knowledge of the available tasks (exploring the limits of the task set), and maximize their competence (performance and skills) across the task set.

Keywords: decision making; exploration; intrinsic motivation; novelty; video games

References

  1. Cereb Cortex. 2015 May;25(5):1241-51 - PubMed
  2. Psychol Sci. 2009 Aug;20(8):963-73 - PubMed
  3. Proc Natl Acad Sci U S A. 2010 Dec 7;107(49):20911-6 - PubMed
  4. Conscious Cogn. 2012 Mar;21(1):422-30 - PubMed
  5. Neurosci Res. 2012 May;73(1):68-72 - PubMed
  6. Philos Trans R Soc Lond B Biol Sci. 2012 May 19;367(1594):1280-6 - PubMed
  7. Trends Cogn Sci. 2013 Nov;17(11):585-93 - PubMed
  8. PLoS One. 2012;7(5):e36399 - PubMed
  9. Pers Soc Psychol Bull. 2012 Mar;38(3):317-30 - PubMed
  10. Pers Soc Psychol Bull. 2008 Feb;34(2):196-209 - PubMed
  11. Neurosci Biobehav Rev. 2010 Apr;34(5):660-9 - PubMed
  12. Cognition. 2014 Sep;132(3):335-41 - PubMed
  13. J Neurosci. 2014 Feb 5;34(6):2148-54 - PubMed
  14. Exp Brain Res. 2014 Jul;232(7):2167-77 - PubMed
  15. Am Psychol. 2000 Jan;55(1):68-78 - PubMed
  16. J Neurosci. 2014 Jun 4;34(23):7947-57 - PubMed
  17. Top Cogn Sci. 2016 Apr;8(2):492-502 - PubMed
  18. Front Neurorobot. 2007 Nov 02;1:6 - PubMed
  19. Neuroimage. 2012 Jul 2;61(3):723-9 - PubMed
  20. J Neurosci. 2012 May 2;32(18):6117-25 - PubMed
  21. Neuroimage. 2014 Feb 1;86:194-202 - PubMed

Publication Types