Home  ›  Comparing Continual Task Learning in Minds and Machines

Comparing Continual Task Learning in Minds and Machines

Written By Thursday, October 20, 2022 Add Comment Edit

Comparing continual task learning in minds and machines

https://doi.org/10.1073/pnas.1800755115

Journal: Proceedings of the National Academy of Sciences , 2018 , № 44

Publisher: Proceedings of the National Academy of Sciences

Authors: Timo Flesch, Jan Balaguer, Ronald Dekker, Hamed Nili, Christopher Summerfield

Abstract

Significance Humans learn to perform many different tasks over the lifespan, such as speaking both French and Spanish. The brain has to represent task information without mutual interference. In machine learning, this "continual learning" is a major unsolved challenge. Here, we studied the patterns of errors made by humans and state-of-the-art neural networks while they learned new tasks from scratch and without instruction. Humans, but not machines, seem to benefit from training regimes that blocked one task at a time, especially when they had a prior bias to represent stimuli in a way that encouraged task separation. Machines trained to exhibit the same prior bias suffered less interference between tasks, suggesting new avenues for solving continual learning in artificial systems.

List of references

  1. S Legg M Hutter A collection of definitions of intelligence. arXiv:10.1207/s15327051hci0301_2. Preprint posted June 25 2007. (2007).
  2. GI Parisi R Kemker JL Part C Kanan S Wermter Continual lifelong learning with neural networks: A review. arXiv:1802.07569v2. Preprint posted February 21 2018. (2018).
  3. J Kirkpatrick, , Overcoming catastrophic forgetting in neural networks. Proc Natl Acad Sci USA 114, 3521–3526 (2017).
    https://doi.org/10.1073/pnas.1611835114
  4. RM French, Catastrophic forgetting in connectionist networks. Trends Cogn Sci 3, 128–135 (1999).
    https://doi.org/10.1016/S1364-6613(99)01294-2
  5. V Mnih, , Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015).
    https://doi.org/10.1038/nature14236
  6. V Mnih Asynchronous methods for deep reinforcement learning. arXiv:1602.01783v2. Available at: arxiv.org/abs/1602.01783 [Accessed May 11 2018]. (2016).
  7. JL McClelland, BL McNaughton, RC O'Reilly, Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychol Rev 102, 419–457 (1995).
    https://doi.org/10.1037/0033-295X.102.3.419
  8. D Kumaran, D Hassabis, JL McClelland, What learning systems do intelligent agents need? Complementary learning systems theory updated. Trends Cogn Sci 20, 512–534 (2016).
    https://doi.org/10.1016/j.tics.2016.05.004
  9. T Schaul J Quan I Antonoglou D Silver Prioritized experience replay. arXiv:10.1038/nature14236. Preprint posted November 18 2015. (2015).
  10. S Goode, RA Magill, Contextual interference effects in learning three badminton serves. Res Q Exerc Sport 57, 308–314 (1986).
    https://doi.org/10.1080/02701367.1986.10608091
  11. LE Richland, JR Finley, RA Bjork, Differentiating the contextual interference effect from the spacing effect. Proceedings of the 26th Annual Meeting of the Cognitive Science Society (Lawrence Erlbaum, Mahwah, NJ), pp. 1624 (2004).
  12. D Rohrer, RF Dedrick, S Stershic, Interleaved practice improves mathematics learning. J Educ Psychol 107, 900–908 (2015).
    https://doi.org/10.1037/edu0000001
  13. N Kornell, RA Bjork, Learning concepts and categories: Is spacing the "enemy of induction"? Psychol Sci 19, 585–592 (2008).
    https://doi.org/10.1111/j.1467-9280.2008.02127.x
  14. PF Carvalho, RL Goldstone, What you learn is more than what you see: What can sequencing effects tell us about inductive category learning? Front Psychol 6, 505 (2015).
    https://doi.org/10.3389/fpsyg.2015.00505
  15. G Wulf, CH Shea, Principles derived from the study of simple skills do not generalize to complex skill learning. Psychon Bull Rev 9, 185–211 (2002).
    https://doi.org/10.3758/BF03196276
  16. PF Carvalho, RL Goldstone, Putting category learning in order: Category structure and temporal arrangement affect the benefit of interleaved over blocked study. Mem Cognit 42, 481–495 (2014).
    https://doi.org/10.3758/s13421-013-0371-0
  17. SM Noh, VX Yan, RA Bjork, WT Maddox, Optimal sequencing during category learning: Testing a dual-learning systems perspective. Cognition 155, 23–29 (2016).
    https://doi.org/10.1016/j.cognition.2016.06.007
  18. S Monsell, Task switching. Trends Cogn Sci 7, 134–140 (2003).
    https://doi.org/10.1016/S1364-6613(03)00028-7
  19. E Tulving, DM Thomson, Encoding specificity and retrieval processes in episodic memory. Psychol Rev 80, 352–373 (1973).
    https://doi.org/10.1037/h0020071
  20. M Minear, P Shah, Training and transfer effects in task switching. Mem Cognit 36, 1470–1483 (2008).
    https://doi.org/10.3758/MC.336.8.1470
  21. N Kriegeskorte, M Mur, P Bandettini, Representational similarity analysis–Connecting the branches of systems neuroscience. Front Syst Neurosci 2, 4 (2008).
  22. N Kriegeskorte, RA Kievit, Representational geometry: Integrating cognition, computation, and the brain. Trends Cogn Sci 17, 401–412 (2013).
    https://doi.org/10.1016/j.tics.2013.06.007
  23. KE Stephan, WD Penny, J Daunizeau, RJ Moran, KJ Friston, Bayesian model selection for group studies. Neuroimage 46, 1004–1017 (2009).
    https://doi.org/10.1016/j.neuroimage.2009.03.025
  24. L Rigoux, KE Stephan, KJ Friston, J Daunizeau, Bayesian model selection for group studies–Revisited. Neuroimage 84, 971–985 (2014).
    https://doi.org/10.1016/j.neuroimage.2013.08.065
  25. J Daunizeau, V Adam, L Rigoux, VBA: A probabilistic treatment of nonlinear models for neurobiological and behavioural data. PLoS Comput Biol 10, e1003441 (2014).
    https://doi.org/10.1371/journal.pcbi.1003441
  26. FG Ashby, WT Maddox, Human category learning. Annu Rev Psychol 56, 149–178 (2005).
    https://doi.org/10.1146/annurev.psych.56.091103.070217
  27. MC Hout, SD Goldinger, RW Ferguson, The versatility of SpAM: A fast, efficient, spatial method of data collection for multidimensional scaling. J Exp Psychol Gen 142, 256–281 (2013).
    https://doi.org/10.1037/a0028860
  28. N Kriegeskorte, M Mur, Inverse MDS: Inferring dissimilarity structure from multiple item arrangements. Front Psychol 3, 245 (2012).
    https://doi.org/10.3389/fpsyg.2012.00245
  29. R Ratcliff, Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psychol Rev 97, 285–308 (1990).
    https://doi.org/10.1037/0033-295X.97.2.285
  30. I Higgins Early visual concept learning with unsupervised deep learning. arXiv:1606.05579v3.Available at: arxiv.org/abs/1606.05579 [Accessed December 17 2017]. (2016).
  31. JG Bremner, AM Slater, SP Johnson, Perception of object persistence: The origins of object permanence in infancy. Child Dev Perspect 9, 7–13 (2015).
    https://doi.org/10.1111/cdep.12098
  32. M McCloskey, NJ Cohen, Catastrophic interference in connectionist networks: The sequential learning problem. Psychol Learn Motiv 24, 109–165 (1989).
    https://doi.org/10.1016/S0079-7421(08)60536-8
  33. R Goldstone, Influences of categorization on perceptual discrimination. J Exp Psychol Gen 123, 178–200 (1994).
    https://doi.org/10.1037/0096-3445.123.2.178
  34. FA Soto, FG Ashby, Categorization training increases the perceptual separability of novel dimensions. Cognition 139, 105–129 (2015).
    https://doi.org/10.1016/j.cognition.2015.02.006
  35. RL Goldstone, A Gerganov, D Landy, ME Roberts, Learning to see and conceive. Cognitive Biology (MIT Press, Cambridge, MA), pp. 163–188 (2009).
    https://doi.org/10.7551/mitpress/9780262012935.003.0153
  36. BC Love, DL Medin, TM Gureckis, SUSTAIN: A network model of category learning. Psychol Rev 111, 309–332 (2004).
    https://doi.org/10.1037/0033-295X.111.2.309
  37. MH Herzog, KC Aberg, N Frémaux, W Gerstner, H Sprekeler, Perceptual learning, roving and the unsupervised bias. Vision Res 61, 95–99 (2012).
    https://doi.org/10.1016/j.visres.2011.11.001
  38. T Qian, RN Aslin, Learning bundles of stimuli renders stimulus order as a cue, not a confound. Proc Natl Acad Sci USA 111, 14400–14405 (2014).
    https://doi.org/10.1073/pnas.1416109111
  39. ML Kalish, S Lewandowsky, JK Kruschke, Population of linear experts: Knowledge partitioning and function learning. Psychol Rev 111, 1072–1099 (2004).
    https://doi.org/10.1037/0033-295X.111.4.1072
  40. LX Yang, S Lewandowsky, Knowledge partitioning in categorization: Constraints on exemplar models. J Exp Psychol Learn Mem Cogn 30, 1045–1064 (2004).
    https://doi.org/10.1037/0278-7393.30.5.1045
  41. S Lewandowsky, L Roberts, L-X Yang, Knowledge partitioning in categorization: Boundary conditions. Mem Cognit 34, 1676–1688 (2006).
    https://doi.org/10.3758/BF03195930
  42. AGE Collins, MJ Frank, Cognitive control over learning: Creating, clustering, and generalizing task-set structure. Psychol Rev 120, 190–229 (2013).
    https://doi.org/10.1037/a0030852
  43. AGE Collins, The cost of structure learning. J Cogn Neurosci 29, 1646–1655 (2017).
    https://doi.org/10.1162/jocn_a_01128
  44. F Mathy, J Feldman, A rule-based presentation order facilitates category learning. Psychon Bull Rev 16, 1050–1057 (2009).
    https://doi.org/10.3758/PBR.16.6.1050
  45. M Rigotti, , The importance of mixed selectivity in complex cognitive tasks. Nature 497, 585–590 (2013).
    https://doi.org/10.1038/nature12160
  46. S Fusi, EK Miller, M Rigotti, Why neurons mix: High dimensionality for higher cognition. Curr Opin Neurobiol 37, 66–74 (2016).
    https://doi.org/10.1016/j.conb.2016.01.010
  47. ML Mack AR Preston BC Love Medial prefrontal cortex compresses concept representations through learning. bioRxiv:10.1101/178145. (2017).
    https://doi.org/10.1101/178145
  48. F Zenke B Poole S Ganguli Continual learning through synaptic intelligence. arXiv:1703.04200v3. Available at: arxiv.org/abs/1703.04200 [Accessed December 17 2017]. (2017).

Publications that cite this publication

Find all citations of the publication

About this publication

Number of citations 37
Number of works in the list of references 48
Indexed in Scopus Yes
Indexed in Web of Science Yes

ASJC classification

millerhumbecioned.blogspot.com

Source: https://ouci.dntb.gov.ua/en/works/ldrkjoB7/

0 Response to "Comparing Continual Task Learning in Minds and Machines"

Post a Comment

Subscribe to: Post Comments (Atom)

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel