深度學習讀書會

因為文獻太多一次讀書會不可能面面俱到,採取follow每個領域最重要的1~2個研究者的最具代表性的工作的方式,挑選出下面文章重點研讀。文章前的標籤代表類型,NB=神經生物學發現,CM=計算模型,ML=機器學習演算法。

腦與deep learning讀書會主講人聯繫方式

姓名 專業背景 電郵
張巍
於翮
劉彥平
王長明
王靜
肖達 北京郵電大學計算機學院 教師
曹旭東
袁行遠
吳令飛
何永振

Overview: deep architecture in brain and machine(1次)

主講人:肖達

時間:2013年6月16日

相關領域研究者:James DiCarlo

  • 【NB】Chris I. Baker (2004) Visual Processing in the Primate Brain. In "Handbook of Psychology, Biological Psychology", Wiley.
  • 【NB】【CM】DiCarlo JJ, Zoccolan D, Rust NC. (2012) How does the brain solve visual object recognition? Neuron, 73(3):415-34.
  • 【CM】Cadieu CF, et al. (2013) The Neural Representation Benchmark and its Evaluation on Brain and Machine. International Conference on Learning Representations (ICLR) 2013.

Early visual system (retinal ganglion cell, LGN, V1), canonical cortical circuits(1.5次)

集智俱樂部《腦與deep learning讀書會》第2期

題目

Descriptive, Mechanistic and Interpretive Models of Primary Visual Cortex

主講人

  • 肖達,北京郵電大學計算機學院教師。
  • 袁行遠,前淘寶網數據挖掘與並行計算高級演算法工程師,現辭職休假中。

提綱

  1. Descriptive models (What):
    • Responses of a Neuron in an Intact Cat Brain, (Hubel Wiesel http://www.youtube.com/watch?v=8VdFf3egwfg 牆內視頻http://v.youku.com/v_show/id_XNDc0MTkxODc2.html)
    • Contrast sensitivity of Human
    • Receptive Fields and Edges Detection Program Demo
  2. Machanistic Models (How):
    • Oriented Receptive Fields and Position-Less Receptive Fields
    • Fourier Decomposition hypothesis
    • Build Self-Organizing Map for V1
  3. Interpretive Models (Why):
    • What is the Best Multi-Stage Architecture for Object Recognition
  4. The columnar organization of the neocortex and its implication for computer vision

參考文獻

  • 【NB】Matteo Carandini (2012) Area V1. Scholarpedia, 7(7):12105. http://www.scholarpedia.org/article/Area_V1
  • 【NB】【CM】Carandini M, et al. (2005) Do we know what the early visual system does? Journal of Neuroscience, 25:10577-10597.
  • 【NB】Douglas, RJ and Martin, KAC (2007) Recurrent neuronal circuits in the neocortex. Current Opinion in Biology, 17:496-500.
  • 【NB】Douglas, RJ and Martin, KAC (2010) Canonical cortical circuits. Chapter 2 in Handbook of Brain Microcircuits 15-21.
  • 【ML】Kevin Jarrett, Koray Kavukcuoglu, Marc』Aurelio Ranzato, and Yann LeCun. (2009) What is the Best Multi-Stage Architecture for Object Recognition? in Proc. International Conference on Computer Vision (ICCV』09).

learning features(selectivity) & sparse coding, cortical maps(0.5次)

Bruno Olshausen

【CM】Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Research, 37 (23), 3311-3325.

【CM】Bednar JA. (2012) Building a mechanistic model of the development and function of the primary visual cortex. Journal of Physiology (Paris), 106:194-211.

learning transformations(invariance)(1次)

Aapo Hyvarinen, Yan Karklin

【CM】Hyvarinen, A. and Hoyer, P. (2001). A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Research, 41(18):2413–2423.

【CM】Karklin, Y., & Lewicki, M. S. (2009). Emergence of complex cell properties by learning to generalize in natural scenes. Nature, 457(7225), 83-85.

【CM】Adelson E.H. and Bergen J.R. (1985) Spatiotemporal energy models for the perception of motion. Journal Opt. Soc. Am.

【ML】Ian J. Goodfellow, Quoc V. Le, Andrew M. Saxe, Honglak Lee, and Andrew Y. Ng. (2009) Measuring invariances in deep networks. Advances in Neural Information Processing Systems (NIPS).

補充

【ML】Q.V. Le, et, al. Building high-level features using large scale unsupervised learning. ICML, 2012.


V2(1次)

【NB】Lawrence C. Sincich and Jonathan C. Horton (2005) The Circuitry of V1 and V2: Integration of Color, Form, and Motion. Annu. Rev. Neurosci. 28:303–26.

【NB】Roe AW, Lu HD, Chen G (2008) Functional architecture of area V2. Encyclopedia of Neuroscience (Squire L, ed.). Elsevier, Oxford, UK.

【CM】Cadieu C.F. & Olshausen B.A. (2012) Learning Intermediate-Level Representations of Form and Motion from Natural Movies. Neural Computation.

【ML】Zou, W.Y., Zhu, S., Ng, A., and Yu, K. (2012) Deep learning of invariant features via simulated fixations in video. In Advances in Neural Information Processing Systems (NIPS).

【CM】Gutmann MU & Hyvarinen A (2013) A three-layer model of natural image statistics. Journal of Physiology-Paris.

專題討論:learning mid-level features(形式待定)

【ML】Memisevic, R., Exarchakis, G. (2013) Learning invariant features by harnessing the aperture problem. International Conference on Machine Learning (ICML).

【ML】Kihyuk Sohn, Guanyu Zhou, Chansoo Lee, and Honglak Lee. (2013) Learning and Selecting Features Jointly with Point-wise Gated Boltzmann Machines。 Proceedings of the 30th International Conference on Machine Learning (ICML).

【ML】Roni Mittelman, Honglak Lee, Benjamin Kuipers, and Silvio Savarese. (2013) Weakly Supervised Learning of Mid-Level Features with Beta-Bernoulli Process Restricted Boltzmann Machines. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

V4, shape perception(1次)

Charles E. Connor

【NB】Pasupathy, A. & Connor, C.E. (2002) Population coding of shape in area V4. Nature Neuroscience 5: 1332-1338.

【NB】Connor, C.E. (2007) Transformation of shape information in the ventral pathway. Current Opinion in Neurobiology 17: 140-147.

【NB】【CM】Roe AW, et al. (2012) Towards a unified theory of visual area V4. Neuron 74(2):12-29.

【NB】【CM】Cadieu C, Kouh M, Pasupathy A, Connor CE, Riesenhuber M, Poggio T. (2007) A model of V4 shape selectivity and invariance. Journal of Neurophysiology, 98(3), 1733-50.

IT, object & face recognition(1次)

Keiji Tanaka, Doris Tsao

【NB】Charles G. Gross (2008) Inferior temporal cortex. Scholarpedia, 3(12):7294. http://www.scholarpedia.org/article/Inferior_temporal_cortex

【NB】Tanaka, K. (1996). Inferotemporal cortex and object vision. Annual Review of Neuroscience, 19, 109–139.

【NB】【CM】Tsao DY, Livingstone, MS. (2008) Mechanisms for face perception. Annual Review of Neuroscience, 31: 411-438.

【NB】【CM】Tsao D.Y., Cadieu C. and Livingstone M. (2010) Object Recognition: Physiological and Computational Insights. Chapter 24 in Primate Neuroethology. Edited by M. Platt and A. Ghazanfar. Oxford University Press.

海馬體,記憶,睡眠(1次)

待補充

視覺系統的發育和進化,低等動物的視覺(1次)

Jon Kaas

The Evolution Of The Visual System In Primates 待補充

應用實踐

深度學習項目資源表

Deep Learning for Industrial Application 2.0

Deep Learning for Industrial Application

Deep Learning and it's Industrial Applications

深度學習在工業領域的應用匯總

Deep Learning Tutorial & Project Index

Pytorch & Project Index