Graduate School of Engineering Osaka University
Division of Electrical, Electronic and Information Engineering Department of Quantum Electronic Device Engineering
Biosystems and Devices Area
TEL.+81-6-6879-7786
E6-E(5F),
2-1 Yamadaoka, Suita,
Osaka 565-0871 Japan
RESEARCH
Study on Biological and Artificial Visions with Electronic and Neuromorphic Devices
The image of the real world is projected onto the retina that resides on the back of the eyeball. Neuronal computation of visual information initiates with the transduction of light (photons) into electrical analog-like signals by the photoreceptor cells in the retina. The image stream in a form of the electrical signals is filtered/processed by the retinal neural circuits in real time (Fig. 1), and then is transformed to spatio-temporal distributions of electrical digital-like pulse signals by the retinal output neurons which send the signals to the brain through their nerve fibers. Here, it should be emphasized that the retina is not merely an image sensing device, but is an intelligent neuronal circuitry system that extracts various information fundamental to the perception of complex natural visual scenes. Therefore it is important and significant to elucidate the computational principle and the relevant circuit structures not only from the view point of neuroscience (⇒ Research 1-1) but also from the view point of engineering (⇒ Research 1-2, 1-3).
"Artificial vision" in the field of medical bioengineering often refers to visual prostheses, which aim to partially restore visual function in the patients with acquired blindness by means of artificial control of neuronal electrical activities in the visual nervous system.
Previous clinical studies on humans demonstrated that the electrical repetitive pulse stimulus delivered through a microelectrode to the retina or the visual cortex in the brain elicits perception of a spatially localized light spot, called phosphene. This fact encourages engineers, scientists, and medical doctors to develop the artificial vision based on spatial patterns of phosphenes corresponding to the real world images. Particularly, the cortical visual prostheses are considered to be applicable to the blinds caused by dysfunctions of the peripheral nervous system, such as glaucoma and diabetic retinopathy in the retina or infarction in the lateral geniculate nucleus. In order to develop such a prosthesis, we have to overcome various problems. Firstly, designing how to transmit the information extracted by the artificial eye to the implanted electrodes in the visual cortex is crucial. In our prototype system, the output of the artificial retina is transmitted from an external device to an internal (implanted) device with wireless communication, and then is distributed to current-mode stimulator devices, each of which drives multiple intracortical electrodes (⇒ Research 2-1). Secondly, deigning how to stimulate the visual cortex in an efficient and effective manner is challenging. One of our strategies to tackle this is to measure the cortical neural responses to electrical stimuli in experimental animals in vivo/vitro by employing the voltage-sensitive dye imaging technique (⇒ Research 2-2). In addition, we are developing a head-mount display system that simulates phosphene patterns in real-time based on anatomical, physiological, and psychophysical observations in clinical studies in humans and animal experiments in the literature (⇒ Research 2-3). Development of neural interface electrodes not merely for recording but rather for stimulation is another critical issues for the next-generation prostheses (⇒ Research 2-4).
[Research 1-1]How do the retinal neural circuits process and encode the image?
The ganglion cells, the output neurons of the retina, respond to visual stimuli by generating temporal sequences of the electrical pulses (1-2 millisecond in duration) called action potentials or spikes. There are more than one million ganglion cells in the human retina. This implies that optical images of the real world projected onto the retina are encoded as spatio-temporal distributions of the spikes in those cells, as shown in the right figure. We study how the real world images are processed and encoded by the retinal neural circuits. For this purpose, the spike responses of the ganglion cells to visual stimuli are recorded from the retina tissue isolated from a rodent animal by using a multi-electrode array as well as a single micron-size electrode. Hence, we can analyze those data to examine if and how the visual stimuli can be recovered/decoded from the spike sequences.
The retina, which dynamically processes visual information in natural environment, is a complex, highly integrated neural circuit. Therefore, there are many technical difficulties in unraveling such a computational dynamics solely through physiological experiments. Specifically, it has been little known what kind of information is sent to the brain by the retina under the presence of active and passive eyeball/head movements. One of our strategy to tackle this is to reconstruct a known part of, hypothetical, and/or anticipatory functional circuit in the retina by means of integrated electronic circuit technology (i.e. retina emulator or artificial retina). Taking advantage of its programmable architecture, the circuit can be built and modified with reference to various numerical models of the retinal neural circuit proposed from previous physiological experiments. Moreover, our artificial retina can be installed in an eye-movement robot (referred to as “artificial eye”), thereby enabling us to analyze what the artificial eye sees, namely, what kind of visual information is captured, processed, and encoded by the retina for sending it to the brain. Our artificial retina performs image processing based on the computational principle and the timescale similar to those in the biological retina, being completely different from the conventional digital image processing in industries. By visualizing and examining the output of the artificial retina in response to the visual input under the naturalistic eye movements, the possibility that the biological retina can execute information processing in much more advanced manner than traditionally believed is recently found. Not only gaining insight into such biological vision, but also utilizing those to engineer a completely new and biologically relevant robot vision must be meaningful and exciting.
Under construction
Under construction
[Research 2-2]Experiment study of the electrical stimulation to visual cortex
Under construction
Under construction
In order to efficiently control neuronal activities in a nerve tissue, like the brain or spinal cord, by means of electric, optical, and/or thermal stimulation, it is important to apply the stimulus to the close vicinity of the target neurons. From previous clinical studies on the visual cortex stimulation, it has been known that the threshold of the electrical current stimulus required to induce phosphenes is lower with the intra-cortical stimulation than the cortical surface stimulation by a factor of about 100. However, inserting such a stimulating electrode into a nerve tissue faces problems/challenges, such as those related to the foreign-body reactions of tissues (e.g. gliosis or inflammatory infiltrate), and to irreversible electrochemical reactions at the electrode-electrolyte interface during stimulus current injection (e.g. water electrolysis or metal deposition). For example, the geometrical electrode size (e.g. cross-sectional area) should be smaller in order to avoid the former problem, but at the same time, the small electrode size can result in higher risk of the latter problem. For the next-generation neural prostheses, which may use a large number of stimulation channels, therefore, we need a novel interface that solves both of those problems. Currently, we are employing nano materials for realizing bio-compatible, electrochemically suitable, mechanically flexible, and geometrically small stimulating electrodes.
