Undergrad Research Project - Optimization and application of augmented, depth-based and saliency-based image processing methods for low resolution retinal prostheses

Spring 2016

Arya Hezarkhani
Shawn Kelly
Project description

Modern image processing algorithms are being used to offer meaningful data to the low resolution retinal implants used in patients suffering from macular degeneration and retinitis pigmentosa. These devices are low power and occupy a limited space; therefore, the limited dimensions of the prosthesis limits the total phosphene (implanted electrode) count possible on the chip to be implanted in the retina. Low resolution results from the limited size of the electrode array, which creates a necessity for intelligent image transformations that preserve the salient features of a patient’s surroundings for ambulatory navigation. A series of algorithms were studied and will be assessed for application in the Neural Devices Engineering Laboratory (NDEL) devices.

After preliminary assessment, [1] demonstrates promising results and produces informative images via iso-disparity techniques and ground segmentation of traversable from non-traversable space. By applying iso-disparity techniques, the algorithm differentiates ground from non-ground and emphasizes obstructions in the traversable path on the ground. The image resolution used in the study shows potential for the algorithm’s cross-platform scalability to NDEL implant resolution. The most informative results produced by this methodology are dependent, however, on reasonable dynamic range (preferably well over 2 bits). Furthermore, and most optimally, a retinal implant device could provide multiple modes of use, chosen by the patient, that offer both intensity-based and/or augmented, depth-based image processing for different conditions, necessities, experiences, and uses: for example, leisurely landscape observation as opposed ambulatory navigation, respectively.

Additional study of [2] and [3] provided further insight into the different approaches of image processing for similar uses and applications. [3] demonstrates an algorithm meant to optimize the processing pathway??"starting with image simplification and ending with spatial scene retargeting based on feature salience??"in a way that, ultimately, provides the patient with an image made up of ON/OFF µLEDs (this study uses a matrix of µLEDs to display the processed image for the patient) outlining the most salient feature(s) of the patient’s surroundings. Reviewing the results of this study indicates that the algorithm most likely works better for simplistic, portrait shots; however, the lack of dynamic range and intensity-based details would only provide the patient with a “grid-of-dots” that may be difficult to decipher, which fails to offer scalability to images of higher complexity (i.e. landscapes). The results displayed in [2] demonstrate, strictly, the extraction of potentially salient features of a patient’s surroundings that may present themselves as obstructions along an ambulatory path. However, the saliency models formed using the algorithm deliver an image of lower quality with less information than those in [1] and [3]. Moving forward, MATLAB and other platforms will be used to manipulate images for algorithm testing. Essential to the success of any algorithm will be general knowledge of the hardware capabilities that may pose latency and processing restraints on computation (i.e. dynamic range, total pixel count, dimensions of electrode array, processing power, etc.).

I intend to use the findings from these papers, in addition to further future research of image segmentation algorithms, specifically depth-based and salience-based methodologies, to develop and heuristically apply algorithms optimal for NDEL’s device. The goal is to develop and apply algorithms that (a) provide informative indications as to the relative location of obstructions along any given ambulatory path for the improvement of patient safety, and (b) that also provide quality displays of landscapes that improve the quality of visual satisfaction and quality of life for the patient.

[1] C. McCarthy, N. Barnes, and P. Lieby, “Ground surface segmentation for navigation with a low resolution visual prosthesis,” Conf Proc IEEE Eng Med Biol Soc., 2011 [2] N. Parikh, L. Itti, J. Weiland, “Saliency-based image processing for retinal prostheses,” J Neural Eng. 2010 Feb [3] W. Al-Atabany, B. McGovern, K. Mehran, R. Berlinger-Palmini, and P. Degenaar, “A Processing Platform for Optoelectronic/Optogenetic Retinal Prosthesis,” IEEE Transactions on Biomedical Engineering, Vol. 60, No. 3, March 2013

Return to project list