Super Vision is not Science Fiction

Seeing through blocks with x-ray eyes, or targeting enemies kilometers away with eagle-sharp eyesight, is superhuman vision possible outside of sci-fi movies?

Prof. FAN Zhiyong, Professor of Department of Electronic and Computer Engineering, may have wondered the same as a child too. Growing up in a rustic Chinese small town in Shaanxi Province, he used to be the kid with endless, exotic questions about physics, chemistry, and astrology, among other fields, to his family and teachers. The engineering spirit in him developed as early as in his kindergarten days, having made more than 100 models of spaceships and other vehicles.

An idea comes of age

“My uncle was a physics teacher and he asked me to throw one question at him every day,” recalls Prof. Fan. With an inquisitive mind, he has since developed the exceptional ability to connect ideas belonging to discrete areas. “It’s a special skill to be able to fuse thoughts that appear far apart together and create something new and useful,” he says.

As a big fan of sci-fi movies such as Star Trek and I, Robot, a wild thought of making a “superhuman eye” to be used in humanoid robots and for the visually impaired hit him instinctively. “I first had the idea in 2016 to model an artificial eye filled with nanomaterials on a natural human eye.”

Having earned a Bachelor’s degree in Physical Electronics at Fudan University and a PhD in Interdisciplinary Materials Science at the University of California, Prof. Fan’s research efforts have spanned electrical engineering, materials science and engineering, and materials chemistry and physics. He now focuses on engineering novel nanostructures with functional materials for technological applications including energy conversion, electronics, and sensors, among others.

He shook the science world in 2020 when he unveiled the world’s first three-dimensional artificial eye with capabilities better than existing bionic ones, which are still in poor resolution with 2D flat image sensors, raising hopes it could help restore eyesight for the vision impaired.

Together with his PhD student Dr. GU Leilei, his research team successfully replicated the visual signal transmission to reflect what the eye sees onto the computer screen by connecting nanowire light sensors to a bundle of liquid-metal wires serving as nerves behind the man-made retina during an experiment. 

Seeing the unseen

This new bionic eye may offer superior resolution and hence sharper vision because its density of photoreceptors is six to nine times higher than on human retinas. “The difficulty is in how many wires we can connect to the artificial retina, and it’s something we’re working on increasing,” he says. 

The artificial eye would also expand the visible light spectrum. While the human eye can only detect wavelengths from 380 to 700 nanometers, the bionic eye can respond to wavelengths of up to 800 nanometers, approaching that of infrared light. In effect, everything emitting heat could become apparent even if there were no visible light.

Moreover, humans have a blind spot in our eyes, but the bionic eye’s light sensors scatter across the entire man-made retina and could each feed signals through its own liquid-metal wire at the back, thereby eliminating the blind spot as they do not have to route through a single spot. 

With electronic-to-nerve interfaces research already well under way, it is hoped that one day these nanowire retinas could help robots “see” or be directly implanted and attached to the optic nerves of visually impaired patients.

Given how much the artificial eye may be superior to the human eye, some have raised the prospect of general adoption of the artificial eye simply for the improved capabilities. Prof. Fan is no exception, but he says the risks involved in such implants and the ethical issues that follow need to be thought through.

“It’s not like a Google glass that everyone can see you’re wearing. If you have superman eyes people simply can’t tell the difference. So that brings up a lot of problems, including privacy,” he says. 

More than meets the eye

Before Prof. Fan’s innovation can benefit anyone’s eyesight, much still needs to be done. Since the research was featured in Nature, Prof. Fan’s team has been working with researchers at the Chinese Academy of Sciences to find ways to carry out biocompatibility tests of artificial retina on animals.

Biocompatibility is how a material interacts with a living organism. In this context, it means connecting nanowire light sensors directly to the nerves of animals. This is so critical as a milestone that, in his roadmap, he plans to dedicate the next five years to getting this right before expanding the tests to include human subjects.

Meanwhile, they are also working with Department of Mechanical and Aerospace Engineering at HKUST on adopting a new material system to expand the spectral response of the artificial eye. “The results have been promising so far and raised hopes that it will let the artificial eye have night vision in the future,” he says. 

Besides helping the vision impaired restore eyesight, he envisions a future in which cameras, currently carrying flat image sensors and thus suffering from narrow fields of view, could be fitted with 3D, hemispherical image sensors that extend the maximum area the camera can image.

“Imagine the kinds of applications you could do with it. It could be a fancy camera on your cell phone, and you could also put it on robots, drones, autonomous vehicles, or even space exploration rovers. All these benefit from wide field of view cameras,” he says.

Nature holds the answer

More importantly, he hopes his research provides the scientific community with an inspiration to learn from nature.

“We’re surrounded by animals, blue sky, blue ocean, green mountains, trees, and many other things created by mother nature that we often overlook. But if we spend some time to pay attention, there’re a lot of things we can learn from all things around us, like fish, cats, and dogs,” he says.

“If we are humble enough, we can get a lot of inspiration from nature,” he says. In his case when designing the artificial eye, he looked to octopus’ eyes, which have many more photoreceptors than vertebrates, meaning they can see better in the dark compared with humans.

His love of nature, coincidentally, is also one of the many reasons that made him a loyal member of HKUST. While HKUST’s global ranking attracted his attention when he joined in 2010 shortly after finishing his postdoctoral research at UC Berkeley, he admits it was the panoramic sea view from campus that confirmed to him this was the place to be.

“When you’re deep in your thoughts and dealing with a lot of work, you just need to step back from the computer and look out of the window and be calmed and encouraged again,” he says.

After 11 years at the University, he says what continues to amaze him about the University is its supportive and conducive environment for young professors, demonstrated by the grants available to them and superb laboratory equipment, as well as the collegial and friendly culture among colleagues and students. Today, outside his office or laboratory, he is often seen in the campus’ seaside soccer pitch scoring goals with fellow school and student teammates.

“Our department has brought in many new colleagues in recent years, bringing with them new research areas to keep reinventing the department, which is wonderful to see,” he says. “Having said that, the friendly culture has never faded. It’s really like a family.”