How an AI Microscope Is Learning to Do Science

This article is part of a series about the many ways our scientists are using—and developing—AI tools for biomedical research. Sign up for our newsletter to have these stories delivered right to your inbox.

In a lab at Gladstone Institutes, a dish of human neurons sits on a robotic microscope. The brain cells were made from a patient with Parkinson’s disease, and the microscope has been given a goal: figure out what it takes to make these sick cells behave like healthy ones.

Beyond that goal, scientists are not giving the microscope any instructions. Instead, it relies on artificial intelligence (AI).

“The thinking microscope has the potential to fundamentally change how we do biology.”
—Deepak Srivastava, MD

The microscope can independently trigger molecular changes in cells, watch what happens, and learn from the results. Without any human input, it designs new experiments and carries them out. Each one gets it closer to a potential therapy for Parkinson’s.

“The thinking microscope is a pretty ambitious project; it’s never been done before,” says Steven Finkbeiner, MD, PhD, director of the Center for Systems and Therapeutics at Gladstone. “One of our main aspirations is really to accelerate science and discovery.”

Part of what makes the new platform so fast is that the microscope doesn’t run one experiment at a time—it runs thousands simultaneously, each on a single cell.

“The thinking microscope has the potential to fundamentally change how we do biology,” says Deepak Srivastava, MD, president of Gladstone. “It can lead us to the types of drugs that keep cells from dying, which we need to ultimately overcome complex diseases.”

The Game That Inspired a Microscope

In 2016, Google’s AI division built a system called AlphaGo to play Go, the ancient chess-like board game so complex that most experts considered it beyond the reach of any computer. But powered by AI, AlphaGo quickly disproved that assumption.

AlphaGo was trained on thousands of human expert games, and then improved by playing millions of games of Go against itself. Soon, it was able to defeat the world Go champion.

Steven Finkbeiner and his team designed an AI-powered thinking microscope that can design and conduct experiments without scientists needing to intervene.

The next year, a new version named AlphaGo Zero mastered how to play the game on its own—without any human intervention. This AI model was simply given the rules of the game, and started by playing against itself. Using a type of AI called reinforcement learning, it figured out what worked and got better with every round. Within just three days, it surpassed AlphaGo.

“The AI managed to learn deeper principles about the game than any human had been able to grasp,” says Finkbeiner. “And it did it more quickly and with fewer computational steps than had ever been done before.”

“Could we use AI not only to analyze data, but also to design and conduct experiments, without scientists needing to intervene? It turns out, we can.”
—Steven Finkbeiner, MD, PhD

When Finkbeiner heard about the effort, however, he wasn’t thinking about board games. He was thinking about biomedicine. Was there a way to build a system that would learn the unwritten rules of human biology in the same iterative way that AI learned Go?

“I wondered if we could create an instrument to leverage machine learning in science,” he says. “Could we use AI not only to analyze data, but also to design and conduct experiments, without scientists needing to intervene? It turns out, we can.”

His insight became the thinking microscope.

The Power to Learn

Finkbeiner’s team had already invented a powerful robotic microscope. It’s a fully automated platform that can track individual cells in a dish for hours, days, or even months, allowing researchers to watch disease as it unfolds in the cell.

The system can help identify patterns in how cells change over time, get sick, and die that would be invisible to a researcher looking only at snapshots of cells. But finding patterns in existing data was still passive. To actually change what happens to those cells, the microscope needed to do more than watch.

To give the robotic microscope the ability to learn cellular biology, Finkbeiner’s group had to give it a way to control cellular processes.

By integrating reinforcement learning into the system, the scientists empowered the thinking microscope to make decisions without needing instructions, giving it the ability to predict what could make cells healthier and test its theories autonomously.

They engineered special light-sensitive proteins and introduced them into cells. Each protein responds to a specific wavelength of light and, when activated, turns on a particular cellular response. Shining one wavelength of light on a cell, for instance, might activate a cascade of molecules involved in stress response, while shining another wavelength might switch on proteins known to be activated in a disease like Parkinson’s or ALS.

The researchers then equipped the microscope with thousands of miniscule mirrors that can direct precise beams of light onto individual cells, or even specific regions within a cell, turning on just those pathways they want to probe. They could turn on different molecular programs in adjacent cells, or at different times, to see how the effects compare.

This ability to carry out its own experiments is necessary for the thinking microscope to act without a scientist’s input.

This is how the thinking microscope works. The AI model designs an experiment and instructs the microscope to carry it out. Using miniscule mirrors, the microscope directs beams of light precisely onto individual cells, which contain light-sensitive proteins that trigger a particular cellular response. The AI then analyzes the effects of the experiment, learns from the data and, based on its observations, designs the next set of experiments—repeating the cycle again.

Early on, Finkbeiner and his team programmed the system to do simple autonomous (or closed-loop) experiments—it analyzed cell images and chose what perturbation to do next based on the results of the analysis. But that decision tree was hard coded, meaning the microscope didn’t have to think. It simply needed to follow instructions.

“To close the loop takes only one experiment,” Finkbeiner says. “But to generate knowledge—which is what we want to do—we need to run experiments iteratively and longitudinally in order to develop a model of biology, then test and refine that model.”

So eventually, they gave the microscope an AI-powered brain.

By integrating reinforcement learning into the system, the scientists empowered the microscope to make decisions without needing instructions. The AI can use its observations to develop a model of the underlying biology of cells in a dish, predict what could make the cells healthier, then test its theory by instructing the microscope to do a next set of experiments.

“With the thinking microscope, we can do experiments maybe a thousand, a million times faster and cheaper than we can with conventional approaches.”
—Steven Finkbeiner, MD, PhD

“If we’re trying to solve a disease, we aim to tell the microscope: this is what a sick cell looks like, this is what a healthy cell looks like,” Finkbeiner says. “As you do experiments and look at how those experiments have changed the cell, take note of anything that makes the sick cell look more like a healthy cell, and do more of that.”

The researchers can now devise thousands of initial experiments in a single well of a dish—a different experiment for each cell—and have the microscope learn from each one, as well as from the results across all the cells. Then, the thinking microscope can identify patterns in how different cells respond, tailor the next round of experiments accordingly, perform the experiments, and repeat the cycle again.

“Our goal is to gain a deep understanding of what’s happening in cells and how they respond to perturbations, so that we can create a blueprint for how to intervene to achieve a particular goal—for instance, reversing or preventing a disease,” Finkbeiner says.

Stress Testing

One of the first things Finkbeiner’s team asked the microscope to figure out was how much cellular stress is too much for brain cells.

Oxidative stress, in which chemically unstable molecules build up in a cell, is a normal part of cell biology. But high levels of oxidative stress are known to kill cells, and the process has been implicated in neurological diseases, such as Alzheimer’s disease, Parkinson’s disease, and amyotrophic lateral sclerosis (ALS).

Scientists have struggled to understand exactly where the line is between a dose that cells can safely withstand and a dose that hurts cells. It’s the kind of question that, to even begin to unravel, requires many dozens of experiments with different doses of oxidative stress under different starting conditions.

But Finkbeiner thought his new system could tackle it.

“With the thinking microscope, we can do experiments maybe a thousand, a million times faster and cheaper than we can with conventional approaches,” he says. “And none of that was possible before.”

The microscope simultaneously tested the effects of thousands of different levels of oxidative stress on neurons in the same dish. What would have taken years to map out was compressed into weeks.

The scientists tasked the microscope with testing different levels of cellular stress in neurons. As a result, it provided the first fine-resolution map of the relationship between doses of oxidative stress and cell death. Seen here is Shijie Wang, a scientist in Finkbeiner’s lab, working on the thinking microscope.

As the dose of oxidative stress increased, the cells died dramatically earlier—and the thinking microscope provided the first fine-resolution map of that relationship. To their surprise, the scientists also found that very low levels of oxidative stress actually caused the cells to live longer than they would without any stress.

“We suspect the improvement in survival may represent a phenomenon called hormesis,” Finkbeiner says. “It’s stress-induced resilience. You can think of it as ‘what doesn’t kill you makes you stronger.’”

With hormesis, a small amount of stress can help a cell withstand a later stress that would otherwise be lethal. Finkbeiner believes the body possesses resiliency mechanisms that likely stave off disease, which helps explain why so many people only develop neurological diseases, such as Alzheimer’s or Parkinson’s, in their sixties or later.

His lab is now using the thinking microscope to demonstrate that directly and create the first ever map of hormesis.

“If we can better understand hormesis and how cells adapt to stress, we can find ways to use those mechanisms to build resilience against many diseases,” he says.

A New Kind of Science

The oxidative stress experiments are just the beginning. As the thinking microscope gets better at running its own experiments and learning from them, Finkbeiner plans to point it at harder questions, and to let it find questions that scientists wouldn’t have asked in the first place.

“Some of the experiments the thinking microscope will think up and do are ones that humans probably would never come up with,” he says. “We have certain biases in terms of which experiments we select. The microscope doesn’t.”

Finkbeiner’s team aims to use the thinking microscope to understand why neurons die in neurodegenerative diseases, as well as how these diseases evolve, and how they can be prevented or treated.

“Our fantasy is, one day, to task the AI with figuring out what it takes to make a cell that has Parkinson’s disease look like a healthy cell— basically discovering what we need to do to fix the disease.”
—Steven Finkbeiner, MD, PhD

These are some of the most challenging questions facing researchers who study these conditions. Despite decades of study, neurodegenerative diseases remain poorly understood, and most clinical trials for new treatments fail or have only marginal benefits in patients.

“One of the biggest obstacles is simply the complexity of human biology,” Finkbeiner says. “I think one of the powerful things about artificial intelligence is that it’s an unrivaled tool for handling complexity. It’s the first time in my career where I really felt like the power of the tool was commensurate with the task.”

The thinking microscope has drawn the attention of some of the biggest names in AI. Google, Microsoft Research, OpenAI, and NVIDIA are all actively collaborating with Finkbeiner’s lab. Their interest isn’t purely scientific. Getting AI to control the physical world is one of the major frontiers in AI right now, and the thinking microscope is one of the first real examples of it working in a research setting.

The ultimate goal for Finkbeiner and his team is to use the thinking microscope to discover what needs to be done at a cellular level to fix diseases like Alzheimer’s, ALS, and Huntington’s. In the photos, Shijie Wang is loading a plate of cells onto the microscope.

“One of the main reasons I wanted to get into the AI field is it moves so fast, so much faster than biology,” Finkbeiner says. “It has already proven its value for revealing insights from our data that had previously eluded us. By hooking our scientific wagon to AI, I think we can significantly speed up science.”

For patients, the shorter the time the better.

Parkinson’s disease was first described over 100 years ago and there is still no drug that slows its progression. Research in Alzheimer’s, ALS, and Huntington’s has been similarly slow and frustrating. Finkbeiner hopes that a new kind of science, where experiments can be faster and invisible patterns can be spotted, will turn over a new leaf for these diseases.

“Our fantasy is, one day, to task the AI with figuring out what it takes to make a cell that has Parkinson’s disease look like a healthy cell— basically discovering what we need to do to fix the disease,” says Finkbeiner. “This is very much like Google trying to win a game of Go—but hopefully we’re solving a much more important problem.”