Large language models use a surprisingly simple mechanism to retrieve some stored knowledge

Caption:Researchers from MIT and elsewhere found that complex large language machine-learning models use a simple mechanism to retrieve stored knowledge when they respond to a user prompt. The researchers can leverage these simple mechanisms to see what the model knows about different subjects, and also possibly correct false information that it has stored.
Image Credit: Copilot / DALL-E 3 / AI generated from Scientific Frontline prompts

Large language models, such as those that power popular artificial intelligence chatbots like ChatGPT, are incredibly complex. Even though these models are being used as tools in many areas, such as customer support, code generation, and language translation, scientists still don’t fully grasp how they work.

In an effort to better understand what is going on under the hood, researchers at MIT and elsewhere studied the mechanisms at work when these enormous machine-learning models retrieve stored knowledge.

They found a surprising result: Large language models (LLMs) often use a very simple linear function to recover and decode stored facts. Moreover, the model uses the same decoding function for similar types of facts. Linear functions, equations with only two variables and no exponents, capture the straightforward, straight-line relationship between two variables.

The researchers showed that, by identifying linear functions for different facts, they can probe the model to see what it knows about new subjects, and where within the model that knowledge is stored.

A self-cleaning wall paint

Qaisar Maqbool and Günther Rupprechter
Photo Credit: Courtesy of Technische Universität Wien

A breakthrough in catalysis research leads to a new wall paint that cleans itself when exposed to sunlight and chemically breaks down air pollutants.

Typically, beautiful white wall paint does not stay beautiful and white forever. Often, various substances from the air accumulate on its surface. This can be a desired effect because it makes the air cleaner for a while – but over time, the color changes and needs to be renewed.

A research team from TU Wien and the Università Politecnica delle Marche (Italy) has now succeeded in developing special titanium oxide nanoparticles that can be added to ordinary, commercially available wall paint to establish self-cleaning power: The nanoparticles are photocatalytically active, they can use sunlight not only to bind substances from the air, but also to decompose them afterwards. The wall makes the air cleaner – and cleans itself at the same time. Waste was used as the raw material for the new wall paint: metal scrap, which would otherwise have to be discarded, and dried fallen leaves.

New All-Liquid Iron Flow Battery for Grid Energy Storage

Lead author and battery researcher Gabriel Nambafu assembles a test flow battery apparatus.
Photo Credit:  Andrea Starr | Pacific Northwest National Laboratory

A commonplace chemical used in water treatment facilities has been repurposed for large-scale energy storage in a new battery design by researchers at the Department of Energy’s Pacific Northwest National Laboratory. The design provides a pathway to a safe, economical, water-based, flow battery made with Earth-abundant materials. It provides another pathway in the quest to incorporate intermittent energy sources such as wind and solar energy into the nation’s electric grid.

The researchers report in Nature Communications that their lab-scale, iron-based battery exhibited remarkable cycling stability over one thousand consecutive charging cycles, while maintaining 98.7 percent of its maximum capacity. For comparison, previous studies of similar iron-based batteries reported degradation of the charge capacity two orders of magnitude higher, over fewer charging cycles.

Iron-based flow batteries designed for large-scale energy storage have been around since the 1980s, and some are now commercially available. What makes this battery different is that it stores energy in a unique liquid chemical formula that combines charged iron with a neutral-pH phosphate-based liquid electrolyte, or energy carrier. Crucially, the chemical, called nitrogenous triphosphonate, nitrilotri-methylphosphonic acid or NTMPA, is commercially available in industrial quantities because it is typically used to inhibit corrosion in water treatment plants.

Thousands of tons of microplastics found in Moreton Bay

Dr Elvis Okoffo has tested samples of mud from Moreton Bay for microplastics.
Photo Credit: Courtesy of University of Queensland

University of Queensland researchers estimate there could be up to 7000 tons of microplastics polluting vital ecosystems in Brisbane’s Moreton Bay.

Dr Elvis Okoffo from UQ’s Queensland Alliance for Environmental Health Sciences said the team measured plastic stored within 50 surface sediment samples collected across Moreton Bay.

“The level of plastic contamination we found is equivalent to three Olympic swimming pools full of plastic or 1.5 million single use plastic bags,” Dr Okoffo said.

“The main types of plastic detected were polyethylene (PE) and polyvinyl chloride (PVC).

“PE is used for single-use items such as plastic food wrapping, bags and bottles and PVC is used in pipes, building materials, electronics, and clothing.

Research uncovers a rare resin fossil find: A spider that aspires to be an ant

Ant-mimicking spider in fossilized resin
Photo Credit: George Poinar Jr / Oregon State University

Arachnophobia can make humans flee at the sight of a brown recluse, black widow or even a daddy long legs, but animal predators of spiders know no such fear.

That’s why paleobiologist George Poinar Jr. explains, some spider species have developed the defense of deception. They masquerade as a much less desirable prey – ants – and Poinar’s recent paper in Historical Biology presents an early record of an ant-mimicking spider in fossilized resin.

“Ants are particularly good creatures for spiders to pretend to be – many animals find ants distasteful or dangerous to eat,” said Poinar, who has a courtesy appointment in the Oregon State University College of Science. “Ants are aggressive in their own defense – they have a strong bite as well as stinging venom, and they can call in dozens of nestmates as allies. Spiders, meanwhile, have no chemical defenses and are loners, which makes them vulnerable to being hunted by larger spiders, wasps and birds – predators that would rather avoid ants. So, if a spider can be like an ant, it’s more likely to be unbothered.”

Spiders that disguise themselves as ants live in many locations around the globe but until now most had been able to avoid detection from fossil researchers as well as predators. The specimen that Poinar describes, which he named Myrmarachne colombiana, was entombed in a type of fossilized resin known as copal.

Signs of life detectable in single ice grain emitted from extraterrestrial moons

An artist’s rendition of Saturn’s moon Enceladus depicts hydrothermal activity on the seafloor and cracks in the moon’s icy crust that allow material from the watery interior to be ejected into space. New research shows that instruments destined for the next missions could find traces of a single cell in a single ice grain contained in a plume.
Illustration Credit: NASA/JPL-Caltech

The ice-encrusted oceans of some of the moons orbiting Saturn and Jupiter are leading candidates in the search for extraterrestrial life. A new lab-based study led by the University of Washington in Seattle and the Freie Universität Berlin shows that individual ice grains ejected from these planetary bodies may contain enough material for instruments headed there in the fall to detect signs of life, if such life exists.

“For the first time we have shown that even a tiny fraction of cellular material could be identified by a mass spectrometer onboard a spacecraft,” said lead author Fabian Klenner, a UW postdoctoral researcher in Earth and space sciences. “Our results give us more confidence that using upcoming instruments, we will be able to detect lifeforms similar to those on Earth, which we increasingly believe could be present on ocean-bearing moons.”

The open-access study was published March 22 in Science Advances. Other authors in the international team are from The Open University in the U.K.; NASA’s Jet Propulsion Laboratory; the University of Colorado, Boulder; and the University of Leipzig.

The Cassini mission that ended in 2017 discovered parallel cracks near the south pole of Saturn’s moon Enceladus. Emanating from these cracks are plumes containing gas and ice grains. NASA’s Europa Clipper mission, scheduled to launch in October, will carry more instruments to explore in even more detail an icy moon of Jupiter, Europa.

Mystery of unexplained kidney disease revealed to patients

Professor John Sayer
“What we are now able to do is give some patients a precise diagnosis, which allows their investigations, treatment and management to be tailored to their needs for the best possible outcomes.”
Photo Credit: Courtesy of Newcastle University

Scientists have identified a new method of analyzing genomic data in a major discovery that means patients with unexplained kidney failure are finally getting a diagnosis.

Experts at Newcastle University have worked with data from Genomics England 100,000 Genomes Project to establish a diagnosis in patients with unexplained kidney failure.

There are numerous reasons for kidney failure, which if left untreated is life-threatening, but often patients do not get a precise diagnosis which can make their best course of treatment unclear.

Missing genetic data

Research, published in the Genetics in Medicine Open, has now revealed that for these patients areas in their genome are missing so are not detected as faulty when using the routine genetic pipelines to analyze data. 

Scientists say that as this missing gene has now been identified, and mutations within it found, they have been able to classify this as NPHP1-related kidney failure.

Messenger RNAs with multiple “tails” could lead to more effective therapeutics

Graphic showing scientists adding "tails" to mRNA molecules
Illustration Credit: Catherine Boush, Broad Communications

Messenger RNA (mRNA) made its big leap into the public limelight during the pandemic, thanks to its cornerstone role in several COVID-19 vaccines. But mRNAs, which are genetic sequences that instruct the body to produce proteins, are also being developed as a new class of drugs. For mRNAs to have broad therapeutic uses, however, the molecules will need to last longer in the body than those that make up the COVID vaccines. 

Researchers from the Broad Institute of MIT and Harvard and MIT have engineered a new mRNA structure by adding multiple “tails” to the molecules that boosted mRNA activity levels in cells by 5 to 20 times. The team also showed that their multi-tailed mRNAs lasted 2 to 3 times longer in animals compared to unmodified mRNA, and when incorporated into a CRISPR gene-editing system, resulted in more efficient gene editing in mice. 

The new mRNAs, reported in Nature Biotechnology, could potentially be used to treat diseases that require long-lasting treatments that edit genes or replace faulty proteins. 

“The use of mRNA in COVID vaccines is fantastic, which prompted us to explore how we could expand the possible therapeutic applications for mRNA,” said Xiao Wang, senior author of the new paper, a core institute member at the Broad and an assistant professor of chemistry at MIT. “We’ve shown that non-natural structures can function so much better than naturally occurring ones. This research has given us a lot of confidence in our ability to modify mRNA molecules chemically and topologically.”

Decoding the plant world’s complex biochemical communication networks

PhD candidate Shannon Stirling in Natalia Dudareva’s Lab, transfers DNA into a petunia by using a syringe to inject bacterium into the stigma to activate targeted genes, then isolating the resulting proteins.
Photo Credit: Purdue Agricultural Communications / Tom Campbell

A Purdue University-led research team has begun translating the complex molecular language of petunias. Their grammar and vocabulary are well hidden, however, within the countless proteins and other compounds that fill floral cells.

Being rooted to the ground, plants can’t run away from insects, pathogens or other threats to their survival. But plant scientists have long known that they do send warnings to each other via scent chemicals called volatile organic compounds.

“They use volatiles because they can’t talk,” said Natalia Dudareva, Distinguished Professor of Biochemistry and Horticulture and Landscape Architecture at Purdue. “Plants inform neighboring plants about pathogen attacks. It looks almost like immunization. Under normal conditions, you don’t see any changes in the receiver plant. But as soon as a receiver plant is infected, it responds much faster. It’s prepared for response.”

Plant scientists have long known about this immunization-like priming, but until a few years ago, they had no way to study the process. They needed a marker showing that the plants had detected the volatile compounds.

Dudareva and 13 co-authors describe new details of the detection process in the March 22, 2024, issue of the journal Science. The team includes researchers from Purdue; Université Jean Monnet Saint-Etienne in France; and the University of California, Davis.

Two keys needed to crack three locks for better engineered blood vessels

Two proteins can trigger the signaling cascades needed to help differentiate stem cells into endothelial cells that can form tubular-like vessels in a dish, according to a team led by Penn State researchers. The finding has implications for developing drug-testing platforms and other clinical applications. 
Image Credit: Lian Lab / Pennsylvania State University

Blood vessels engineered from stem cells could help solve several research and clinical problems, from potentially providing a more comprehensive platform to screen if drug candidates can cross from the blood stream into the brain to developing lab-grown vascular tissue to support heart transplants, according to Penn State researchers. Led by Xiaojun “Lance” Lian, associate professor of biomedical engineering and of biology, the team discovered the specific molecular signals that can efficiently mature nascent stem cells into the endothelial cells that comprise the vessels and regulate exchanges to and from the blood stream.

They published their findings in Stem Cell Reports. The team already holds a patent on foundational method developed 10 years ago and has filed a provisional application for the expanded technology described in this paper.

The reserchers found they could achieve up to a 92% endothelial cell conversion rate by applying two proteins — SOX17 and FGF2 — to human pluripotent stem cells. This type of stem cell, which the researchers derived from a federally approved stem cell line, can differentiate into almost any other cell type if provided the right proteins or other biochemical signals. SOX17 and FGF2 engage three markers in stem cells, triggering a growth cascade that not only converts them to endothelial cells but also enables them to form tubular-like vessels in a dish.