Blame bad incentives for bad science

Most of us spend our careers trying to meet — and hopefully exceed — expectations. Scientists do too. But the requirements for success in a job in academic science don’t always line up with the best scientific methods. The net result? Bad science doesn’t just happen — it gets selected for.

What does it mean to be successful in science? A scientist gets a job and funding by publishing a lot of high-impact papers with novel findings. Those papers and findings beget awards and funding to do more science — and publish more papers. “The problem that we face is that the incentive system is focused almost entirely on getting research published, rather than on getting research right,” says Brian Nosek, a psychologist at the University of Virginia in Charlottesville.

This idea of success has become so ingrained that scientists are even introduced when they give talks by the number of papers they have published or the amount of grant funding they have, says Marc Edwards, a civil engineer at Virginia Polytechnic Institute and State University in Blacksburg.

But rewarding researchers for the number of papers they publish results in a “natural selection” of sloppy science, new research shows. The idea of scientific “success” equated as number of publications promotes not just lazy science but also unethical science, another paper argues. Both articles proclaim that it’s time for a culture shift. But with many scientific labs to fund and little money to do it, what does a new, better scientific enterprise look like?

As young scientists apply for tenure-track academic jobs, they may bring an application filled with tens to dozens of papers. Hiring committees can often no longer read or evaluate all of them. So they may come to use numbers as shorthand — numbers of papers published, how many times those papers have been cited and whether the journals the papers are published in are high-impact. “Real evaluation of scientific quality is as hard as doing the science in the first place,” Nosek says. “So, just like everyone else, scientists use heuristics to evaluate each other’s work when they don’t have time to dig into it for a complete evaluation.”

Too much reliance on the numbers means that scientists can — unintentionally or not — game the system. They can publish novel results from experiments with low power and effort. Those novel results inflate publication numbers, increase grant funding and get the scientist a job. Ideally, other scientists would catch this careless behavior in peer review, before the studies are published, weeding out poorly done studies in favor of strong ones. But Paul Smaldino, a cognitive scientist at the University of California, Merced, suspected that when the scientific idea of “meeting expectations” on the job is measured in publication rates, bad science would always win out.

So Smaldino and his colleague Richard McElreath at the Max Planck Institute for Evolutionary Anthropology in Leipzig, Germany, decided to create a computer simulation of the scientific “ecosystem,” based on a model for natural selection in a biological ecosystem. Each “lab” in the simulation was represented by a number. Those labs that best met the parameters for success survived and reproduced, spawning other labs that behaved in the same way. Those labs that didn’t meet expectations “died out.”
The model allowed Smaldino and McElreath to manipulate the definitions of “success.” And when that success was defined as publishing a lot of novel findings, labs succeeded when they did science that was “low effort” — sloppy and probably irreproducible. Research groups doing high-effort, careful science didn’t publish enough. And they went the way of the dinosaurs.

Even putting an emphasis on replication — in which labs got half credit for double-checking the findings of other groups — couldn’t save the system. “That was a surprise for us,” Smaldino says. He assumed that if the low-effort labs got caught by failures to replicate, their success would go down. But scientists can’t replicate every single study, and in the simulation, the lazy labs still thrived. “The most successful are still going to low effort,” he explains, “because not everyone gets caught.” Smaldino and McElreath published their findings September 21 in Royal Society Open Science.

“I think the results they get are probably reasonable,” says John Ioannidis, a methods researcher at Stanford University in California. “Once you have bad practices they can propagate and ruin the scientific process and become dominant. And I think there’s some truth to it, unfortunately.”

The publish-or-perish culture may be having negative consequences already, Edwards says. “I’ve … seen ethical researchers leave academia, not enter in the first place or become unethical,” he says. Scientists might slice their research findings thinner, trying to publish more findings with less data, breaking experiments down to the least publishable unit. That in itself is not unethical, but Edwards worries the high stakes places scientists on the edge of a slippery slope, from least publishable units to sliced-and-diced datasets. “With the wrong incentives you can make anyone behave unethically, and academia is no different.”

Using a theoretical model of his own, Edwards and his colleague Siddhartha Roy show that, at some point, the current academic system could lead a critical mass of scientists to cross the line to unethical behavior, corrupting the scientific enterprise and losing the public’s trust. “If we ever reach this tipping point where our institutions become inherently corrupt, it will have devastating consequences for humanity,” Edwards says. “The fate of the world depends as never before on good trustworthy science to solve our problems. Will we be there?” Edwards and Roy report their model September 22 in Environmental Engineering Science.

To stay away from the slippery slope, scientists will need to change what scientific success looks like. Here’s the rub, though. Scientists are the primary people watching scientists work. When papers go through peer review at scientific journals, ideas get examined in peer-review committees for grant funding or a scientist is being considered for an academic job, it’s other scientists who are guarding those gates to scientific success. A single scientist might be publishing papers, peer-reviewing other peoples’ papers, submitting grants, serving on review committees for other peoples’ grants, editing a journal, applying for a job and serving on a hiring committee — all at the same time. And so the standards for scientific integrity, for rigorous methods, do not reside with the institutions or the funders or the journals. Those standards are within the scientists themselves. The inmates really do run the scientific asylum.

This is not an inherently bad thing. Science needs people with appropriate expertise to read the highly specialized stuff. But it does mean that a movement for culture change needs to come from within the scientific enterprise itself. “This is more likely to happen if you have a grassroots movement where lots of scientists are convinced and are used to performing research in a given way, leading to more reliable results,” Ioannidis says.

What produces more reliable research, though, still requires … research. “I think these are questions that could be addressed with scientific studies,” Ioannidis says. “This is where I’m interested in taking the research, to get studies that are telling us to [do science] this way, [or] this type of leadership is better…. You can test policies.” Science needs more studies of science.

The first step is admitting that problems exist in the current structure. “We’re bought into it — we invested our whole career into the game as it exists,” Edwards says. “We are taught to be cowards when it comes to addressing these issues, because the personal and professional costs of revealing these problems is so high.” It can be painful to see sloppy science exposed. Especially when that science is performed by colleagues and friends. But Edwards says fixing the system will be worth the pain. “I don’t want to wake up someday and realize I’m in a culture akin to professional cycling, where you have to cheat to compete.”

The solution is to add incentives for having an excellent research process, regardless of outcome, Nosek says. Scientists need to be rewarded, funded and promoted for careful, thorough research — even if it doesn’t produce huge differences and groundbreaking results. Nosek points to ideas like registered reports. These are systems where scientists report their experimental plans and methods to a journal, and the journal accepts the paper — whether or not the research produces any noteworthy results.

Despite his results, Smaldino is optimistic that incentives can change, allowing the best science to rise to the top. “I think science is great,” he says. “I think in general scientists aren’t bad scheming people.” The dire predictions of the models don’t have to come to pass. “This is not a condemnation of science,” Smaldino says. “I love science — there’s no other way to learn a lot of things that are important to learn about the world. But the science we do can always be better.”

Stone adze points to ancient burial rituals in Ireland

A stone chopping tool found in Ireland’s earliest known human burial offers a rare peek at hunter-gatherers’ beliefs about death more than 9,000 years ago, researchers say.

The curved-edge implement, known as an adze, was made to be used at a ceremony in which an adult’s largely cremated remains were interred in a pit, says a team led by archaeologist Aimée Little of the University of York in England. Previous radiocarbon dating of burned wood and a bone fragment from the pit, at a site called Hermitage near the River Shannon, places the material at between 9,546 and 9,336 years old.
A new microscopic analysis revealed a small number of wear marks on the sharpened edge of the still highly polished adze, which was probably attached to a wooden handle, the researchers report online October 20 in the Cambridge Archaeological Journal. Little’s group suspects someone wielded the 19.4-centimeter-long adze to chop wood for a funeral pyre or to fell a tree for a grave marker. A hole dug into the bottom of the riverside pit once held a tall wooden post indicating that a person lay buried there, the scientists suspect.

Once the adze fulfilled its ritual duties, a hard stone was ground across the tool’s sharp edge to render it dull and useless, further microscopic study suggests. The researchers regard this act as a symbolic killing of the adze. The dulled tool blade was then placed in the pit, next to the post grave marker, perhaps to accompany the cremated individual to the afterlife.
“By 9,000 years ago, people in Ireland were making very high quality artifacts specifically to be placed in graves, giving us a tantalizing glimpse of ancient belief systems concerning death and the afterlife,” Little says. Her conclusion challenges a popular assumption among researchers that stone tools found in ancient hunter-gatherers’ graves belonged to the deceased while they were still alive. In that scenario, tools and other grave items played no role in burial activities and rituals.
Archaeologist Erik Brinch Petersen of the University of Copenhagen is skeptical. No other European stone adzes or axes from around 10,000 to 6,000 years ago display blunted edges, Petersen says. That makes it difficult to say how such an unusual artifact was used or whether it was intended to accompany a cremated person to the afterlife. In addition, researchers have found only a few European cremations from the same time period.
Since there was no practical reason to turn an effective tool into a chunk of stone that couldn’t cut, Little responds, intentionally dulling the adze’s edge was likely a ritual act. Whatever the meaning, people in Ireland made polished stone tools several thousand years before such implements achieved widespread use in Europe with the arrival of agriculture, Little says.

Excavations in 2001 revealed the Hermitage burial pit. Two small stone tools lay near the polished adze. A couple more burial pits turned up nearby. One contained cremated remains of an adult human from around 9,000 years ago; the other held roughly 8,600-year-old cremated remnants too fragmentary to enable a species identification.

“Hermitage was a special place known about and returned to over hundreds of years,” Little says.

First signs of boron on Mars hint at past groundwater, habitability

SAN FRANCISCO — A new element has been found in Mars’ chemical arsenal.

While sampling rocks from Gale Crater, the Curiosity rover detected boron concentrations of about 10 to 100 parts per billion. The discovery is the first find of boron on the Red Planet and hints that the Martian subsurface may have once been habitable for microbes, scientists reported December 13 at the American Geophysical Union’s fall meeting.

The boron was discovered in veins of calcium sulfate. Such features on Earth indicate that nonacidic groundwater with a temperature of around zero to 60° Celsius once flowed through the area — conditions favorable to microbial life. As groundwater evaporates, boron and calcium sulfate are left behind.

How this process unfolded on Mars is uncertain, the researchers said, though they expect more clues to be uncovered as Curiosity continues its trek.

Heart-hugging robot does the twist (and squeeze)

A new squishy robot could keep hearts from skipping a beat.

A silicone sleeve slipped over pigs’ hearts helped pump blood when the hearts failed, researchers report January 18 in Science Translational Medicine. If the sleeve works in humans, it could potentially keep weak hearts pumping, and buy time for patients waiting for a transplant.

To make the device contract, biomedical engineer Ellen Roche and colleagues lined it with two sets of narrow tubes. One set encircles the sleeve, like bracelets; the other runs from top to bottom. When air pumps through the tubes, the sleeve compresses (like a clenched fist) and twists (like wrung-out laundry). Those actions mimic how the layers of the heart contract.
Researchers programmed the sleeve to sync with the heart’s motion. And like a healthy heart, the robot sleeve’s double squeeze gets blood moving.

Roche’s team, which did the work while she was at Harvard University, triggered heart failure in six pigs and then measured the volume of blood pumped by the heart with and without the sleeve’s help. Heart failure cut the volume roughly in half, to about 1 liter of blood per minute. But the sleeve restored the pumped volume to about 2½ liters per minute — just about normal, Roche, now at National University of Ireland, Galway, and colleagues report.

Big genetics study blazes path for bringing back tomato flavor

An analysis of nearly 400 kinds of tomatoes suggests which flavor compounds could bring heirloom deliciousness back to varieties that were bred for toughness over taste.

About 30 compounds are important in creating a full-bodied tomato flavor, says study coauthor Harry Klee of the University of Florida in Gainesville. He and colleagues have identified 13 important molecules that have dwindled away in many mass-market varieties. Some of the flavor compounds deliver such a thrill to the human sensory system that even a modest increase could make a big difference, the researchers report January 26 in Science.
“I think this will definitely help,” says Alisdair Fernie, who was not part of the study but has studied tomato chemistry at the Max Planck Institute of Molecular Plant Physiology in Potsdam, Germany. “Taste is incredibly complex,” he says, so creating more appealing commercial varieties “for certain, requires a holistic approach,” he says.

To achieve that holistic view, the researchers teamed up with geneticists at China’s Agricultural Genomics Institute in Shenzhen, who determined the full genetic makeup of a whopping 398 kinds of tomatoes, wild as well as heirloom and commercial. The scientists ran 96 varieties of tomatoes through taste-testing panels, looking for genetic and chemical similarities among those varieties ranked tastiest.

Much of what makes some tomatoes taste better is actually smell, Klee points out. Tongues can detect relatively few qualities, such as sweetness, acidity and softness. Chemical detectors in the nasal passages are far more varied and sensitive. So what really puts the “Mmmm” into a tomato is the whoosh of air forced up into the nasal passages as someone swallows. Airborne compounds, known as volatiles, are abundant in tomatoes, and Klee looks to them for flavor magic.

Of these volatile compounds, some appear in even the tastiest tomatoes at minuscule levels — only parts per trillion. But human senses respond so strongly to the odors that a little bit goes a long way. Tomatoes should taste noticeably better if researchers can breed just four or five heirloom versions of volatile-producing genes back into commercial varieties, Klee says.

Increasing the sweetness of today’s tomatoes, on the other hand, may be tougher. About 80 percent of the sugar in commercial tomatoes comes from the leaves and is transferred to the big red globes as they mature (SN: 7/28/12, p. 18). Because breeders have done such a great job of maximizing the number of fruits on a plant, the plants would need lots of leaves to sweeten them all. So the price of sweeter tomatoes would be making them smaller, and fewer.
“Now we come to the real crux of the problem,” Klee says. “I have to fix the flavor, but I can’t compromise all of the stuff that breeders have done to the modern tomatoes to make them healthier, more productive, more disease resistant and more shippable,” he says.

And let’s not forget about what happens to tomatoes after they’re picked, says Ann Powell, who studied tomato ripening and disease resistance at the University of California, Davis and is now at the National Science Foundation. Cooling weakens flavor, as cooks who shriek at the horror of storing tomatoes in refrigerators have long known. Therefore, Powell says, another study of Klee’s from 2016 — on how chilling can turn on and off genes — makes an important companion to the new work. A combination of breeding better plants and coddling them strategically may be the way forward for tastier tomatoes.

DNA points to millennia of stability in East Asian hunter-fisher population

In a remote corner of eastern Russia, where long winters bring temperatures that rarely flicker above freezing, the genetic legacy of ancient hunter-gatherers endures.

DNA from the 7,700-year-old remains of two women is surprisingly similar to that of people living in that area today, researchers report February 1 in Science Advances. That finding suggests that at least some people in East Asia haven’t changed much over the last 8,000 years or so — a time when other parts of the world saw waves of migrants settle in.
“The continuity is remarkable,” says paleogeneticist Carles Lalueza-Fox of the Institute of Evolutionary Biology in Barcelona, who was not involved with the work. “It’s a big contrast to what has been found in Europe.”

In Western Europe especially, scientists studying ancient DNA have put together a picture of flux, says study coauthor Andrea Manica. “Every few thousand years, there are major turnovers of people.” Around 8,000 years ago, he says, migrating farmers replaced hunter-gatherers in the area. And a few thousand years after that, Bronze Age migrants from Central Asia swept in.

In DNA collected from the bones and teeth of these ancient peoples, scientists can spot genetic signatures of different populations. When a population of farmers balloons, Lalueza-Fox says, the signatures of hunter-gatherers are mostly erased.

But whether that’s true across the globe is unclear, says Manica, of the University of Cambridge. “We wanted to see what happened in other places…. Asia is huge compared to Europe, and it’s been neglected.”
Manica’s team collected DNA from the skeletons of five ancient people found in a cave called Devil’s Gate. The cave rests in a far east finger of Russia, tucked along the border of China and North Korea, and holds human remains, scraps of textiles and bits of broken pottery.

Researchers gathered enough DNA from two of the people to piece together about 6 percent of the genome, the complete set of genetic instructions inside a cell’s nucleus. That’s not much, Manica says, but it’s enough to compare the Devil’s Gate denizens with other people. The researchers analyzed the genomes of people strewn across the far reaches of the continent — from the Dolgan in Siberia to the Thai thousands of kilometers south.

Genetically, the 7,700-year-old women closely resembled the Ulchi, a small group of hunter-fishers who still live off the land today. Manica can’t say whether the Ulchi are direct descendants of the two Devil’s Gate women, or just closely related. But the find suggests a pocket of stability in East Asia — a place where hunter-gatherers weren’t swept out by, or folded into, booming groups of farmers.

Perhaps farming didn’t take off there because the cold climate wasn’t good for growing crops, Manica says. Or maybe the ideas and technologies from farmers and other migrants made it to the Ulchi without an accompanying influx of people. (The Ulchi aren’t like primitive hunter-gatherers of the past. They farm a bit, and have adopted new ways to fish, hunt and store food, he points out.)

“This shows that ideas can travel without people moving with them,” Manica says.

That makes sense, Lalueza-Fox says. But scientists now need more data — additional samples from East Asia, and Southeast Asia, too, he says. “I have a feeling the whole story will be much more complicated.”

Mapping rainforest chemistry from the air reveals 36 types of forest

To some forest creatures, a tree is a home. To scientists, it’s a beacon. A new way of mapping forests from the air by measuring chemical signatures of the tree canopy is revealing previously unrecognized biodiversity.

The swath of tropical forest covering the Peruvian Andes Mountains and the Amazon basin is one of the most biodiverse places on Earth. But it’s such a wild and remote region that variation within the forest is hard to spot.
“If you look in Google Earth, it just looks like a big green blanket,” says study coauthor Greg Asner, an ecologist at the Carnegie Institution for Science in Stanford, Calif.

Up close, it’s a different story. Each tree species has a distinctive set of chemical traits, such as levels of nutrients like nitrogen and phosphorus in the leaves. Collectively, those characteristics can reveal a lot about the makeup of the forest.
To peek beneath the green blanket, Asner and colleagues divided 76 million hectares of forest into 100-kilometer squares. The researchers measured levels of water, nitrogen, phosphorus and calcium in the trees’ leaves via aircraft by measuring the wavelengths of light reflected by the forest canopy, taking samples from small areas of each square. They also mapped leaf levels of lignins and polyphenols, two chemicals used for defense. Using that data, the scientists identified 36 unique types of forest — a much more nuanced view than the broad categories currently used for classification, the researchers report January 27 in Science. They parceled those highly specific forest types into six groups that roughly aligned with the country’s topography and geography.
Parsing out these differences in forests at such a fine scale is important for guiding conservation efforts, Asner says. A particular spot might appear at a distance to be the same as its surroundings but may actually contain species found nowhere else.

The team is now carrying out similar studies in northern Borneo and Ecuador. Eventually the researchers hope to boost their sensors into orbit to map biodiversity around the globe.

‘Monkeytalk’ invites readers into the complex social world of monkeys

The social lives of macaques and baboons play out in what primatologist Julia Fischer calls “a magnificent opera.” When young Barbary macaques reach about 6 months, they fight nightly with their mothers. Young ones want the “maternal embrace” as they snooze; mothers want precious alone time. Getting pushed away and bitten by dear old mom doesn’t deter young macaques. But they’re on their own when a new brother or sister comes along.
In Monkeytalk, Fischer describes how the monkey species she studies have evolved their own forms of intelligence and communication. Connections exist between monkey and human minds, but Fischer regards differences among primate species as particularly compelling. She connects lab studies of monkeys and apes to her observations of wild monkeys while mixing in offbeat personal anecdotes of life in the field.

Fischer catapulted into a career chasing down monkeys in 1993. While still in college, she monitored captive Barbary macaques. That led to fieldwork among wild macaques in Morocco. In macaque communities, females hold central roles because young males move to other groups to mate. Members of closely related, cooperative female clans gain an edge in competing for status with male newcomers. Still, adult males typically outrank females. Fischer describes how the monkeys strategically alternate between attacking and forging alliances.

After forging her own key scientific alliances, Fischer moved on to study baboons in Africa, where she entered the bureaucratic jungle. Obtaining papers for a car in Senegal, for instance, took Fischer several days. She first had to shop for a snazzy outfit to impress male paper-pushers, she says. Fischer and her local guide then shuttled from one government official to another until a well-timed phone call from a local police chief to a key bureaucrat finally produced the forms.

Monkeys get the job done using their own brand of intelligence, Fischer writes. Macaques and baboons navigate their home regions expertly, discern small quantities and object sizes pretty well, and know who’s socially dominant over whom. These abilities are somewhat humanlike, but Fischer draws a bright line between monkeys’ and people’s social lives. Our primate relatives specialize in tracking comrades’ behaviors, she holds, rather than trying to infer others’ plans and desires. And unlike human groups, monkey communities don’t steadily accumulate knowledge and innovations or communicate in languagelike ways, Fischer contends.

So what if monkeys don’t write books or gossip about each other? Their social lives are complex enough to remain largely a mystery to humans, Fischer concludes. The gritty work of conducting long-term studies, especially in the wild, can illuminate the worlds inhabited by monkeys.

Math-anxious brains tackle simple problems differently

SAN FRANCISCO — When faced with simple math problems, people who get jittery about the subject may rely more heavily on certain brain circuitry than math-savvy people do. The different mental approach could help explain why people with math anxiety struggle on more complicated problems, researchers reported March 25 at the Cognitive Neuroscience Society’s annual meeting.

While in fMRI machines, adults with and without math anxiety evaluated whether simple arithmetic problems, such as 9+2=11, were correct or incorrect. Both groups had similar response times and accuracy on the problems, but brain scans turned up differences.

Specifically, in people who weren’t anxious about math, lower activation of the frontoparietal attention network was linked to better performance. That brain network is involved in working memory and problem solving. Math-anxious people showed no correlation between performance and frontoparietal network activity.

People who used the circuit less were probably getting ahead by automating simple arithmetic, said Hyesang Chang, a cognitive neuroscientist at the University of Chicago. Because math-anxious people showed more variable brain activity overall, Chang speculated that they might instead be using a variety of computationally demanding strategies. This scattershot approach works fine for simple math, she said, but might get maxed out when the math is more challenging.

Squabbles in star nurseries result in celestial fireworks

A stellar game of chicken between two young stars about 500 years ago has produced some fantastic celestial fireworks, new images released on April 7 by the European Southern Observatory reveal.

Whether or not the stellar duo collided is unclear. But their close encounter sent hundreds of streamers of gas, dust and other young stars shooting into space like an exploding firecracker. Using the Atacama Large Millimeter/submillimeter Array in Chile, John Bally of the University of Colorado Boulder and colleagues made the first measurements of the velocities of carbon monoxide gas in the streamers. From the data, they identified the spot where the stars probably interacted and determined that the encounter ripped apart the stellar nursery in which the stars were born. Such a cataclysmic event flung nursery debris into space at speeds faster than 540,000 kilometers an hour.

The dueling stars were born in a stellar nursery called Orion Molecular Core 1, about 1,500 light-years from Earth behind the Orion Nebula. There, gas weighing 100 suns collapses under its own gravity, making the material dense enough for embryotic stars to take shape. Gravity can pull those stellar seeds toward each other, with some grazing or colliding with each other and violently erupting. In this case, the encounter produced a kick as powerful as the energy the sun emits over 10 million years.

This explosion may have initially released a burst of infrared light lasting years to decades. If so, such spars among young stars might explain mysterious infrared flashes observed in other galaxies, the scientists suggest.