Good post. There appear to be 2 things. One is skill and integrity, as you say taking care of the details, in carrying out the research. And the other is deliberate misrepresentation for personal benefit. I don’t think we will ever get perfect repeatability, there are too many variables to control. Even the ITP doesn’t get that. But honesty and integrity can be cleaned up - starting with the journals and peer review.
Here’s a good article on China, they are at least aware of the problem.

https://www.linkedin.com/pulse/unpacking-new-chinese-guidelines-responsible-research-rob-johnson-0doue

One of the interesting conclusions on China - Trust the results more if the lead researcher is female…they’re more honest.

And the Journal “Nature” has some good, paywalled articles. I’d love to get the full text.

Elite researchers in China say they had ‘no choice’ but to commit misconduct

https://www.nature.com/articles/d41586-024-01697-y

1 Like

lol ‘no choice’ sounds hilarious. but in many ways i understand the pressure. a lot of us have been there.

here’s another example of something most people probably would not know unless they practiced it: when you culture certain kinds of cells, for example neurons in a petri dish, the way the neurons grow over time in the medium are significantly influenced by how they were dissected. Dissection is something you can’t really just report in a methods section or tell someone how to do and then voila! It’s an art. Now, theoretically you ought to be able to design your experiments such that the variability in the growth of the neurons shouldn’t affect your results THAT much, but biology is so complex, that if you neurons aren’t even growing the same way each time you do the experiment, how can you have confidence in the downstream results?

3 Likes

Yes I think the glaring elephant in the room that the public doesn’t appreciate, even though it’s sitting in plain sight, is that rapamycin is a single variable (more variables if you’re talking about dosing, brand, adjuvants, etc). but let’s ignore those other specific variables and just compare the order of magnitude of difference in variable count between CR and rapamycin. Rapamycin you’re introducing the exact same molecule to both the organisms of study and to humans. CR, on the other hand, you’re removing completely different compounds. In mice, you’re removing whatever they’re being fed (you have to believe that you could map 1:1 what they eat to humans, but when put that way seems nuts to do so) vs removing whatever humans are being fed. The combinatoric equation to calculate out just how many variables are involved in both scenarios probably would match the order of atoms in the universe (reason being that you’re factoring in individuality with humans vs controlled cohorts with mice or whatever other organism).

1 Like

Here’s the other factor that bugs me. The caloric restriction studies are done by restricting mouse chow. Is mouse chow healthy or is it the equivalent of a mousy McDonald’s meal every day? If the latter, then of course CR will be beneficial. Would a CR diet be helpful to someone who is a healthy pescovegetarian? I’m not sure.

I remember reading a pair of marmoset CR studies done at Midwest USA zoos. The cohort of marmosets that ate an equivalent to a normal American diet benefitted from CR. The cohort of marmosets that ate a healthy vegetarian-based diet showed NO benefit from CR. Are all the CR studies just pointing to the fact that if you eat less ‘crap’ and ultra-processed foods you will live longer?

2 Likes

Apologies for this tangent but I think this is a very interesting parable about the story of the Stanford president (although it could be about David Sinclair).

Once again, let’s go back to the drama coming out of Stanford as example. You have high impact research, which could have significant potential, both in terms of the ego of the principal investigator, who I presume this was President Marc Tessier-Lavigne from Stanford. And so, the idea, the problem is that, then they want to publish in high impact journals such as science, or nature, which increases the pressure to generate results, which will verify their theory. For example, the neurological basis of Alzheimer’s disease, which would be a fantastic scientific breakthrough.

However, it becomes a spiral, and apparently, the laboratory is a pressure cooker, where those with data which support that proposition are favored, and those who do not produce data to support that proposition weren’t favored, according to the Stanford article. So it becomes a cycle, and I believe it’s really the principal investigator who has to step forward, and say, “I’m here to do this, to find out what’s going on,” instead of, “To establish what I want to be the truth.” So we find the truth, versus I want this to be the truth, therefore it is the truth.

And the sad thing is eventually somebody finds out, as we saw in this article, that it is non-verifiable data, and that’s where this house of cards peak begins to collapse.

A. J., it’s almost like a system problem, where there’s a system where superstar scientists, pressure people in the lab, who are rewarded when they find the right thing, so that the superstar scientists can publish the science in nature, and then, become somebody who discovers the neurological basis for Alzheimer’s disease, and then, they become a great person.

Unfortunately, this is going on for a long time, but an additional caveat to this is nowadays with the whole social media internet world, I believe it may be amplified to a certain extent.

A. J. Kierstead:

Yeah. I mean, the rockstar scientist that’s also getting this scientific journal article reshared by the New York Times, Washington Post, USA Today, you start hitting that end of things, I mean, it only makes you look even bigger, and it gets you those high-end, like the Stanford President role, things like that, which are very lucrative, and give you even more respect, which doesn’t necessarily shift down the pressures that are required to maintain it.

Stan Kowalski:

Yes, you’re right. And it goes to the egos, and when I did 20 years of biochemistry research, and there were people like that, I knew people like that at that time who, and I thought to myself, if you want to be famous here in the wrong business, why aren’t you in show business? That’s where you get become famous, not science, but still in all they have that mindset.

For example, I had one scientist say to me, “My dream is to have my name in lights in nature, or science.” And I thought to myself, that’s an odd way of looking at things. But then, that mindset then begins to generate this kind of atmosphere, and the pressure is from the top to bottom, and then, there’s additional pressure, because there’s funding. In other words, we have to find results to satisfy NIH, or NSF, who’s ever funding this research.

And there are shades of data, which are either unreliable, unverifiable, or they may be outright fraud.

So for example, there could be a series of experiments where there’s always a question like, this could be A, or B, but we just want it to be B, and we won’t test A again, and again, to make sure that we’re right. So there’s various shades of this kind of problem.

Thankfully, science is to a great extent, it’s an iterative process, and also, it’s competitive.

So the competition can be good, and bad. In this case, with Stanford, the competition had a bad effect, because I believe the attitude of the president was, “I’m going to beat everybody. I’m going to win.” But the competition would be good, because other scientists can say, “Well, let’s take a closer look at what you have.” That’s like a two-sided coin as well.

https://law.unh.edu/blog/2023/07/legal-impact-research-controversy-stanford-university

3 Likes

Mallapaty, S. (2024). Elite researchers in China say they had’no choice’but to commit misconduct. Nature .

Elite researchers in China say they had ‘no choice’ but to commit misconduct

Anonymous interviewees say they engaged in unethical behaviour to protect their jobs — although others say study presents an overly negative view.

By * [Smriti Mallapaty]

Interviews with staff and students at three elite Chinese universities revealed a sense of pressure to publish. Credit: Hao Qunying/Costfoto/Sipa USA via Alamy

“I had no choice but to commit [research] misconduct,” admits a researcher at an elite Chinese university. The shocking revelation is documented in a collection of several dozen anonymous, in-depth interviews offering rare, first-hand accounts of researchers who engaged in unethical behaviour — and describing what tipped them over the edge. An article based on the interviews was published in April in the journal Research Ethics 1.

The interviewer, sociologist Zhang Xinqu, and his colleague Wang Peng, a criminologist, both at the University of Hong Kong, suggest that researchers felt compelled, and even encouraged, to engage in misconduct to protect their jobs. This pressure, they conclude, ultimately came from a Chinese programme to create globally recognized universities. The programme prompted some Chinese institutions to set ambitious publishing targets, they say.

The article offers “a glimpse of the pain and guilt that researchers felt” when they engaged in unethical behaviour, says Elisabeth Bik, a scientific-image sleuth and consultant in San Francisco, California.

But other researchers say the findings paint an overly negative picture of the Chinese programme. Zheng Wenwen, who is responsible for research integrity at the Institute of Scientific and Technical Information of China, under the Ministry of Science and Technology, in Beijing, says that the sample size is too small to draw reliable conclusions. The study is based on interviews with staff at just three elite institutes — even though more than 140 institutions are now part of the programme to create internationally competitive universities and research disciplines.

Rankings a game

In 2015, the Chinese government introduced the Double First-Class Initiative to establish “world-class” universities and disciplines. Universities selected for inclusion in the programme receive extra funding, whereas those that perform poorly risk being delisted, says Wang.

Between May 2021 and April 2022, Zhang conducted anonymous virtual interviews with 30 faculty members and 5 students in the natural sciences at three of these elite universities. The interviewees included a president, deans and department heads. The researchers also analysed internal university documents.

The university decision-makers who were interviewed at all three institutes said they understood it to be their responsibility to interpret the goals of the Double First-Class scheme. They determined that, to remain on the programme, their universities needed to increase their standing in international rankings — and that, for that to happen, their researchers needed to publish more articles in international journals indexed in databases such as the Science Citation Index.

Some universities treated world university rankings as a “game” to win, says Wang.

As the directive moved down the institutional hierarchy, pressure to perform at those institutes increased. University departments set specific and hard-to-reach publishing criteria for academics to gain promotion and tenure.

Some researchers admitted to engaging in unethical research practices for fear of losing their jobs. In one interview, a faculty head said: “If anyone cannot meet the criteria [concerning publications], I suggest that they leave as soon as possible.”

Zhang and Wang describe researchers using services to write their papers for them, falsifying data, plagiarizing, exploiting students without offering authorship and bribing journal editors.

One interviewee admitted to paying for access to a data set. “I bought access to an official archive and altered the data to support my hypotheses.”

An associate dean emphasized the primacy of the publishing goal. “We should not be overly stringent in identifying and punishing research misconduct, as it hinders our scholars’ research efficiency.”

Not the whole picture

The authors “hit the nail on the head” in describing the relationship between institutional pressure and research misconduct, says Wang Fei, who studies research-integrity policy at Dalian University of Technology.

But she says it’s not the whole picture. Incentives to publish high-quality research are part of broader reforms to the higher-education system that “have been largely positive”. “The article focuses almost exclusively on the negative aspects, potentially misleading readers into thinking that Chinese higher education reforms are severely flawed and accelerating research misconduct.”

Tang Li, a science- and innovation-policy researcher at Fudan University in Shanghai, agrees. The first-hand accounts are valuable, but the findings could be biased, she says, because those who accepted the interview might have strong feelings and might not represent the opinions of those who declined to be interviewed.

Zheng disagrees with the study’s conclusions. In 2020, the government issued a directive for Double First-Class institutes. This states specifically that evaluations should be comprehensive, and not just focus on numbers of papers, she says. Research misconduct is a result not of the Double First-Class initiative, but of an “insufficient emphasis on research integrity education”, says Zheng.

Punishing misconduct

The larger problem, says Xiaotian Chen, a library and information scientist at Bradley University in Peoria, Illinois, is a lack of transparency and of systems to detect and deter misconduct in China. Most people do the right thing, despite the pressure to publish, says Chen, who has studied research misconduct in China. The pressure described in the paper could just be “an excuse to cheat”.

The Chinese government has introduced several measures to crack down on misconduct, including defining what constitutes violations and specifying appropriate penalties. They have also banned cash rewards for publishing in high-impact journals.

Wang Peng says that government policies need to be more specific about how they define and punish different types of misconduct.

But Zheng says that, compared with those that apply in other countries, “the measures currently taken by the Chinese government to punish research misconduct are already very stringent”.

The authors also ignore recent government guidance for elite Chinese institutions to break with the tendency of evaluating faculty members solely on the basis of their publications and academic titles, says Zheng.

Tang points out that the road to achieving integrity in research is long. “Cultivating research integrity takes time and requires orchestrated efforts from all stakeholders,” she says.

And the pressure to publish more papers to drive up university rankings “is not unique to China”, says Bik. “Whenever and wherever incentives and requirements are set up to make people produce more, there will be people ‘gaming the metrics’.”

References

  1. Zhang, X. & and Wang, P. Res. Ethics https://doi-org.library.smcvt.edu/10.1177/17470161241247720 (2024).
3 Likes

However, the pressure is higher in China. That’s why 50% of their research is faked. At Western universities that number is closer to 10%. Japanese universities are the most reputable.

2 Likes

All three may help with rapa induced diabetes / less optimal glucose control

But since Dr B mentioned metformin, not that it may be the only one that helps with rapa rebound effects (even if that may not have been why he mentioned it) see below*

And acarbose may be the only one that helps keep mTORC2 up while negative side effect of rapa is to knock it down

*See this rich comment

If you’re still reading (and still concerned), then consider pairing rapamycin with metformin.

Metformin inhibits mTORC1 without releasing negative feedback loops and overstimulating AKT. It stimulates AMPK by inhibiting mitochondrial complex I. AMPK then phosphorylates IRS-1 (Insulin Receptor Substrate 1), whereas rapamycin suppresses IRS-1 phosphorylation. Metformin also inhibits MEK/ERK in the presence of growth factors, while rapamycin activates MEK/ERK by releasing feedback inhibition (Rozengurt et al. 2014) 1. In male NcZ10 mice, combining rapamycin and metformin corrected for their independent downsides (Reifsnyder et al. 2022) 1. Similar results were seen with 4 weeks of combination treatment in male Balb/c mice (4–6 weeks old) (Albawardi et al. 2023).

1 Like

What if the choice is getting to optimal body composition on a very light almost zero CR, without rapa vs over weight/over visceral fat, but with rapa?

1 Like

I think that starts to get complex (and very individual) very quickly. How long has a person been over-weight, why are they overweight, are they still exercising and just large (you can be in great shape but still be “overweight”)… and I think its a hard question to answer. I think there is a lot more potential risk in this scenario as disease processes may already be started (diabetes, blood pressure, etc.) that make the addition of rapamycin more complex and needing of analysis.

While we are all on a continuum (of fitness/optimal weight/disease risk, etc.) the more you deviate from the ideal for a given age, the more it seems (to me) that the risk increases, and the more you need professional medical expertise to track key blood metrics and other physiological metrics to make sure things are moving in the right direction.

2 Likes

And another opinion on CR (or a variant of CR):

“While there are clearly some benefits of [fasting], it’s very difficult to measure what’s happening cellularly,” says Attia. Those benefits also come with a huge downside, decreasing muscle mass, which is why Attia hasn’t done a multi-day fast since 2020. “Today, I just don’t feel that that trade-off is worthwhile, at least at the extreme level that I was doing.”

2 Likes

Had heard that Japanese and German universities are most reputable…

2 Likes

i like tangents. tangents are good. it’s where new insights are formed (no pun intended, as this is especially true in science). what I pity about the modern state of science is that it has become intertwined with the public to the degree that we see such malformed incentives form as demonstrated by the Stanford scenario. back in the day (i come from a family of scientists too) science was more detached from mainstream culture, more isolated in its walled gardens. the democratization of information has surely been great for society, but as with anything, it’s a double-edged sword.

scientists, now realizing they have even more of a voice in broader society and have works that can boost their reputation in the public eye, naturally as humans will think about the incentives at play. and the extreme end of this is leaving your post to become a podcast influencer (we all know who this is). and scientists can always resort to appeals of authority with much more buy-in from the public because science is supposed to be the most objective of pursuits. that is why every podcast starts with ‘this is just about the information, this is not advice.’

the pity is that once these incentives are formed, they are hard to revert. and you see this proven to play out when scientists double down on their results instead of acknowledging that the hypothesis was false. this is why they end up falsifying data-there is too much at stake. how can you reconcile the cognitive dissonance that you’ve advocated for a certain biological model or medical intervention for the past 30 years, and now have been proven to be false? or, as a podcaster, if you have accumulated social capital by touting the benefits of a certain supplement or intervention, how do you walk that back 30 years later when it’s proven to be ineffective? so what podcasters do now is what I alluded to: plausible deniability.

what would i do as a scientist? i would keep studying new things, never anchor on one single focus area for my entire career (e.g. sirtuins or resveratrol). the scientists i have respected the most in my life are the ones who can repeatedly break new grounds in different fields. their reputation isn’t dependent on a single theory being true, and there is the added benefit of having proof of repeated breakthroughs. It’s why people like Elon and Zuck succeed; they’re willing to go into domains they had no previous background in and were outsiders.

2 Likes

Don’t forget this paper too! Genetic Variation in the Murine Lifespan Response to Dietary Restriction: from Life Extension to Life Shortening - PMC

[we discussed it in kaeberlein’s class]

1 Like

Yes, that would back up what @CronosTempi said about the importance of genetics and his experience on the CR Society forum.

And I agree that likely genetics can even trump rapamycin or other interventions. Especially when, like in your study, they test 41 different genetic strains of mice.

After feeling a little guilty about derailing this thread, I went back to reread the whole thread to get back on track. It suddenly occurred to me to try out the “summarize this thread” button, hopefully to separate the wheat from the chaff and get rid of my tangential posts.

WTF!! It went from 67 posts to 14…and looking at those posts, there are some essentials like -

Now how is that a good summary of the essential Points?
Four of the 14 are clearly non-essential…
And a 5th was the much appreciated full-text article from “Nature”
My conclusion is that the “summarize this thread” button doesn’t work.

I appreciated @Neo posting the link to an old thread that I had missed.

But it reminded me that no matter how much reading of old threads I do, there’s always something valuable that I missed…thus the trying out the “summarize this thread” button…so much for that.
Hopefully there’s AI help available, I need to explore.

1 Like

New comment on the original mice paper: Dietary restriction interventions: lifespan benefits need resilience and are limited by immune compromise and genetics

The authors found that CR, especially at 40% restriction, extended lifespan 1.5 to 3 times more than IF. However, the more intense CR protocol reduced certain health markers, such as B-lymphocyte count and lean body mass, which could compromise disease resistance, especially in humans. This highlights the need for caution in applying CR or IF interventions for aging.
The lowering of metabolism has been proposed as a key mechanism by which CR extends lifespan. Yet, Di Francesco et al.’s findings challenge this notion; reductions in mitochondrial respiration, blood glucose, and energy expenditure did not correlate with longevity. Instead, they identified resilience to aging and stress-induced weight loss, particularly the loss of white adipose tissues, as a key determinant of longevity across all dietary groups
Intriguingly, a meta-analysis encompassing various murine studies hints at the possibility that DR protocols may impair post-infection resilience, raising concerns about its potential compromise of immune function.
Furthermore, the Di Francesco study highlights the significant role of genetic factors in lifespan, suggesting they may outweigh the effects of CR and IF. While genetics are generally thought to account for about 30% of interindividual aging variability, lifestyle and environmental factors contribute the remaining 70%. Given this, the modest impact of CR is striking, particularly since it is one of the most effective dietary interventions for aging.
Caution is necessary when developing DR interventions, as disparities may emerge between lifespan extension and the preservation of organismal fitness or immune function. Given human exposure to pathogens, biomarkers that reflect immune health will be essential to avoid negative impacts on disease morbidity, particularly post-infection. Moreover, the challenge of developing effective DR strategies is compounded by their reduced efficacy when initiated later in life.

So IF (one or two days per week) is significantly inferior to CR @RapAdmin.

The original paper also concluded:

Second, our findings more generally imply that the effects of DR on health and lifespan may be partially non-overlapping, and certain lifespan-extending properties of DR may in fact be detrimental to other aspects of physiological health. For example, although mice on 40% CR are healthy by most measures, we saw indications of adverse effects including life-long loss of lean mass, lower body temperature, food-seeking behaviour (an indication of hunger) and changes in immune repertoire that could potentially confer susceptibility to infection. These effects in mice may raise concerns regarding the potential risks of extreme DR for humans.
Finally, whether IF and CR would extend lifespan in humans awaits definitive investigation. Owing to differences in metabolic rates, the human equivalent of these DR interventions is unclear. Although further work is needed to dissect the complex physiological effects of DR, our findings suggest that human responses to DR will be highly individualized based on genetic context, that moderate reduction of caloric intake and regular daily feeding and fasting cycles are key contributing factors, and that specific blood biomarkers can predict an individual’s ability to benefit from certain physiological effects of DR, while withstanding others, to maximize its health benefits and longevity effects.

So even 20% CR could be detrimental. Moderate CR (~10%?) is probably fine, but it will depend on individuals.

2 Likes

“When mice are handled (for a functional test, for example), they tend to lose weight as a stress response. Interestingly, the strongest positive predictor of lifespan was stress-related resiliency to weight loss after handling”.

Hahahaha I am gonna cry now. If I fast I lose a lot of weight, if I have the flu I lose a lot of weight, one day without going to the gym I lose a lot of weight…

Here is a podcast with Gary Churchill on this study. Very good overview of conclusions. In general CR does increase odds of life extension but not all effects are positive. Benefits probably do not come from weight loss. Diet represents ~8% of longevity effects.

1 Like

Actually, the CR research is a lot more complicated than that. I hate to be the bearer of bad news, but a study like this is of very limited value, to put it diplomatically. The genetic variability is only one problem. A bigger problem is in the variability of the intervention itself in the studies. For example, the classic DR/CR simply means lower calories. But hidden in this, are studies that simply take the standard diet, and cut the amounts of food, this is sometimes called DR (dietary restriction), and studies that cut the calories, but re-balance various aspects of the diet, sometimes called undernutrition without malnutrition, and in human subjects, the distinction between CR and CRON (Caloric Restriction with Optimal Nutrition); in animal models of this, the researchers would cut the amount of food, but compensate the lost macro or micronutrients, sometimes both, so that even though the animal consumes less food like in DR, they get added vitamins and minerals to bring them back to 100% RDA, same for macros, by adjusting protein or fat or carbs proportions. So, imagine that you cut the food amount and end up with fewer calories, but also inadequate protein, so you adjust by giving fewer carbs, but more protein, you end up with same calories, but adequate protein. You can even adjust food volume, by providing greater or equal volume of food but less calorie dense, you eliminate or diminish the hunger. In human CRONies, some would consume huge amounts of calorie poor vegetables, like cabbage. One of the triggers of hunger is the signal of stretched stomach. A stomach that is stuffed signals satiety. You are stuffed with cabbage, not hungry, but low calories.

The point is, that the design of the studies matters a great deal. It is far beyond “cut calories”. To give an extreme example, semi-starvation on a poor quality diet in Africa results in a very short lifespan. A well designed diet with the same level of semi-starvation calories results in a longer lifespan than ad libitum free eating normal diet.

There were two CR monkey studies, where one showed CR longevity benefit, and one did not. There were several differences between them (including sabotage by lab techs who “compassionately” secretly added food to the CR’d monkeys, lol!), but the diet composition was one of the key differences - one of them had good diet as would be found in their natural environment, fruits, tubers etc., and the others had terrible quality processed chow very high in sugar. Of what value would it be to average these two studies and draw conclusions about CR based on that. Yet this is what happens when you grab a bunch of studies without accounting for the dramatically different protocols. Not worth very much, sadly.

The quality of the diet matters a huge deal in CR. And it goes both ways. For example, if you took the Standard American Diet, and simply cut the amount - what is responsible for the effect? Eating fewer calories, or eating less toxic food like sugary drinks, processed meats, snacks and other processed foods? Less calories, or less toxins? What effect does it have if you feed an animal a processed chow from ingredients they would not encounter naturally or you feed them their natural diet? Do you think processed food is physiologically harmful only in humans? What do you think lab chow is?

The design of a study is of paramount impact. If I malnourish an animal with key nutrients in inadequate amounts, I can impair their immune system. Starving Africans have very poor immunity and die in droves from trivial infections. In CR studies with re-balanced diets, the CR’d animals have superior immunity compared to ad libitum controls, as the immune system is upregulated by hormetic stress. In other straight DR studies, the restricted animals with inadequate nutrition had poor immunity.

Again, once you get deep into an area, and read tons of studies, you can spot poor design of studies, including metastudies.

I would be very careful taking this study conclusions at face value.