Bridging consensus and controversy in fear conditioning research via meta-analysis
A meta-analysis of the effect of protein synthesis inhibitors on rodent fear conditioning
Abstract
Recommendation: posted 12 October 2023, validated 12 October 2023
Garofalo, S. (2023) Bridging consensus and controversy in fear conditioning research via meta-analysis. Peer Community in Neuroscience, 100144. 10.24072/pci.neuro.100144
Recommendation
The study conducted by Carneiro and colleagues (Carneiro et al., 2023) seeks to explore the specific circumstances that influence the impact of protein synthesis inhibitors on the formation and endurance of fear-related memories. This investigation takes the form of a comprehensive meta-analysis of existing literature, making two significant contributions. Firstly, it enhances our understanding of fear conditioning by corroborating established interpretations (Schroyens et al., 2019, 2021) while also introducing novel insights. Secondly, it contributes to the ongoing discourse within behavioral neuroscience regarding the practicality and challenges of applying systematic reviews and meta-analyses to pre-clinical research (Prinz et al., 2011; Errington et al., 2021).
To delve deeper into the subject, the authors conducted distinct meta-analyses for different injection sites and target sessions, thus examining the intervention's effects under varying conditions. Their findings highlight the robust influence of protein synthesis inhibitors on memory consolidation and reconsolidation, but suggest a lack of significant impact on extinction, potentially attributed to the limited number of studies on this topic. Notably, their analysis pinpoints certain well-recognized influencing factors, such as intervention timing and re-exposure duration. However, other proposed boundary conditions, such as memory age and training strength, do not appear to significantly influence the effect size, possibly due to a limited number of studies. This leads to the conclusion that while meta-analyses are valuable for consolidating existing knowledge, substantiation through well-powered, confirmatory experiments is imperative.
Moreover, the research underlines the substantial heterogeneity among individual experiments, particularly within studies, which poses challenges for meta-analysis. Aggregating studies using various methodologies increases the capacity to identify influencing factors, emphasizing the importance of these approaches. The study also addresses the limitations of existing meta-analysis methods and suggests that additional sources of variability and difficulties in replication may exist, extending beyond the usual boundary conditions. These challenges could be attributed to biases within the literature, random error, or variations in experimental protocols.
The paper highlights the significance of rigorous and reproducible research practices in pre-clinical investigations, emphasizing the need for an iterative process that combines data synthesis with empirical testing. While meta-analyses serve as valuable tools for knowledge consolidation, the authors stress that they cannot replace high-powered, confirmatory replication studies. Consequently, they advocate for a more holistic and interconnected approach to experimental science, incorporating data synthesis with empirical validation to enhance the reliability of research findings.
References
Carneiro CFD, Amorim FE, Amaral OB (2023) A meta-analysis of the effect of protein synthesis inhibitors on rodent fear conditioning. , 2022.10.11.509645. https://doi.org/10.1101/2022.10.11.509645
Errington TM, Mathur M, Soderberg CK, Denis A, Perfito N, Iorns E, Nosek BA (2021) Investigating the replicability of preclinical cancer biology. eLife, 10, e71601. https://doi.org/10.7554/eLife.71601
Prinz F, Schlange T, Asadullah K (2011) Believe it or not: how much can we rely on published data on potential drug targets? Nature Reviews. Drug Discovery, 10, 712. https://doi.org/10.1038/nrd3439-c1
Schroyens N, Alfei JM, Schnell AE, Luyten L, Beckers T (2019) Limited replicability of drug-induced amnesia after contextual fear memory retrieval in rats. Neurobiology of Learning and Memory, 166, 107105. https://doi.org/10.1016/j.nlm.2019.107105
Schroyens N, Sigwald EL, Van Den Noortgate W, Beckers T, Luyten L (2021) Reactivation-Dependent Amnesia for Contextual Fear Memories: Evidence for Publication Bias. eNeuro, 8, ENEURO.0108-20.2020. https://doi.org/10.1523/ENEURO.0108-20.2020
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article. The authors declared that they comply with the PCI rule of having no financial conflicts of interest in relation to the content of the article.
This study was supported by grants E-26/111.277/2014, E-26/201.544/2014 and E-26/203.222/2017 from FAPERJ. C.F.D.C. received scholarships from Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) and an award from Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro (FAPERJ) for the conduction of this project (Edital bolsa TEC Mestrado Nota 10 2017).
Evaluation round #1
DOI or URL of the preprint: https://doi.org/10.1101/2022.10.11.509645
Version of the preprint: 2
Author's Reply, 30 Aug 2023
Decision by Sara Garofalo, posted 14 Jun 2023, validated 15 Jun 2023
Dear Authors,
I am happy to announce that two reviewers have now completed their revision of your manuscript.
Both the reviewers and I believe that your paper is very interesting, well written and that it provides useful insights into current literature on fear conditioning. Nevertheless, a few concerns were raised that need to be answered before recommending the paper for publication.
Please find below all the major and minor points highlighted by the reviewers.
Best,
Sara Garofalo
PCI Neuro Recommender
Reviewed by anonymous reviewer 1, 16 Apr 2023
This paper by Carneiro and colleagues attempts to use meta-analysis to tackle the heterogeneity of the basic literature on fear conditioning, especially with respect to the role of protein synthesis inhibition in extinction, consolidation and reconsolidation. While taking a meta-analytic approach to the basic literature is an interesting idea, it appears to me that its main benefit at present is to provide validation to the consensus in the literature - e.g., that protein synthesis is required for consolidation and reconsolidation, especially in the hippocampus and amygdala. It was also comforting, as a behavioural neuroscientist, to read their findings that there was little evidence of publication bias on these topics. As discussed by the authors, where the literature was less clear, such as on the role of protein synthesis in extinction, their meta-analysis found fewer significant effects. I suggest some fairly minor revisions with the aim of improving the clarity of the paper and foregrounding its benefit to basic scientists - which in my view is to help identify blindspots or areas of contention where further research is required.
Main Concerns
I have a few main concerns or suggestions which I think might help improve the paper.
- My main suggestion here is for the discussion to do a bit more on disagreement between the literature and the meta-analysis. I may be wrong, but it seems to me that this is where the value of the approach is - and it's something that is alluded to in the introduction (line 77). I think that emphasising these points of disagreement between meta-analysis and the general thrust of the literature (which can often be dominated by a small number of thought leaders) can help the field to progress. By pointing out areas where things are definitely unclear, meta-analysis can help scientists to generate hypotheses to test. Without this element, it seems to me that the meta-analysis essentially repeats the consensus points in the literature and is relatively silent on the difficult or contentious topics.
- In the introduction (lines 39-43), it is suggested that scientists assume protocol discrepancies are responsible for divergent results (I would further suggest that it is also the polite assumption, rather than suggesting that colleagues have found divergent result because they are wrong, even if they are inadvertent victims of statistical noise). However, the discussion doesn't seem to consider whether this is a valid assumption, although the authors make comments along these lines (lines 660-663). Would the authors care to comment a bit more on this?
- Along those lines, I believe some sub-headings within the discussion would be helpful.
- Amnestic agents are raised in the introduction (line 69) but their importance is not explained. It would be helpful to mention prior to this point why they are important (e.g., potential clinical utility) in order to make it clearer why we would care about the robustness of their effects.
- I disagree with the suggestion that there is no easy answer with respect to trusting empirical studies or meta-analysis (line 700). To me, the answer is simple. It's replication.
Minor Comments
- It seems odd to me that the results of drug dose were not described. I would have assumed that drug dosage would be important, if just as an additional example of a sanity check?
- Line 56 - negative unconditioned stimulus - may be better written as "aversive" unconditioned stimulus.
- On the usage of males over females (lines 669-671), the sex as a biological variable (SABV) is relatively recent (the NIH SABV policy only came into effect in 2016), so it seems unfair to criticise a dataset that includes papers up to 2018 for not complying. I would say that this finding isn't interesting, but simply a symptom of the time it takes for policy recommendations to be implemented and for replication studies with female animals to be published.
- I also wonder about the use of citations per year (lines 141-143). Since papers tend to accrue most of their citations within a certain window (I would guess it's somewhere between 2-5 years after publication). Did the authors consider using a citation metric that considers paper/citation life-cycle?
Reviewed by Emiliano Merlo, 09 Jun 2023
In this manuscript, Carneiro et al. conduct a meta-analysis on the studies reporting the effect of protein synthesis inhibitors (PCI) on fear memory formation and persistence. The manuscript is very well written, clearly explaining the rationale for the analysis, methods, results and data interpretation. The main and supplementary elements are clear and help the reader follow the work and understand the findings.
Overall, I think the study constitutes an important analysis for the field, corroborating some long-standing interpretations (i.e., fear memory formation relies on protein synthesis in some specific brain regions), but also providing additional information for the effect of specific factors in modulating the effect of PSI on memory processing.
I find the study will be of interest to both the learning and memory community and the general reader interested in a synthesis of the role of protein synthesis on memory processing. Besides some suggestions, that I leave for the author’s consideration, I can recommend PCI Neuroscience to publish this preprint.
Suggestions:
- A definition of memory reconsolidation and extinction should be included. These are central concepts to the manuscript, and the general reader may find it useful to access a definition before delving into the main aspects of the work.
- In the “Study Selection” section, one of the selection criteria was that protein synthesis inhibitors to be considered should “…directly affect the process of translation…”. Antisense oligo deoxynucleotides (ASOs), among other effects, block the mRNA-ribosome interaction and prevent translation. Why did the authors decide to leave these molecules out of the analysis? In the last 20 years, there has been a considerable amount of work using this technology to study memory formation and retrieval dependent processes. Exclusion of ASOs from the analysis should be clarified.
- Given the cut-off date (31/12/2018), some readers may be wondering why papers from the more recent 4.5 years have not been analysed.
- Figure 2: labels on each item indicating the memory phase and manipulation will help the readers understand what each funnel plot is showing at a glance.