Misinformation at machine speed: Why trusted content matters more than ever

A chance to transform research

Researchers and their institutions work diligently every day to make breakthroughs and advance human progress. This work relies on credible evidence, documented through trusted sources and methods, and shared widely through respectable platforms and publications. In providing the bedrock for scientific endeavors, trusted research enables effective policy, healthcare, and other crucial societal decisions. However, trust in the reliability of scientific knowledge has been under increasing threat from the growth of misinformation and disinformation. This troubling trend has accelerated in recent years with the introduction of generative AI (GenAI) platforms (United Nations, 2025).

Yet GenAI technologies also offer transformative support for research practices and are increasingly part of the work environment in research institutions. Despite the media hype about GenAI, the community of researchers, librarians, and academic leaders is legitimately concerned with upholding the integrity of research while accessing the advantages of a technology that also has the potential to undermine it.

As AI tools become embedded in everyday research practice, the question is no longer whether researchers will use AI, but whether they can trust it. The reliability of AI-supported research depends not only on the performance of a given tool, but on the quality, governance, and transparency of the content that underpins it.

What makes research trustworthy?

Trustworthy research is the outcome of a process that is “robust, rigorous, and transparent at all stages of design, execution, and reporting” (Moher et al., 2020). Research literature, along with the journals and publishers that support it, are trusted if they reflect three interdependent qualities of accuracy, authority, and transparency — and safeguard those qualities through integrity processes that contribute to this robustness.

Accuracy

Accuracy is measured through a combination of factors. One is recency: research relies on the most up-to-date understanding of a research topic. Another is validity, meaning the results were found by applying a valid and reliable methodology. Lastly, verifiability ensures that the data presented is rooted in evidence that can be reproduced using the same methodology. This replication strengthens the credibility of the research, as does the choice of study type and its level of evidence.

Importantly, citations within the literature should meet the same accuracy standards as the research itself.

Authority

Authority is established through both expertise, background and experience, and ethical practices.

An author’s education, current and past research, affiliated institution and other associated projects help establish their credibility as experts in their field.

When it comes to journals and publishers, standards and policies demonstrate credibility and a strong commitment to ethics在新的选项卡/窗口中打开 by employing procedures to protect the scholarly record and editorial independence. They also uphold and participate in community efforts to enhance publishing standards.

High-quality publishers and journals address the dilemmas a new technology, such as AI, poses to research integrity with policies that guide authors and editors in upholding established standards for research accuracy, authority, and transparency while allowing them to employ a technology that can aid their research.

Transparency

Ethics inform authority, but they also drive transparency. Transparency in research refers to the visibility and understandability of every aspect of the research process. It allows a reader to see affiliations and contributions of the authors and may include disclosures and conflicts of interest. Transparency also shows the resources used by the authors as the foundation for their research, whether prior literature or data, through standard citations and reference lists. As part of upholding transparency, published works and the journals where they are published adhere to widely established guidelines for citation.

Transparency also applies to demonstrating bias mitigation in individual studies, journals, or research databases, and should apply to content in AI tools used for research as well. To enhance the rigor and neutrality of a research database, journals or publishers may employ independent, external boards to assess the inclusion of data or research.

Transparency is indispensable to trusted research as it enables accurate and verifiable results, and validation through peer review.

Safeguarding the scholarly record

In a trusted research ecosystem, accuracy, authority, and transparency are consistently held to a high standard through rigorous processes, including the practice of peer review and establishing integrity safeguards.

Peer review

While there are several methods of peer review, the core concept is the same: experts in the field review the research prior to publication to determine the quality of its findings and methodology.

For decades, peer review has been identified as the most important factor in determining trustworthy research (Tenopir et al., 2015). Notably, peer review “remains the best way to moderate scientific literature” (Drozdz & Ladomery, 2024).

As one UK professor describes, peer review serves as “the litmus test for quality and rigor.” They emphasize that, “publishers have to manage and monitor processes and outputs to ensure high quality and set clear expectations for researchers and the publication of research” (Researcher of the Future — a Confidence in Research Report, 2025).

Integrity oversight

Peer review alone does not protect the integrity of research. There also needs to be a protected publishing process. Unfortunately, in recent years the scientific publishing world has seen an increase in the systematic manipulation of the publishing process and a rising complexity of ethics cases (Stuckey, 2025).

To uphold trust, publishers can take steps to counter these developments by identifying a range of suspicious publishing activities and paper submissions. Elsevier takes a two-pronged approach, employing both the latest technology and strict human oversight.

Beyond proactive measures, retractions and corrections are vital to integrity in research and protecting the credibility of science. In fact, 85% of researchers agree that these practices are important to preserving the scientific record (Researcher of the Future — a Confidence in Research Report, 2025).

The publishing community must collaborate to continually improve these processes and maintain confidence in research.

Why does trusted research matter for AI tools?

The risks of misinformation

Conventionally, misinformation in research output stems from flawed methodologies, fabricated data, or other improper research practices. However, the introduction of AI tools presents a new potential for misinformation because of their ability to hallucinate responses, including citations (Westreich, 2025). A department head at a Chinese university noted, “when I found that AI often fabricates fake literature on its own, I became very cautious about using GenAI in academic research” (Researcher of the Future — a Confidence in Research Report, 2025).

Recently, false research citations have become high-profile news, appearing in courtrooms and at Deloitte. These exposures may be why only 22% of researchers rate current GenAI tools as trustworthy (Researcher of the Future — a Confidence in Research Report, 2025).

Furthermore, a recent poll of science professionals found that more than 1 in 4 (27%) did not know which scientific content is part of their organization’s AI systems, making those professionals more vulnerable to inaccuracies and incomplete information (Technology Networks, 2025).

At a minimum, false citations and hallucinations stemming from GenAI tools waste researchers’ time and increase the likelihood of retractions in the published record. At worst, these tools expose researchers or their institution to reputational and financial costs, put grant funding at risk and degrade trust in the research ecosystem and higher education overall.

Despite these concerns about the trustworthiness of AI tools, almost 60% of researchers use AI tools for work and 69% expect that AI tools will save them time in the next several years (Researcher of the Future — a Confidence in Research Report, 2025).

Assessing trusted sources: content is the foundation

In response to these developments, the ability to discern real, trusted research and citations has never been more critical. For increased confidence, AI-powered research tools must meet high standards of research excellence and integrity. Meeting that standard begins with the content the tools build on.

Content, in this instance, refers to the foundational material used for training the models behind AI tools, which is how these tools can influence the spread of misinformation. This material is also used for response generation and any citations used within the response.

What content is used in AI tools?

For AI-powered research tools to uphold principles of trust, they depend on the accuracy, authority and transparency of the content base they are built on.

Content in general-purpose AI tools

Lack of authority

The authority of trusted research depends on the expertise of an individual and the ethics, standards, and policies of journals and publishers. The content of the open internet, where anyone can contribute, lacks these proper guardrails. This can be concerning as many of the large AI platforms, like ChatGPT, are trained on broad web-scale data and may draw on large portions of the open web and other sources (OpenAI, n.d.).

AI tools with content drawn from the open internet can be useful for brainstorming ideas or improving writing. Some can also help debug code, generate Excel formulas or offer basic translation (Harvard University Information Technology, 2023).

Although they can access some trusted research content, large AI platforms more often reference commercial websites or crowd-sourced message board comments. An August 2025 study of citation patterns of three large AI platforms, ChatGPT, Google AI and Perplexity, reported these top three sources of citations for each (Lafferty, 2025):

• ChatGPT: Wikipedia, Reddit, Forbes

• Google AI: Reddit, YouTube, Quora

• Perplexity: Reddit, YouTube, Gartner

As an example of the potential problems with this content, Wikipedia articles, while found to be generally reliable, are not peer-reviewed and can be openly edited by anyone, making them susceptible to illegitimate sources, editor’s bias, and outdated information (Thomas, 2018).

Additionally, AI tools can struggle to evaluate the trustworthiness of the content they search, presenting factual and false information with equal confidence (MIT Management, 2024).

Reduced accuracy

Research tools must keep their content up to date with the latest scientific findings. However, AI tools differ in how often they update, with some reflecting current information and others lagging months or even years behind. This delay can lead to the spread of inaccurate scientific information and is particularly problematic when used for research purposes.

An up-to-date scientific record also includes marking or removing retracted articles. Maintaining this level of accuracy can be challenging for some AI platforms. A recent study found that ChatGPT failed to flag articles that had been marked as problematic or retracted (Khedkar, 2025).

Diminished transparency

Content within a general-purpose tool does not always have the transparency needed to meet research standards. Responses from these tools provide seemingly authoritative responses but lack visibility into the source material used. When references are provided, it may not be clear how the information was used to generate the response.

Lack of proper citation makes it difficult for users to evaluate the credibility, authority and the context of the material. This limits a user’s ability to determine if information reflects peer-reviewed research or a web message board. Additionally, identifying bias or conflicts of interest from the original author becomes more challenging.

Transparency also extends to how information is obtained by AI systems. Reports have described some AI tools deploying harvesting bots to crawl information-rich websites to “download as much content as possible, as quickly as possible” (Dohe, 2025). This process overloads servers and limits humans from being able to access sites (Dohe, 2025). This is a growing problem for institutions, particularly for libraries and their digital collections.

Content in research-grade AI tools

In contrast, research-grade AI tools are purpose-built for research workflows, grounded in trusted content and designed to support human judgment. The trusted foundation of curated, peer-reviewed literature meets the requirements for accuracy, authority and transparency.

Similar to the safeguards in the research process, well-governed content foundations are selected and maintained through a transparent selection and evaluation process that is regularly re-evaluated for quality by independent oversight.

But not all research-focused AI tools are equal. When it comes to content within currently available tools, it often differs across seven categories. Evaluating content through this framework helps researchers identify relevant, trustworthy information, supporting both their research goals and the principles of research integrity.

Content quality and rigor

• Types of content: Understanding the types of content behind AI tools allows you to make informed decisions about the trustworthiness of information. A quality research-grade AI tool is typically built on a trusted foundation of scientific research: peer-reviewed literature. Some tools built for research purposes include pre-prints, conference proceedings, reviews and other types of material that haven’t been peer-reviewed. These materials can support some research goals, but it’s important to carefully evaluate their quality and role in the research field. Either way, ensure your AI tool’s responses contain citations, so you can verify the sources used.

• Content curation: In addition to content type, it’s important to understand the tool’s content selection process or in other words: who or what determines which content is included – and excluded – from the tool? The content selection process provides another dimension to content quality that may not often be considered. General tools that scrape the open internet have limited checks on the included content types and provenance. Human-curated content strategies with a set framework for inclusion help to provide improved quality and credibility. Using independent review boards to select content adds another layer of trust and rigor to the results by increasing transparency, authority and providing consistent oversight. Without this, low-quality and biased content can enter the system and impact research outputs.

Text availability

Within tools using peer-reviewed content, coverage can vary widely. The type of text includes abstracts, full text journal articles and book chapters. The best coverage depends on your research goals and your research phase. A wide array of abstracts works well for gaining breadth of current research. This is better suited for beginning a research project where you’re likely trying to understand the landscape. Abstracts can also be helpful for keeping up with the latest research in a timely manner.

Full text has two main types: journal articles and book chapters. Full text journal articles provide the latest updates and research in a field, while book chapters provide in-depth exploration and a cumulation of knowledge on a specific topic. In contrast to abstracts, full text has an in-depth view of the content, allowing for investigation into the findings, methodologies and limitations. This is particularly important when synthesizing relevant articles and identifying gaps for new research.

Volume

This refers to the size of the content base within the AI tool, specifically full text access as abstracts are available to everyone. Tools using peer-reviewed content can include open access as well as subscription content. Open access content is available to everyone while subscription content is available through an institution’s library. Subscription content will vary depending on the institution and their specific needs.

Greater volume of content can reduce the chance of missing an article relevant to the research topic. The fewer full text articles, the less confident you can be in the validity of the response.

Subject

Each content base has a different mix of subject makeup, depending on where they receive their content. Researchers will want to ensure the AI tool has the necessary content for their subject specialty.

From an institutional perspective, considerations depend on the needs of the institution. As collaborative and interdisciplinary research are on the rise, tools with a broader range of subject coverage can meet the needs of a changing research ecosystem (Researcher of the Future — a Confidence in Research Report, 2025)

Publisher

Understanding which publishers contribute to an AI tool’s content base supports understanding the quality of the content included. Ensuring the tool’s content comes from reputable publishers can increase confidence in the AI response as being scientifically sound.

The publishers involved can provide further insights into the volume of content and the subject matter. Comprehensive cross-publisher full text content and data support more informed decisions and research quality.

Update frequency

Understanding how frequently the content base is updated is essential to staying up to date on the latest research and reducing the risks of missing insights. Additionally, updates play a key role in maintaining the scholarly record. Frequent routine content updates help to manage retractions, provide critical transparency, and build confidence in the research and platform.

Taken together, these dimensions of content determine not only the quality of AI-generated outputs, but ultimately the confidence researchers have in these tools.

Moving beyond research content

As AI continues to transform the research ecosystem, the integrity of the content powering these tools becomes critical in determining whether they drive progress or perpetuate misinformation. By integrating the values of trustworthy research into AI tools and the way we use them, we make it possible for AI to act as a catalyst for innovation rather than a source of distortion.

Trustworthy content is only part of the equation. The same principles — authority, accuracy, and transparency — can guide the evaluation of AI tool functionality. Tools designed with these principles at the center can support research acceleration without compromising integrity. This is the direction the research ecosystem must move toward as AI adoption continues to grow.

In a subsequent article, we’ll explore how these values shape the features and reliability of AI solutions for research.