A system that converts research papers and their code into callable AI tools successfully processed 74 of 100 computational biology papers without manual cleanup. Paper2Agent proposed 599 tools from those papers, and 593 passed its automated checks. The result offers a measurable route from reading a method to running it, while showing that missing code, data and working environments still block more than a quarter of the papers tested. [1] [2]
Computational papers often publish a manuscript, repository and tutorial as separate objects. A researcher who wants to reuse the method must locate the right code, install its dependencies, identify valid inputs and reconstruct the order of operations. Paper2Agent packages that work as a Model Context Protocol, or MCP, server. MCP is a standard interface through which an AI assistant can discover and call named functions, read resources and follow stored workflow instructions. [1]
The conversion follows the same order a careful user would. Specialist agents find and clone the repository, build an isolated environment, locate tutorials and execute them with their example data. Another agent turns generalizable steps into functions with file-based inputs and outputs. A verifier then compares generated files, numbers and figures with the tutorial references, allowing up to six repair attempts for each function. A function that still fails loses its MCP label and is excluded from the server. [1] [2]
The large evaluation sampled 100 computational biology papers by moving backwards from December 2025 in the bioRxiv bioinformatics category, without filtering for documentation quality or maintenance. The 26 failures came from missing executable code, unavailable data or model files, dependency problems and scripts that could not be generalized. On 300 tutorial-derived questions, Paper2Agent using Claude Sonnet 4 scored 91.2%, compared with 80.3% for Sonnet 4 given the repository directly and 86.3% when that baseline used the newer Sonnet 4.6. These are internal paper benchmarks, not a comparison with every scientific-agent system. [1]
A separate AlphaGenome case shows what the packaging changes in practice. The system built 22 validated genomics tools in about 45 minutes, then exposed them through an agent that could score variants and generate visualizations. Across five runs of 15 novel questions, it matched the curated answers on all questions, while an assistant with direct repository access averaged 78.7%. Two domain experts graded the outputs, and the study's accuracy measures agreement with those reference answers rather than the validity of every possible biological interpretation. [1] [3]
The evidence supports using paper agents as tested interfaces to published work, especially when a team already needs to repeat a documented workflow. It does not make the agents authoritative scientists: open-ended reasoning remains under human review, and generated servers inherit security, licensing and maintenance obligations from their source packages. The next useful test is an independent lab reproducing the same analyses after dependencies and data have changed, while recording how often human intervention is needed and whether the scientific conclusions remain correct. [1] [2]