SciDER: Scientific Data-centric End-to-end Researcher
Abstract
SciDER automates scientific research by processing raw experimental data through collaborative agents that generate hypotheses and experimental designs while executing code, demonstrating superior performance in data-driven discovery compared to general-purpose models.
Automated scientific discovery with large language models is transforming the research lifecycle from ideation to experimentation, yet existing agents struggle to autonomously process raw data collected from scientific experiments. We introduce SciDER, a data-centric end-to-end system that automates the research lifecycle. Unlike traditional frameworks, our specialized agents collaboratively parse and analyze raw scientific data, generate hypotheses and experimental designs grounded in specific data characteristics, and write and execute corresponding code. Evaluation on three benchmarks shows SciDER excels in specialized data-driven scientific discovery and outperforms general-purpose agents and state-of-the-art models through its self-evolving memory and critic-led feedback loop. Distributed as a modular Python package, we also provide easy-to-use PyPI packages with a lightweight web interface to accelerate autonomous, data-driven research and aim to be accessible to all researchers and developers.
Community
SciDER is designed as a data-centric end-to-end system that flexibly automates the scientific research lifecycle. The system integrates a research framework comprising ideation, data analysis, experimentation, and iterative improvement. It supports flexible inputs such as text, raw data, code, and prior papers and codebases. SciDER also offers a lightweight web interface where researchers
can upload their data and research topics, allowing the system to automatically create a closed-loop research cycle to propose and verify new ideas.
The contributions are threefold:
- We introduce SciDER, a modular system that automates the full research lifecycle through specialized agents and an innovative self-evolving memory mechanism that supports continuous test-time memorizing and learning.
- We propose a data-centric approach that grounds code generation of experiments in autonomous experimental analysis, enabling superior performance on interdisciplinary research problems.
- Extensive empirical analyses that SciDER greatly outperforms current baselines on AI-Idea-Bench, MLEBench, and SciCode benchmarks, demonstrating its efficacy in managing challenging scientific reasoning and coding tasks at the research level.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Idea2Story: An Automated Pipeline for Transforming Research Concepts into Complete Scientific Narratives (2026)
- S1-NexusAgent: a Self-Evolving Agent Framework for Multidisciplinary Scientific Research (2026)
- DSGym: A Holistic Framework for Evaluating and Training Data Science Agents (2026)
- FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents (2026)
- DataCross: A Unified Benchmark and Agent Framework for Cross-Modal Heterogeneous Data Analysis (2026)
- Why LLMs Aren't Scientists Yet: Lessons from Four Autonomous Research Attempts (2026)
- Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoning (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper