CCRI: Research Infrastructure: NEW: Semantic Scholar Open Data Platform: Enabling Research Into Scientific Search and Discovery

The exponential growth of scientific publication makes it difficult for scientists to track developments in their field and make connections between different advances. In response, artificial-intelligence researchers have started to develop techniques that allow computers to ‘read’ scientific papers and automatically classify topics, extract key results, summarize contributions, identify connections, and select a personalized set of papers that may be of special interest to each scientist.

The enduring vision is to build AI systems that can process an immense corpus of scholarly documents and augment the capabilities of human scientists – accelerating scientific discovery and helping humanity quickly confront disasters such as the COVID-19 pandemic. The proposed Semantic Scholar Open Data Platform builds infrastructure to support this research by first gathering a comprehensive set of papers and arranging for efficient indexing.

The system processes PDF-formatted papers to extract information and use advanced analytic processing approaches to provide researchers access to results. The infrastructure will dramatically lower the barrier to entry for newcomers to the field of scholarly document processing, improve reproducibility of experiments, and accelerate innovation in the important area of AI-augmented scientific discovery

The infrastructure proposed is unique, because alternative sources of academic papers are either closed, incomplete, have limited programmatic access, or have been retired. The proposed Semantic Scholar Open Data Platform has three parts: 1) a comprehensive set of online services enabling researchers to programmatically search, filter, extract, summarize, and analyze a large and continually-updated corpus of documents; 2) a new mechanism that enables researchers to curate their own domain-specific text corpora, as the team previously created the CORD-19 dataset for coronavirus research; 3) open source software, including pretrained language models and user interface templates to serve as research building blocks.

Together the infrastructure will dramatically lower the barrier to entry for newcomers to the field of scholarly document processing, improve reproducibility of experiments, and accelerate innovation in the important area of AI-augmented scientific discovery. Fortunately, the recent increase in research in scholarly document processing (e.g., the rapid uptake of our CORD-19 dataset) shows that the computer and information science community has the interest and capability to develop new technologies that accelerate science and help meet global societal challenges, such as pandemics and climate change.

The resulting advances in AI-augmented scientific discovery will benefit all areas of science, spurring medical advances, creating new jobs, and improving access for blind researchers. We will improve global infrastructure by providing open services, data sets, code, and associated educational materials. The team will also engage with underrepresented STEM students and through K-12 outreach.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.