Crawley: A Tool for Web Platform Discovery. Dobriy, D. & Polleres, A. In Proceedings of the 22nd International Semantic Web Conference (ISWC2023) – Posters and Demos Track, 2023. To appear
Crawley: A Tool for Web Platform Discovery [pdf]Paper  abstract   bibtex   
Crawley, a Python-based command-line tool, provides an automated mechanism for web platform discovery. Incorporating capabilities such as Search Engine crawling, web platform validation and recursive hyperlink traversal, it facilitates the systematic identification and validation of a variety of web platforms. The tool’s effectiveness and versatility are demonstrated via two successful use cases: the identification of Semantic MediaWikis instances, as well as the discovery of Open Data Portals including OpenDataSoft, Socrata, and CKAN. These empirical results underscore Crawley’s capacity to support web-based research. We further outline potential enhancements of the tool, thereby positioning Crawley as a valuable tool in the field of web platform discovery.

Downloads: 0