Evaluating NextCloud as a File Storage for Apache Airavata

Evaluating NextCloud as a File Storage for Apache Airavata. Kariyattin, S., Marru, S., & Pierce, M. In Proceedings of the Practice and Experience on Advanced Research Computing - PEARC '18, pages 1-4, 7, 2018. ACM Press.

Paper

Evaluating NextCloud as a File Storage for Apache Airavata [link]

Website doi abstract bibtex

Science gateways enable researchers from broad communities to access advanced computing and storage resources. The researchers analyze large amounts of data using the compute resources and the generated results, usually files are saved in the storage. Consider a scenario where a researcher has large output data files of historically run experiments on an external server. If the researcher wants to move the data to the gateway storage, then the only way to do it is through data transfer. This task would be cumbersome and time consuming. The paper discusses an approach through which historic or any data existing on a different server or in a cloud storage (Google Drive) or in an object storage (Amazon S3) can be ingested into the existing gateway without actually transferring it to the server. We discuss about a software called NextCloud and how it can be used as a gateway storage by integrating it with Apache Airavata. Airavata currently uses local file storage to store user related data files. On the client side, Airavata clients use different protocols like HTTP and SFTP for file transfer. NextCloud is an open source file share and communication platform that provides a common file access layer through its universal file access to different data sources. Integrating NextCloud with Airavata could solve the problem of providing unified file transfer API across all the Airavata clients. As NextCloud supports various external storages, its integration with Airavata would also enable the data ingestion and importing large data from different storage sources to Airavata.

@inproceedings{
 title = {Evaluating NextCloud as a File Storage for Apache Airavata},
 type = {inproceedings},
 year = {2018},
 keywords = {Apache Airavata,File Storage,File Transfer,NextCloud,WebDAV},
 pages = {1-4},
 websites = {http://dl.acm.org/citation.cfm?doid=3219104.3229270},
 month = {7},
 publisher = {ACM Press},
 day = {22},
 city = {New York, New York, USA},
 id = {36bce677-451f-3965-a4d4-f542d3b36a5c},
 created = {2019-10-01T17:20:23.138Z},
 accessed = {2019-08-19},
 file_attached = {true},
 profile_id = {42d295c0-0737-38d6-8b43-508cab6ea85d},
 last_modified = {2020-05-11T14:43:43.797Z},
 read = {false},
 starred = {false},
 authored = {true},
 confirmed = {true},
 hidden = {false},
 citation_key = {Kariyattin2018},
 private_publication = {false},
 abstract = {Science gateways enable researchers from broad communities to access advanced computing and storage resources. The researchers analyze large amounts of data using the compute resources and the generated results, usually files are saved in the storage. Consider a scenario where a researcher has large output data files of historically run experiments on an external server. If the researcher wants to move the data to the gateway storage, then the only way to do it is through data transfer. This task would be cumbersome and time consuming. The paper discusses an approach through which historic or any data existing on a different server or in a cloud storage (Google Drive) or in an object storage (Amazon S3) can be ingested into the existing gateway without actually transferring it to the server. We discuss about a software called NextCloud and how it can be used as a gateway storage by integrating it with Apache Airavata. Airavata currently uses local file storage to store user related data files. On the client side, Airavata clients use different protocols like HTTP and SFTP for file transfer. NextCloud is an open source file share and communication platform that provides a common file access layer through its universal file access to different data sources. Integrating NextCloud with Airavata could solve the problem of providing unified file transfer API across all the Airavata clients. As NextCloud supports various external storages, its integration with Airavata would also enable the data ingestion and importing large data from different storage sources to Airavata.},
 bibtype = {inproceedings},
 author = {Kariyattin, Sachin and Marru, Suresh and Pierce, Marlon},
 doi = {10.1145/3219104.3229270},
 booktitle = {Proceedings of the Practice and Experience on Advanced Research Computing - PEARC '18}
}

Downloads: 0