Home >The Catalogue>Services> PSNC, Long-term, massive data storage @PSNC
SERVICES

PSNC, Long-term, massive data storage @PSNC

TYPE
Data value chain
REGION
-
LANG
English, Polish

PSNC provides massive scale long-term data storage services for academia, R&D and commercial customers. There are two generations and flavours of the service - Generation 1 service: file storage-based, provided since 2010 and Generation 2 service object storage-based - provided since 2015.

Backed by the on-premise data storage, data management infrastructure at PSNC and in partner’s sites, these services ensure that data are stored safely, replicated and are treated according to the local regulations on data storage and privacy.

Generation 1 of the service provides long-term reliable data storage service enabling safe, persistent storage of large volumes of scientific and backup/archive data. The service is available through file protocols: SSH, SFTP WebDAV and GridFTP (currently mostly SFTP). It provides tape-based storage space (12+PB overall) with disk cache front-end (2+PB overall), automated data migration among disk and tape media, automated staging on data access, geographical data replication and integrity checks.

The Generation 2 service is targetted to academic and scientific users and commercial clients. It is based on the object storage systems and services based on the opens source Software Defined Storage systems (Ceph, Swift). The basic access protocol is S3, Swift can be enabled on user’s request. The service is currently oferred based on PSNC infrastructure. Fully-distributed, country-wide service is in preparation within the nationally-funded project involving most partners of the Generation 1 service (9 data centres in Poland). This will ensure coverage of the customer locations and enable data geo-replication for higher data safety.

SERVICE DESCRIPTION

PSNC provides massive scale long-term data storage services for academia, R&D and commercial customers. There are two generations and flavours of the service - Generation 1 service: file storage-based, provided since 2010 and Generation 2 service object storage-based - provided since 2015.

These service constitute alternatives to on-premise storage at user institutions on one hand and to commercial, public cloud data storage / archival services such as Amazon Glacier and other local and global offerings on the other hand.
Backed by the on-premise data storage, data management infrastructure at PSNC and in partner’s sites, these services ensure that data are stored safely, replicated and are treated according to the local regulations on data storage and privacy.

Generation 1 of the service provides long-term reliable data storage service enabling safe, persistent storage of large volumes of scientific and backup/archive data. The service is available through file protocols: SSH, SFTP WebDAV and GridFTP (currently mostly SFTP). It provides tape-based storage space (12+PB overall) with disk cache front-end (2+PB overall), automated data migration among disk and tape media, automated staging on data access, geographical data replication and integrity checks.
The service is provided by PSNC along with the PIONIER (polish NREN) partners. Partners’ sites are distributed which ensures coverage of the country’s client locations as well as enables safe data storage and data/meta-data geo-replication.

The Generation 2 service is targeted to academic and scientific users and commercial clients. It is based on the object storage systems and services based on the opens source Software Defined Storage systems (Ceph, Swift). The basic access protocol is S3, Swift can be enabled on user’s request. The service is currently offered based on PSNC infrastructure. Fully-distributed, country-wide service is in preparation within the nationally-funded project involving most partners of the Generation 1 service (9 data centres in Poland). This will ensure coverage of the customer locations and enable data geo-replication for higher data safety.

The services are targeted mostly to institutions, systems administrators responsible for operational safety of the IT infrastructure (backups) and R&D projects, where massive volumes of data are created and must be protected (archives). Use of the service by individuals hasn’t been wide. Commercial use is increasing, mostly for the Generation 2 service, due to its support for widely adopted S3 data access protocol.

SPECIAL ACCESS CONDITIONS

Generation 1 service is targeted mostly to academics institutions and R&D projects. Users access is based on X.509 or SSL certificates (for high safety), simplified authentication (SSH keys) is also supported on request. Users must acquire credentials from relevant body (e.g. CA) before using the service. The default storage space is 10+TB and can be increased for institutions and R&D projects case-by-case.
Generation 2 service is targeted both to academic and commercial sector. Users are authenticated based S3 credentials, that can be acquired from the service provider.
The default storage space is 10+TB and can be increased for academic case-by-case and 1TB for paying customers, and can be extended, based on the service pricelist.

PREREQUISITES

The basic services are available through the set of data access protocols: Generation 1 (SSH, SFTP) WebDAV and GridFTP; Generation 2: S3, Swift. Workstation users must install client software for relevant protocol - open source and commercial applications are widely available both for SFTP/WebDAV and S3/Swift.

Mobile devices access is limited - thus there are applications supporting SFTP or S3, however it is not recommended for handling large data sets.

Virtual filesystem clients (Linux: based on FUSE, MacOS: based on MacFUSE, Windows: based on Dokany, CallBackFS etc.) are available (some of them paid).

CASE EXAMPLES

Generation 1 service’s most typical use-case is long-term storage of data sets for R&D project and activities - research data are protected for future reference etc. It is also used by institutions accross Poland (300+) for backup data storage - as the ‘target’ that stores data for the B/A applications/engine run on users’ premises.

Generation 2 services is mostly used for storing large volume data that does not fit the ‘traditional’, local storage systems/services in PSNC cloud/computing/HPC platform and by external customers for storing operational backups and research archival data. The service also provides storage back-end for R&D content and data such as scientific repositories, digital cultural heritage repositories, video portals etc.

Commercial use of Generation 2 service focus on long-term datasets storage as well as storage of backup data for B/A applications/engine run on clients’ premises. There is increasing interest in commercial use of S-based service as the backend for multi-media repositories, content serving and distribution application etc

LINKS

Service portal: https://storage.pionier.net.pl
UHD content use-case presentation: https://tnc18.geant.org/core/presentation/162

SERVICE OFFERED BY

MEMBER
HPC4 Poland
TYPE
DIH
COUNTRY
Poland

MORE INFORMATION ABOUT THIS SERVICE