How to download Copernicus Contributing Mission products to object storage

Copernicus Contributing Mission (CCM) products can be ordered in EO Finder https://finder.eo-lab.org. Most of the products can be found in a common repository, when the order is completed. CCM products are ordered differently and can be downloaded to Object Storage only. This manual describes how to order and download CCM products.

Object Storage

First, you need to create an Object Storage. Object Storage can be accessed via the OpenStack Dashboard (Horizon) and console. The Dashboard is a convenient way to manipulate your Object Storages. Go to https://cloud.fra1-1.cloudferro.com/auth/login/?next=/ and log in with SAML2 authentication. Choose a ProjectObject Store -> Containers tab on the left.

../_images/copernicus1.png

Choose “+ Container” and select a name for your new bucket.

../_images/copernicus2.png

You should see newly added container in your Object Storage list.

../_images/copernicus3.png

Console Access

The following steps will require to have applications like openstack-client and s3fs. Most modern distributions will propose to install them from default distribution repositories.

Let’s try to set up console access to your bucket from the console. Access can be performed in your virtual machine or local Linux machine. In order to have access first, you need to download an RC file. Go to your Dashboard and navigate to ProjectAPI Access. Select “Download OpenStack RC File” button and press “OpenStack RC File”.

../_images/copernicus4.png

Save the file in a directory of your choice and go to the same direction in a console. Run the following command to import RC file data. You will be asked to provide a password which is the same as your EO-Lab.org profile password.

source ./cloud_00XXX_1-openrc.sh
Please enter your OpenStack Password for project cloud_00XXX_1 as user

When the password is provided correctly you are authenticated. There will be no prompt about the wrong password in case of a mistake. If the next command results with “The request you have made requires authentication.” message, just run source command once again and provide the correct password.

Run an OpenStack command to obtain EC2 credentials:

openstack ec2 credentials list

You should see the following output similar to this:

+----------------------------------+----------------------------------+----------------------------------+----------------------------------+
| Access                           | Secret                           | Project ID                       | User ID                          |
+----------------------------------+----------------------------------+----------------------------------+----------------------------------+
| e82b581s08bk4doiu2344g19f79f112b | 88ba74d9h3alow5761dce904047652aa | 8b939edi8n3f0a7fk5890ef4cced1e0a | 55a0826d0fl6sm2v0td953vf989fc873 |
+----------------------------------+----------------------------------+----------------------------------+----------------------------------+

Note down Access and Secret keys. They will be needed later in Finder ordering. In the meantime, you can check them to browse your Object Storage bucket from a console. Create a credentials file which includes Access and Secret keys:

echo Access_key:Secret_key > ~/.passwd-s3fs

This command will create a file in your home directory. Now you can mount your bucket with the following command:

s3fs container /mnt/container -o passwd_file=~/.passwd-s3fs -o url=https://cloud.fra1-1.cloudferro.com/auth/login/?next=/:8080 -o use_path_request_style -o umask=0002 -o allow_other

“container” should be changed to your container name and /mnt/container to your favour mount point.

Access to Copernicus Contributing Mission Products

CCM products require to have working CDS-SSO credentials. More information about Copernicus SSO account can be found on the official Copernicus webpage: ssoidp.copernicus.eu. To register an account navigate to the Registration page and provide required data: https://ssoidp.copernicus.eu/idp/umsso20/registration. CDS-SSO ID and Password will be used during the ordering process in Finder. In order to have access to the chosen repository, you need to report it into Copernicus Portal in “Personal Area” and request for the access. Fill in “Account Details” and”User Category” and submit the request.

CCM Catalogue in EO-Lab Finder

Open https://finder.eo-lab.org and login with EO-Lab credentials. Select “Copernicus Contributing Mission” option from Data Catalogue.

../_images/copernicus5.png

There will a Collection also to choose from:

../_images/copernicus6.png

Select processor to CCM download.

../_images/copernicus7.png

And press search. Select products you want to add to order.

../_images/copernicus9.png

When products are added select Cart button in the top right corner:

../_images/copernicus8.png

Now it is time to use EC2 credentials which were generated with “openstack ec2 credentials list” command. s3 bucket is the name of created object storage in OpenStack Dashboard. CDS_SSO username and password were created during registration on Copernicus Portal.

../_images/copernicus10.png

Now you can finish with an Order button. When the order will be completed products will be available in your Object Storage.