EOData Catalogue API Manual on EO-Lab

OData is Open Data Protocol for building and consuming RESTful APIs. Although similarly named, EODATA are Earth Observation data archives and they can be accessed via links that follow the OData standards. OpenSearch is open-source search and analytics suite used for real-time application monitoring, log analytics, and website search.

In this article, you will learn how to search the EODATA catalogue using OData and OpenSearch.

Prerequisites

No. 1 Ability to send HTTP requests

The simplest form of sending HTTP request is to execute Internet address in a browser. Most browsers will understand the output from a site and will present it in a usable manner.

From a command line in a desktop operating system, you can use program called curl, which will execute the address but represent it in a purely textual manner. If curl is not preinstalled, you can install it on your own. On Ubuntu 22.04, you would use

sudo apt install curl

Other Linux distributions will have an identical or a very similar command.

On Windows, curl will be either preinstalled or you can install it from a cmd window.

Depending on the type and version of the operating system on which curl is running, you may have problems with some characters. Be sure to consult the relevant documentation.

There are alternatives to curl but are out of scope of this article.

What We Are Going To Cover

OData

  • Query structure

    • Query by name

    • Query by list

    • Query collection of products

    • Query by publication date

    • Query by sensing date

    • Query by geographic criteria

    • Query by attributes

  • Orderby option

  • Top option

  • Skip option

  • Count option

  • Expand option

    • Expand option - Attributes

    • Expand option - Assets

    • Expand option - Locations

  • Listing product Nodes

OpenSearch

  • General rules

  • Collections

  • Output sorting and limiting

  • Formal queries

  • Geography and time-frame

  • Volatile features

DeletedProducts OData API

  • Endpoint URL

  • Query structure

  • Filter option

  • Query by name

  • Query by ID

  • Query by deletion date

  • Query by deletion cause

  • Query by collection of products

  • Query by sensing date

  • Query by attributes

Additional options

  • Orderby option

  • Expand option

  • Skip option

  • Top option

  • Count option

OData

Query structure

As a general note, OData query consists of elements which in this documentation are called “options”. Interface supports the following search options:

  • filter

  • orderby

  • top

  • skip

  • count

  • expand

Search options should always be preceded with $ and consecutive options should be separated with &.

Consecutive filters within filter option should be separated with and or or. Not operator can also be used e.g.:

https://datahub.eo-lab.org/odata/v1/Products?$filter=not contains(Name,'S2') and ContentDate/Start gt 2022-05-03T00:00:00.000Z and ContentDate/Start lt 2022-05-03T00:10:00.000Z&$orderby=ContentDate/Start&$top=100

To accelerate the query performance, it is recommended to limit the query by acquisition dates e.g.:

ContentDate/Start gt 2022-05-03T00:00:00.000Z and ContentDate/Start lt 2022-05-21T00:00:00.000Z

Filter option

Query by name

To search for a specific product by its exact name:

https://datahub.eo-lab.org/odata/v1/Products?$filter=Name eq 'S1A_IW_GRDH_1SDV_20141031T161924_20141031T161949_003076_003856_634E.SAFE'

To search for products containing S1A in their names:

https://datahub.eo-lab.org/odata/v1/Products?$filter=contains(Name,'S1A') and ContentDate/Start gt 2022-05-03T00:00:00.000Z and ContentDate/Start lt 2022-05-21T00:00:00.000Z

Alternatively to contains, endswith and startswith can be used, to search for products ending or starting with the provided string.

Query by list

In case a user desires to search for multiple products by name in one query, POST method can be used:

https://datahub.eo-lab.org/odata/v1/Products/OData.CSC.FilterList

Request body:

{
    "FilterProducts":
    [
        {"Name": "S1A_IW_GRDH_1SDV_20141031T161924_20141031T161949_003076_003856_634E.SAFE"},
        {"Name": "S3B_SL_1_RBT____20190116T050535_20190116T050835_20190117T125958_0179_021_048_0000_LN2_O_NT_003.SEN3"},
        {"Name": "xxxxxxxx.06.tar"}
    ]
}

Two results are returned, as there is no product named xxxxxxxx.06.tar.

Query collection of products

To search for products within a specific collection:

https://datahub.eo-lab.org/odata/v1/Products?$filter=Collection/Name eq 'SENTINEL-2' and ContentDate/Start gt 2022-05-03T00:00:00.000Z and ContentDate/Start lt 2022-05-03T00:11:00.000Z

The following collections are currently available:

SENTINEL-1

SENTINEL-2

SENTINEL-3

SENTINEL-5P

SENTINEL-6

SENTINEL-1-RTC

LANDSAT-5

LANDSAT-7

LANDSAT-8

SMOS

TERRAAQUA

COP-DEM

ENVISAT

S2GLC

Query by publication date

To search for products published between two dates:

https://datahub.eo-lab.org/odata/v1/Products?$filter=PublicationDate gt 2019-05-15T00:00:00.000Z and PublicationDate lt 2019-05-16T00:00:00.000Z

To define inclusive interval ge and le parameters can be used:

https://datahub.eo-lab.org/odata/v1/Products?$filter=PublicationDate ge 2019-05-15T00:00:00.000Z and PublicationDate le 2019-05-16T00:00:00.000Z

Query by sensing date

To search for products acquired between two dates:

https://datahub.eo-lab.org/odata/v1/Products?$filter=ContentDate/Start gt 2019-05-15T00:00:00.000Z and ContentDate/Start lt 2019-05-16T00:00:00.000Z

Usually, there are two parameters describing the ContentDate (Acquisition Dates) for a product - Start and End. Depending on what the user is looking for, these parameters can be mixed, e.g.:

https://datahub.eo-lab.org/odata/v1/Products?$filter=ContentDate/Start gt 2019-05-15T00:00:00.000Z and ContentDate/End lt 2019-05-15T00:05:00.000Z

Query by geographic criteria

To search for products intersecting the specified polygon:

https://datahub.eo-lab.org/odata/v1/Products?$filter=OData.CSC.Intersects(area=geography'SRID=4326;POLYGON((12.655118166047592 47.44667197521409,21.39065656328509 48.347694733853245,28.334291357162826 41.877123516783655,17.47086198383573 40.35854475076158,12.655118166047592 47.44667197521409))') and ContentDate/Start gt 2022-05-20T00:00:00.000Z and ContentDate/Start lt 2022-05-21T00:00:00.000Z

To search for products intersecting the specified point:

https://datahub.eo-lab.org/odata/v1/Products?$filter=OData.CSC.Intersects(area=geography%27SRID=4326;POINT(-0.5319577002158441%2028.65487836189358)%27)

Disclaimers:

  1. MULTIPOLYGON is currently not supported.

  2. Polygon must start and end with the same point.

  3. Coordinates must be given in EPSG 4326

Query by attributes

To search for products by attributes it is necessary to build a filter with the following structure:

Attributes/OData.CSC.ValueTypeAttribute/any(att:att/Name eq '[Attribute.Name]' and att/OData.CSC.ValueTypeAttribute/Value eq '[Attribute.Value]')

where

  • ValueTypeAttribute can take the following values:

    • StringAttribute

    • DoubleAttribute

    • IntegerAttribute

    • DateTimeOffsetAttribute

  • [Attribute.Name] is the attribute name which can take multiple values, depending on collection (Attachment 1 - Coming soon)

  • eq before [Attribute.Value] can be substituted with le, lt, ge, gt in case of Integer, Double or DateTimeOffset Attributes

  • [Attribute.Value] is the specific value that the user is searching for

To get products with cloudCover < 40% between two dates:

https://datahub.eo-lab.org/odata/v1/Products?$filter=Attributes/OData.CSC.DoubleAttribute/any(att:att/Name eq 'cloudCover' and att/OData.CSC.DoubleAttribute/Value le 40.00) and ContentDate/Start gt 2022-01-01T00:00:00.000Z and ContentDate/Start lt 2022-01-03T00:00:00.000Z&$top=10

To get products with cloudCover< 10% and productType=S2MSI2A and ASCENDING orbitDirection between two dates:

https://datahub.eo-lab.org/odata/v1/Products?$filter=Attributes/OData.CSC.DoubleAttribute/any(att:att/Name eq 'cloudCover' and att/OData.CSC.DoubleAttribute/Value lt 10.00) and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'S2MSI2A') and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'orbitDirection' and att/OData.CSC.StringAttribute/Value eq 'ASCENDING') and ContentDate/Start gt 2022-05-03T00:00:00.000Z and ContentDate/Start lt 2022-05-03T04:00:00.000Z&$top=10

Orderby option

Orderby option can be used to order the products in an ascending (asc) or descending (desc) direction. If asc or desc not specified, then the resources will be ordered in ascending order.

To order products by ContentDate/Start in a descending direction:

https://datahub.eo-lab.org/odata/v1/Products?$filter=contains(Name,'S1A_EW_GRD') and ContentDate/Start gt 2022-05-03T00:00:00.000Z and ContentDate/Start lt 2022-05-03T03:00:00.000Z&$orderby=ContentDate/Start desc

By default, if orderby option is not used, the results are not ordered. If orderby option is used, additional orderby by id is also used, so that the results are fully ordered and no products are lost while paginating through the results.

The acceptable arguments for this option: ContentDate/Start, ContentDate/End, PublicationDate, ModificationDate, in directions: asc, desc

Top option

Top option specifies the maximum number of items returned from a query.

To limit the number of results:

https://datahub.eo-lab.org/odata/v1/Products?$filter=contains(Name,%27S1A_EW_GRD%27)%20and%20ContentDate/Start%20gt%202022-05-03T00:00:00.000Z%20and%20ContentDate/Start%20lt%202022-05-03T12:00:00.000Z&$top=100

The default value is set to 20.

The acceptable arguments for this option: Integer <0,1000>

Skip option

Skip option can be used to skip a specific number of results. Exemplary application of this option would be paginating through the results, however for performance reasons, we recommend limiting queries with small time intervals as a substitute of using skip in a more generic query.

To skip a specific number of results:

https://datahub.eo-lab.org/odata/v1/Products?$filter=contains(Name,%27S1A_EW_GRD%27)%20and%20ContentDate/Start%20gt%202022-05-03T00:00:00.000Z%20and%20ContentDate/Start%20lt%202022-05-03T12:00:00.000Z&$skip=23

The default value is set to 0.

Whenever a query results in more products than 20 (default top value), the API provides a nextLink at the bottom of the page:

"@odata.nextLink": "http://datahub.eo-lab.org/odata/v1/Products?$filter=contains(Name,'S1A_EW_GRD')+and+ContentDate/Start+gt+2022-05-03T00:00:00.000Z+and+ContentDate/Start+lt+2022-05-03T12:00:00.000Z&$skip=20"

The acceptable arguments for this option: Integer <0,10000>

Count option

Count option enables users to get the exact number of products matching the query. This option is disabled by default to accelerate the query performance.

To get the exact number of products for a given query:

https://datahub.eo-lab.org/odata/v1/Products?$filter=contains(Name,%27S1A_EW_GRD%27)%20and%20ContentDate/Start%20gt%202022-05-03T00:00:00.000Z%20and%20ContentDate/Start%20lt%202022-05-03T12:00:00.000Z&$count=True

The acceptable arguments for this option: True, true, 1, False, false, 0

Expand option

Expand option enables users to choose what kind of information do they want to see in detail.

The acceptable arguments for this option: Attributes, Assets, Locations

Expand option - Attributes

To see full metadata of the results, assign value Attributes to Expand option:

https://datahub.eo-lab.org/odata/v1/Products?$filter=contains(Name,%27S1A_EW_GRD%27)%20and%20ContentDate/Start%20gt%202022-05-03T00:00:00.000Z%20and%20ContentDate/Start%20lt%202022-05-03T12:00:00.000Z&$expand=Attributes

For each product, the list of these attributes is available as value of key Attributes. In query above, as of writing of this article, for the first product on the list the beginning of this list will look like this:

../_images/eodata_catalogue_api_manual_01_creodias.png

Expand option - Assets

To see a list of additional assets for products, including quicklooks, use value Assets:

https://datahub.eo-lab.org/odata/v1/Products?$filter=Collection/Name%20eq%20%27SENTINEL-3%27%20and%20contains(Name,%20%27SL_2_FRP___%27)&$expand=Assets

The list of assets is available as value of key Assets. For the first product found by query above (as of writing of this article), this list is as follows:

"Assets": [
     {
       "Type": "QUICKLOOK",
       "Id": "312dbf31-85c6-4613-aefa-ed8f04e5c6eb",
       "DownloadLink": "https://datahub.eo-lab.org/odata/v1/Assets(312dbf31-85c6-4613-aefa-ed8f04e5c6eb)/$value",
       "S3Path": "/eodata/Sentinel-3/SLSTR/SL_2_FRP___/2020/11/27/S3A_SL_2_FRP____20201127T201256_20201127T201556_20201129T020232_0180_065_285_0900_LN2_O_NT_004.SEN3"
     }
]

In this example, this list contains only one item which has the following Type: QUICKLOOK. The download link for that item is available as value of key DownloadLink:

https://datahub.eo-lab.org/odata/v1/Assets(312dbf31-85c6-4613-aefa-ed8f04e5c6eb)/$value

Note that you can download these assets using these links without having to prove your identity.

Expand option - Locations

Products can be available to download in extracted (uncompressed) and/or compressed form, depending on the product.

If you want to see what download methods are available for products you are searching for, assign value Locations to expand option:

https://datahub.eo-lab.org/odata/v1/Products?$filter=Collection/Name%20eq%20%27SENTINEL-1%27%20and%20contains(Name,%27RAW%27)&$orderby=ContentDate/Start%20desc&$top=10&$expand=Locations

You should now get a list of locations as value of key Locations. Each of these locations represents one of available ways to download a product and contains information such as size, checksum and download link.

For example, as of writing of this article, for the first product returned by query above, this list looks like this:

"Locations": [
    {
      "FormatType": "Compressed",
      "DownloadLink": "https://datahub.eo-lab.org/odata/v1/Products(d695b74e-5f82-4b4c-95be-f08a631fac53)/$zip",
      "ContentLength": 1575276240,
      "Checksum": [
        {
          "Value": "46336ae3fd72617177b6d6bd6ee755b3",
          "Algorithm": "MD5",
          "ChecksumDate": "2024-05-28T10:54:29.721000Z"
        }
      ],
      "EvictionDate": "2024-06-27T11:00:26.387Z"
    },
    {
      "FormatType": "Extracted",
      "DownloadLink": "https:/datahub.eo-lab.org/data/v1/Products(d695b74e-5f82-4b4c-95be-f08a631fac53)/$value",
      "ContentLength": 1575275488,
      "Checksum": [
        {
          "Value": "054c188b645c4b8ce7ab7b65abc1a9f0",
          "Algorithm": "MD5",
          "ChecksumDate": "2024-05-28T11:01:34.881769Z"
        },
        {
          "Value": "2c1f95bfc5f88a455061a41f689222512e620fbda8ea2cdea5f5206c32472644",
          "Algorithm": "BLAKE3",
          "ChecksumDate": "2024-05-28T11:01:37.330063Z"
        }
      ],
      "S3Path": "/eodata/Sentinel-1/SAR/IW_RAW__0S/2024/05/28/S1A_IW_RAW__0SDV_20240528T102453_20240528T102523_054070_06930F_897A.SAFE"
    }
  ]

If you want to download compressed form of this product, you can use the following link:

https://datahub.eo-lab.org/odata/v1/Products(d695b74e-5f82-4b4c-95be-f08a631fac53)/$zip

To perform this download, you will need to prove your identity, contrary to situation when you are downloading assets as explained in section above. You can do it using a Keycloak token - see article linked in What To Do Next to learn more.

Listing product Nodes

You can explore files and folders found in a product without downloading it, even if you are not authorized.

Start by accessing the following URL:

https://datahub.eo-lab.org/odata/v1/Products(<PRODUCT_UUID>)/Nodes

where <PRODUCT_UUID> is ID of a product obtained by using a search query. One of methods which can be used for obtaining this ID is by accessing URL found in Query by name above. The ID can then be found as a value of key Id.

Once you have found the ID of the product and accessed the URL ending with the phrase Nodes mentioned above, you should see (as value of key uri) link which allows you to view contents of root directory of the product.

Links to files and folders can be found as values of keys uri. You can use them to go further down the directory structure.

Basically, you can explore products with the following URL patterns:

https://datahub.eo-lab.org/odata/v1/Products(<PRODUCT_UUID>)/Nodes
https://datahub.eo-lab.org/odata/v1/Products(<PRODUCT_UUID>)/Nodes(<NODE_NAME>)/Nodes
https://datahub.eo-lab.org/odata/v1/Products(<PRODUCT_UUID>)/Nodes(<NODE_NAME>)/Nodes(<NODE_NAME>)/Nodes

where

<PRODUCT_UUID>

is ID of the product obtained by search query, and

<NODE_NAME>

is name of element (regular file or directory) inside product returned from previous listing response.

Each of the nodes represents file or folder found within the product.

Only nodes that are folders can have their contents listed. If a folder is empty, you will receive an empty list:

{"result":[]}

Attempting to list a Node for a file also returns an empty list.

Example:

Let’s say that we want to explore the product S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE. First, we must find its ID. One way to achieve this goal is by using method from section Query by name above. In this case, the link used for obtaining ID of our product is:

https://datahub.eo-lab.org/odata/v1/Products?$filter=Name eq 'S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE'

The output should look like this:

{"@odata.context":"$metadata#Products","value":[{"@odata.mediaContentType":"application/octet-stream","Id":"db0c8ef3-8ec0-5185-a537-812dad3c58f8","Name":"S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE","ContentType":"application/octet-stream","ContentLength":0,"OriginDate":"2018-09-27T09:58:32.232Z","PublicationDate":"2018-09-27T10:00:58.635Z","ModificationDate":"2018-09-27T10:00:58.635Z","Online":true,"EvictionDate":"","S3Path":"/eodata/Sentinel-2/MSI/L1C/2018/09/27/S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE","Checksum":[],"ContentDate":{"Start":"2018-09-27T05:12:21.024Z","End":"2018-09-27T05:12:21.024Z"},"Footprint":"geography'SRID=4326;POLYGON ((71.341364137137 -48.727041539228, 71.646935006865 -48.789996101558, 71.593710420137 -48.929941890079, 71.537783906556 -49.075177891591, 71.48146556012 -49.220381137077, 71.424930990672 -49.365672690881, 71.368169948403 -49.511156280189, 71.310581782625 -49.656891295565, 71.287247481622 -49.715681890939, 70.387675347677 -49.732377517566, 70.360279246284 -48.744984111698, 71.341364137137 -48.727041539228))'","GeoFootprint":{"type":"Polygon","coordinates":[[[71.341364137137,-48.727041539228],[71.646935006865,-48.789996101558],[71.593710420137,-48.929941890079],[71.537783906556,-49.075177891591],[71.48146556012,-49.220381137077],[71.424930990672,-49.365672690881],[71.368169948403,-49.511156280189],[71.310581782625,-49.656891295565],[71.287247481622,-49.715681890939],[70.387675347677,-49.732377517566],[70.360279246284,-48.744984111698],[71.341364137137,-48.727041539228]]]}}]}

As you can see from the output, the ID of this product is db0c8ef3-8ec0-5185-a537-812dad3c58f8. The link used for exploring the product will look like this:

https://datahub.eo-lab.org/odata/v1/Products(db0c8ef3-8ec0-5185-a537-812dad3c58f8)/Nodes

Response:

{
   "result":[
      {
         "Id":"S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE",
         "Name":"S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE",
         "ContentLength":0,
         "ChildrenNumber":9,
         "Nodes":{
            "uri":"https://datahub.eo-lab.org/odata/v1/Products(db0c8ef3-8ec0-5185-a537-812dad3c58f8)/Nodes(S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE)/Nodes"
         }
      }
   ]
}

Every Listed Node has uri field, which lists its children.

Access the only uri address visible in output above, which is:

https://datahub.eo-lab.org/odata/v1/Products(db0c8ef3-8ec0-5185-a537-812dad3c58f8)/Nodes(S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE)/Nodes

You should now see the list of items from the root directory of the product. These are some (not all) of them:

{
  "result": [
    {
      "Id": "AUX_DATA",
      "Name": "AUX_DATA",
      "ContentLength": 0,
      "ChildrenNumber": 0,
      "Nodes": {
        "uri": "https://datahub.eo-lab.org/odata/v1/Products(db0c8ef3-8ec0-5185-a537-812dad3c58f8)/Nodes(S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE)/Nodes(AUX_DATA)/Nodes"
      }
    },
    {
      "Id": "DATASTRIP",
      "Name": "DATASTRIP",
      "ContentLength": 0,
      "ChildrenNumber": 1,
      "Nodes": {
        "uri": "https://datahub.eo-lab.org/odata/v1/Products(db0c8ef3-8ec0-5185-a537-812dad3c58f8)/Nodes(S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE)/Nodes(DATASTRIP)/Nodes"
      }
    },
    {
      "Id": "GRANULE",
      "Name": "GRANULE",
      "ContentLength": 0,
      "ChildrenNumber": 1,
      "Nodes": {
        "uri": "https://datahub.eo-lab.org/odata/v1/Products(db0c8ef3-8ec0-5185-a537-812dad3c58f8)/Nodes(S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE)/Nodes(GRANULE)/Nodes"
      }
    }
  ]
}

Let’s suppose that you want to explore the contents of DATASTRIP directory of that product. To do that, access the following link visible in the output above:

https://datahub.eo-lab.org/odata/v1/Products(db0c8ef3-8ec0-5185-a537-812dad3c58f8)/Nodes(S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE)/Nodes(DATASTRIP)/Nodes

This directory contains only one subdirectory:

{
  "result": [
    {
      "Id": "DS_EPAE_20180927T073143_S20180927T051218",
      "Name": "DS_EPAE_20180927T073143_S20180927T051218",
      "ContentLength": 0,
      "ChildrenNumber": 2,
      "Nodes": {
        "uri": "https://datahub.eo-lab.org/odata/v1/Products(db0c8ef3-8ec0-5185-a537-812dad3c58f8)/Nodes(S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE)/Nodes(DATASTRIP)/Nodes(DS_EPAE_20180927T073143_S20180927T051218)/Nodes"
      }
    }
  ]
}

Accessing that subdirectory shows that it contains one file and one folder:

{
  "result": [
    {
      "Id": "MTD_DS.xml",
      "Name": "MTD_DS.xml",
      "ContentLength": 3116641,
      "ChildrenNumber": 0,
      "Nodes": {
        "uri": "https://datahub.eo-lab.org/odata/v1/Products(db0c8ef3-8ec0-5185-a537-812dad3c58f8)/Nodes(S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE)/Nodes(DATASTRIP)/Nodes(DS_EPAE_20180927T073143_S20180927T051218)/Nodes(MTD_DS.xml)/Nodes"
      }
    },
    {
      "Id": "QI_DATA",
      "Name": "QI_DATA",
      "ContentLength": 0,
      "ChildrenNumber": 5,
      "Nodes": {
        "uri": "https://datahub.eo-lab.org/odata/v1/Products(db0c8ef3-8ec0-5185-a537-812dad3c58f8)/Nodes(S2A_MSIL1C_20180927T051221_N0206_R033_T42FXL_20180927T073143.SAFE)/Nodes(DATASTRIP)/Nodes(DS_EPAE_20180927T073143_S20180927T051218)/Nodes(QI_DATA)/Nodes"
      }
    }
  ]
}

OpenSearch

Using OpenSearch interface to query Data Catalogue

Important

Due to the fact that offset is not a recommended form of searching repository pages, we had to implement limit to a maximum of 200k. The requests over the limit will be rejected with the code 400. We encourage you to limit your inquiries by geographic or temporal area.

All queries may be executed as simple HTTP-Get calls, by typing the query in web browser address line, by using any HTTP client, e.g. curl or wget, or from inside of users’ program. The database is accessible free and anonymously (open for anonymous access for everyone, no authorization is used) It may be accessed both from the internal network (virtual machines in Creodias) and from outside, e.g. your home computer. Note, that the actual EO data themselves are restricted to authorized users, only the Data Catalogue is open.

General rules

The queries produce results in JSON format. Base URL:

http://datahub.eo-lab.org/resto/api/collections/search.json?

Most queries are case-sensitive.

Collections

The data are organized in so-called collections, corresponding to various satellites. A query may search for data in all collections, or in one particular collection only. If only one satellite is in the field of interest, the second approach is faster and more efficient, than filtering the general query. For example, to find 10 most recent Sentinel-2 products with cloud cover below 10%, the query should look like:

$ wget -O - "http://datahub.eo-lab.org/resto/api/collections/Sentinel2/search.json?cloudCover=[0,10]&startDate=2022-06-11&completionDate=2022-06-22&maxRecords=10"

while if the collection field is missing in the URL, the products from all the satellites are returned:

$ wget -O - "https://datahub.eo-lab.org/resto/api/collections/search.json?cloudCover=[0,10]&startDate=2022-06-11&completionDate=2022-06-22&maxRecords=10"

As for today the following collections are defined and may be used:

Sentinel1 or SENTINEL-1

Sentinel2 or SENTINEL-2

Sentinel3 or SENTINEL-3

Sentinel5P or SENTINEL-5P

Sentinel6 or SENTINEL-6

Landsat8 or LANDSAT-8

Landsat7 or LANDSAT-7

Landsat5 or LANDSAT-5

Envisat or ENVISAT

Note, that collection names vary a bit from satellite names, as they are used in EO Data repository. For example, the collection is named Sentinel2, while in the repository its data are located within /codede/Sentinel-2/…. branch of the repository tree.

Output sorting and limiting

By default, maximum 20 products are returned only. You may change the limit (beware of long execution time for queries about thousands of products), using the phrase:

maxRecords=nnn

If the query is very general and the number of matching products is large, the next pages of products may be retrieved

page=nnn

You may also change the order of how the products are presented, using the phrase like

sortParam=startDate

will sort the output by observation date. The following orderings are implemented:

startDate

The date when the observation was made (start)

completionDate

The date when the observation was made (end)

published

The date when the product got published in our repository

each of them may be accompanied by

sortOrder=ascending or sortOrder=descending

For example the query

http://datahub.eo-lab.org/resto/api/collections/Sentinel2/search.json?startDate=2021-07-01&completionDate=2021-07-31&sortParam=startDate&maxRecords=20

will return 20 products from July 2021, while the following query would return the next 20:

http://datahub.eo-lab.org/resto/api/collections/Sentinel2/search.json?startDate=2021-07-01&completionDate=2021-07-31&sortParam=startDate&maxRecords=20&page=2

Formal queries

The formal query is invoked as a sequence of sub phrases, separated by &. The result is a conjunction of all sub phrases. It is impossible to use an alternative in the question. The query must be specified as a formal query.

The example of formal query - about cloudless (cloud cover lower or equal to 10%) products for a specific location:

https://datahub.eo-lab.org/resto/api/collections/Sentinel2/search.json?cloudCover=[0,10]&startDate=2021-06-21&completionDate=2021-09-22&lon=21.01&lat=52.22

The queries are in form param=value or param=[minvalue,maxvalue]. Most of the parameters are common for all collections, but some are specific for some them (e.g. cloudCover applies to optical satellites, but polarisation applies to radar ones), or just single one.

Geography and time-frame

The common set of parameters are:

startDate, completionDate

the date limits of the observation. The time may also be specified, e.g. 2021-10-01T21:30:00Z

publishedAfter, publishedBefore

the date limits when the product was published in our repository

lon, lat

geographical position, expressed in military style (EPSG:4326, as decimal fraction of degrees, positive for eastern latitude and northern longitude)

radius

region of interest, defined as a circle with centre in point determined by the longitude and latitude with radius expressed in meters (it won’t work with point manually selected in EOFinder/Data Explorer)

geometry

region of interest, defined as WKT string (POINT, POLYGON, etc.)

box

region of interest, defined as the rectangle with given (west,south,east,north) values

Volatile features

Some terrain-like feature masks are not permanent but describing a single scene only. The most commonly used such feature is cloudiness, or cloudCover, which is defined for most of the products coming from optical sensors. For example:

cloudCover=[0,10]

selects only those scenes, which are covered by clouds by no more than 10%.

Caution: to be meaningful, the cloudiness must be provided with each product, while in many products is missing. If the cloudiness is unknown for the scene, it is marked by a value of 0 or -1. cloudCover=0 is therefore ambiguous: it may either mean totally cloudless sky or the cloudy scene for which cloud cover had not been estimated during original data processing.

Satellite features

instrument

meaningful only for satellites equipped with multiple instruments. The possible values are satellite specific.

productType

the actual types possible are specific for every satellite.

sensorMode

also satellite and sensor specific. E.g. (for Sentinel-1): sensorMode=EW

orbitDirection

ASCENDING or DESCENDING. For most heliosynchronous satellites descending orbits means the day scenes, while ascending means night ones. For many optical satellites (e.g. Sentinel-2) only day scenes are published.

resolution

expected spatial resolution of the product defined in meters.

status:

ONLINE or OFFLINE

Some additional parameters are strictly satellite-specific, e.g. polarisation, which is defined only for Sentinel-1.

For every satellite (collection) its set of query-able parameters may be obtained by a query like:

https://datahub.eo-lab.org/resto/api/collections/Sentinel1/describe.xml

The resulting XML file provides full list of the parameters for the collection, with their very brief descriptions.

DeletedProducts OData API

The DeletedProducts OData endpoint allows users to access information about the deleted products in the Catalogue. This endpoint provides a convenient way to retrieve details about the products that have been deleted from the Catalogue. By utilizing the supported operations and filtering options, users can efficiently access the required deleted products’ details. For the DeletedProducts OData endpoint, requests should be built the same way as for the OData Products endpoint, with the change in the URL Products to DeletedProducts.

Endpoint URL

The Deleted OData endpoint can be accessed using the following URL:

https://datahub.eo-lab.org/odata/v1/DeletedProducts

Query structure

The DeletedProducts OData endpoint supports the same searching options as a standard OData Products endpoint. For more information, please go to section query structure above.

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=not contains(Name,'S1') and DeletionDate gt 2023-04-01T00:00:00.000Z and DeletionDate lt 2023-05-30T23:59:59.999Z&$orderby=DeletionDate&$top=20

PERFORMANCE NOTE

To accelerate the query performance, it is strongly recommended to limit the query by specified dates, e.g.:

DeletionDate gt 2022-05-03T00:00:00.000Z and DeletionDate lt 2023-05-03T00:00:00.000Z

Filter option

To search for products by properties a filter should be built as explained in section about filter options above.

The acceptable products’ properties for OData DeletedProducts endpoint are:

  • Name - search for a specific product by its exact name

  • Id - search for a specific product by its id

  • DeletionDate - search by deletion date

  • DeletionCause - search by deletion cause

  • Collection/Name - search within a specific collection

  • OriginDate - search by origin date

  • ContentDate/Start and ContentDate/End - search by sensing date

  • Footprint - search by geographic criteria

  • Attributes - search by product’s attributes

Query by name

To search for a deleted product by its exact name:

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=Name eq 'S2A_MSIL1C_20210404T112111_N0500_R037_T31VEG_20230209T101305.SAFE'

Query by ID

To search for a deleted product by its Id:

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts(29008eb1-1a51-48a8-9aec-288b00f7debe)

Query by deletion date

To search for products deleted between two inclusive interval dates:

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=DeletionDate ge 2023-04-26T00:00:00.000Z and DeletionDate le 2023-04-27T23:59:59.999Z

Query by deletion cause

To search for products deleted from specific reason:

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=DeletionCause eq 'Duplicated product' or DeletionCause eq 'Corrupted product'

Allowed values of the DeletionCause parameter are:

  • Duplicated product

  • Missing checksum

  • Corrupted product

  • Obsolete product/Other

Query by collection of products

To search for deleted products within a specific collection:

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=Collection/Name eq 'SENTINEL-2' and DeletionDate gt 2023-04-01T00:00:00.000Z and DeletionDate lt 2023-09-30T23:59:59.999Z

For available collections please refer to Link. Also, please note that it is possible that none of the products have been deleted from the available collections.

Query by sensing date

To search for deleted products acquired between two dates:

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=ContentDate/Start gt 2021-09-01T00:00:00.000Z and ContentDate/End lt 2021-09-01T00:05:00.000Z

Query by Geographic Criteria

To search for deleted products intersecting the specified polygon:

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=OData.CSC.Intersects(area=geography'SRID=4326;POLYGON ((-75.000244 -42.4521508418609, -75.000244 -43.4409190460844, -73.643585 -43.432873907284, -73.66513 -42.4443775132447, -75.000244 -42.4521508418609))') and ContentDate/Start gt 2021-01-01T00:00:00.000Z and ContentDate/End lt 2021-04-01T23:59:59.999Z

Query by attributes

To search for products by attributes it is necessary to build a filter with the specified structure as defined Link

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.IntegerAttribute/Value eq 10844) and attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'orbitDirection' and att/OData.CSC.StringAttribute/Value eq 'ASCENDING')

Additional options

Here are the additional options for deleted products query:

Orderby option

Orderby option works the same way as explained in section orderby, above.

For OData DeletedProducts endpoint, acceptable arguments for this option are:

  • ContentDate/Start, ContentDate/End

  • DeletionDate

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=contains(Name,'S1A_EW_GRD') and DeletionDate gt 2023-04-01T00:00:00.000Z and DeletionDate lt 2023-05-30T23:59:59.999Z&$orderby=DeletionDate desc

Expand option

Expand option enables users to see the full metadata of each returned result.

The acceptable arguments for this option:

  • Attributes

To see the metadata of the results:

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=contains(Name,'S2A') and DeletionDate gt 2023-04-01T00:00:00.000Z and DeletionDate lt 2023-05-30T23:59:59.999Z&$expand=Attributes

Skip option

Skip option can be used as defined in section skip option above.

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=contains(Name,'S2A') and ContentDate/Start ge 2021-04-01T00:00:00.000Z and ContentDate/Start le 2021-04-30T23:59:59.999Z&$skip=30

Top option

Top option can be used as defined in section top option above.

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=contains(Name,'S1A_EW_GRD') and ContentDate/Start ge 2021-09-01T00:00:00.000Z and ContentDate/Start le 2021-09-30T23:59:59.999Z&$top=40

Count option

Count option can be used as defined in section count option above.

HTTP Request

https://datahub.eo-lab.org/odata/v1/DeletedProducts?$filter=contains(Name,'S1A_EW_GRD') and DeletionDate gt 2023-04-01T00:00:00.000Z and DeletionDate lt 2023-05-30T23:59:59.999Z&$orderby=DeletionDate desc&$count=True

What To Do Next

To learn how to download a single file from a product or a full product as a ZIP archive, check this article: Using curl and wget to download EODATA products from EO-Lab