S3FS Cache on EO-Lab

By default, S3FS Cache is enabled in preconfigured Linux images on EO-Lab. To examine its directory location you can take a look at the configuration in /etc/fstab.

#s3fs#DIAS /eodata fuse passwd_file=/root/.passwd-s3fs,_netdev,allow_other,use_path_request_style,uid=0,umask=0222,mp_umask=0222,mp_umask=0222,gid=0,stat_cache_expire=20,url=https://eodata.cloudferro.com,use_cache=1,max_stat_cache_size=60000,list_object_max_keys=10000 0 0

The use_cache parameter points to the folder “1” (it is located at the beginning of the filesystem hierarchy /).

Following is an excerpt from the documentation:

-o use_cache (default="" which means disabled) local folder to use for local file cache.

Directory “1” is the destination for products from the EOData repositories during processing or downloading.

If you encounter problems with displaying the folders and/or files inside “/eodata” (e.g. “no such file or directory”) please empty the “/1” folder.

To avoid situations when you cannot remount your resources because the cache directory is occupying a lot of space on the disk, you might want to set a value for the parameter ‘ensure_diskfree’.

A common prompt message when you cannot remount the /eodata repository is this:

clnt_create: RPC: Timed out
s3fs: There is no enough disk space for used as cache(or temporary) directory by s3fs.

In the s3 documentation you can also find more information about the ‘ensure_diskfree’ parameter.

The command

-o ensure_diskfree(default 0)

sets the parameter to ensure free disk space. This determines the threshold of free space on the disk used for the cache file by s3fs. s3fs creates separate files for downloading, uploading and caching items. If the disks free space is smaller than this value, s3fs will not use disk space if possible in exchange for lower performance.