S3 Object Storage

• How to access private object storage using S3cmd or boto3 on EO-Lab
• How to Delete Large S3 Bucket on EO-Lab
• How to Mount Object Storage Container as a File System in Linux Using s3fs on EO-Lab
• S3FS Cache on EO-Lab
• How to use Object Storage on EO-Lab
• How to mount object storage container from EO-Lab as file system on local Windows computer
  • Prerequisites
  • What We Are Going To Cover
  • Software tools used in this article: Rclone, WinFSP, and NSSM
  • How to use the Rclone configuration file
  • Step 1 Download and install the appropriate software
  • Step 2: Enter the connection data
  • Step 3: Perform a test mount
  • Step 4: Tweak the –dir-cache-time option
  • Step 5: Configure automatic mounting of your container
  • Removing software responsible for automatic mounting of eodata
• Bucket sharing using s3 bucket policy on EO-Lab
  • S3 bucket policy
  • Naming conventions used in this document
  • Limitations
  • S3cmd CONFIGURATION
  • Declaring bucket policy
  • SETTING A POLICY ON THE BUCKET
  • Sample scenarios
• How to install s3cmd on Linux on EO-Lab
  • What We Are Going To Cover
  • Prerequisites
  • Installing s3cmd using apt
  • Uninstalling s3cmd using apt
  • What To Do Next
• Configuration files for s3cmd command on EO-Lab
  • Prerequisites
  • Initializing the configuration process
  • Using --configure on an existing file
  • Executing S3 commands
  • Creating a minimal configuration file manually
  • Maintaining separate s3cmd configuration files
  • What To Do Next
• How to access object storage from EO-Lab using boto3
  • Prerequisites
  • Terminology: container and bucket
  • Preparing the environment
  • How to use the examples provided?
  • Running Python code
  • Creating a container
  • Listing buckets
  • Checking when a bucket was created
  • Listing files in a bucket
  • Listing files from particular path in a bucket
  • Uploading file to a bucket
  • Downloading file from a bucket
  • Removing file from a bucket
  • Removing a bucket
  • General troubleshooting
  • What To Do Next
• How to access object storage from EO-Lab using s3cmd
  • What We Are Going To Cover
  • Prerequisites
  • Object storage vs. standard file system
  • Terminology: container and bucket
  • Configuring s3cmd
  • S3 paths in s3cmd
  • Listing containers
  • Creating a container
  • Uploading a file to a container
  • Listing files and folders of the root directory of a container
  • Listing files and folders not in the root directory of a container
  • Removing a file from a container
  • Downloading a file from a container
  • Checking how much storage is being used on a container
  • Removing the entire container
  • What To Do Next
• How to Install Boto3 in Windows on EO-Lab
  • Step 1 Ensure That Python3 is Preinstalled
  • Step 2 Install boto3 on Windows
• How to access object storage from EO-Lab using boto3
  • Prerequisites
  • Terminology: container and bucket
  • Preparing the environment
  • How to use the examples provided?
  • Running Python code
  • Creating a container
  • Listing buckets
  • Checking when a bucket was created
  • Listing files in a bucket
  • Listing files from particular path in a bucket
  • Uploading file to a bucket
  • Downloading file from a bucket
  • Removing file from a bucket
  • Removing a bucket
  • General troubleshooting
  • What To Do Next
• S3 bucket object versioning on EO-Lab
  • Prerequisites
  • What We Are Going To Cover
  • Configuring and testing AWS CLI
  • Assigning bucket names to shell variables
  • Creating a bucket without versioning
  • Enabling versioning on a bucket
  • Uploading file
  • S3 paths
  • Uploading another version of a file
  • Listing available versions of a file
  • Downloading a chosen version of the file
  • Deleting objects on version-enabled buckets
  • Using lifecycle policy to configure automatic deletion of previous versions of files
  • Suspending versioning
  • What To Do Next