How to mount object storage container from EO-Lab as file system on local Windows computer
This article describes how to configure direct access to object storage containers from EO-Lab FRA1-1 cloud in This PC window on your local Windows computer. Such containers will be mounted as network drives, for example:
They all will be automatically mounted and available this way at the same time from all user accounts on your computer. You will be able to access them not only using the Windows File Explorer, but also with other file managers which support standard Windows file operations.
Prerequisites
No. 1 Account
You need a EO-Lab hosting account with access to the Horizon interface https://cloud.fra1-1.cloudferro.com/auth/login/?next=/.
No. 2. Object storage container
You need at least one object storage container on the EO-Lab FRA1-1 cloud. If you do not have one yet, please follow this article: How to use Object Storage on EO-Lab
No. 3. Generated EC2 Credentials
You need to generate EC2 credentials for your account. If you haven’t done that yet, please follow this article to install on Linux: How to generate and manage EC2 credentials on EO-Lab.
To install OpenStack CLI client on Windows have a look at on of these articles:
Then, adjust the instructions from the article How to generate and manage EC2 credentials on EO-Lab accordingly.
No. 4 A local computer running Microsoft Windows
You need a local computer running Microsoft Windows. This article was written for Windows 10 Pro. You need to have access to an account with administrative privileges.
What We Are Going To Cover
Mounting an object storage container as a Windows drive using open source software
Testing the connection to the container
Tweaking the –dir-cache-time option
Setting automatic mounting for a container
Removing software used for mounting
Software tools used in this article: Rclone, WinFSP, and NSSM
Rclone has multiple functions such as managing files in cloud storage and syncing between file systems. In this article, you will use its rclone mount command to mount object storage on your Windows VM.
WinFSP enables accessing custom file systems on Microsoft Windows. In this workflow, it will allow Rclone to mount the S3 storage.
NSSM is a service manager. Here, it will be used for configuring automatic mounting of object storage. You will run it from the command line.
How to use the Rclone configuration file
By default, Rclone creates and uses configuration file in the folder \.config\rclone in home directory of the current user.
In order to streamline the configuration process, you will instead manually create a configuration file called rclone.conf in folder where Rclone binary is stored. There, you will paste the code provided in this article. Each time Rclone is executed, the location of that file will be passed to it in a parameter.
Warning
All users of your computer will be able to access and modify the configuration file. It contains the login credentials written in plain text.
Step 1 Download and install the appropriate software
Download and install Rclone
Skip this step if you have already configured these software tools for accessing object storage.
Start PowerShell.
Use the following commands to
create the directory in which Rclone will be stored and
navigate to it.
mkdir C:\rclone; cd C:\rclone
Without closing PowerShell, open a web browser on your virtual machine. Navigate to the following website: https://rclone.org/downloads. It should contain the table with links to its different versions:
Download the version of Rclone for the Intel/AMD - 64 Bit platform (for Windows operating system).
Download the zip archive. In Microsoft Edge browser it should look like this:
Enter that zip archive, it should contain one directory. Double-click to enter it. Its content should look like this:
Copy the content of that directory to C:\rclone folder which you have previously created using PowerShell.
Download and install WinFsp
Return to Internet browser and navigate to https://winfsp.dev/rel/. Click Download WinFsp Installer:
Run the downloaded installer. The installation process will be similar to installation of other Windows programs. Its step called Custom Setup, however, will require you to make a choice:
Leave the default values intact to install only the Core section and click Next. Complete the installation.
Download and install NSSM
Navigate to https://nssm.cc/download in your browser.
Click the link in the Latest relase section:
A zip file should be downloaded. It should contain one folder. The content of that folder should look like this:
Navigate to win64 folder. It should have one executable file called nssm:
Copy that file to C:\rclone
Step 2: Enter the connection data
Open folder C:\rclone using Windows file manager. File rclone.conf in that folder will be used to store the connection data for object storage. If that file does not exist, create it. Open it using Notepad.
Each section containing the object storage connection data starts with a line containing its name written in square brackets. Below it, there are lines containing different parameters used to connect to that object storage. Sections will have access to all object storage containers with the same pair of access and secret keys. This means that you do not need to create separate sections for different containers using the same pairs of credentials.
Add the following section to the end of this file:
[remote-config]
type = s3
provider = Other
access_key_id = 1234
secret_access_key = 4321
endpoint = s3.fra1-1.cloudferro.com
In the above block, replace 1234 and 4321 with the access and secret key you obtained while following Prerequisite No. 3, respectively.
This code will create a connection called remote-config. If you want to use a different name for your connection than remote-config, replace it in the code above. This name does not have to be the same as the name of one of your containers.
If you want to access other object storage containers using different credentials on your computer, create a separate section with a different name as described above for each key pair.
Step 3: Perform a test mount
In order to test the connection you configured, execute the command below. Replace remote-config with the name of the connection you just configured.
.\rclone.exe --config "C:\rclone\rclone.conf" lsd remote-config:
You should see the list of object storage containers associated with your credentials, for example:
-1 2023-01-18 12:53:14 -1 second-test-container
-1 2023-01-16 13:23:03 -1 test-container
Repeat this process for each key pair you added in Step 2.
To test the mounting of one of your containers, execute the command below without leaving the PowerShell. Replace remote-config with the name of your connection, test-container with the name of your container and E: with the drive letter under which you wish to mount it.
.\rclone.exe mount --config "C:\rclone\rclone.conf" --network-mode remote-config:test-container E: --vfs-cache-mode full --dir-cache-time 1m0s
The option in this command –vfs-cache-mode full should make the mount support standard file system operations.
The option –dir-cache-time 1m0s will be explained in the next step.
You should now get the following output:
The service rclone has been started.
Go to This PC window. You should see the mounted container there:
Enter it and you should see its content there:
To stop the test mount, press CTRL+C in the PowerShell. You should get the following output:
The service rclone has been stopped.
The container should no longer be visible in This PC window.
If pressing CTRL+C does not stop the test mount, make sure that the PowerShell window is focused by left-clicking it. Press a letter on your keyboard, for example A, and try pressing CTRL+C again.
You can perform such tests for all object storage containers you wish to use on your computer.
Do not close PowerShell yet.
Step 4: Tweak the –dir-cache-time option
In Step 3, you performed a test mount of your object storage container using the following command:
.\rclone.exe mount --network-mode remote-config:test-container E: --vfs-cache-mode full --dir-cache-time 1m0s
Tweaking the option –dir-cache-time is important especially if you intend to use your container on multiple physical and/or virtual machines. This includes using the container on your virtual machine and the Horizon dashboard. You might discover that the changes made to the bucket on another computer do not appear on your Windows VM. Using the Refresh option of the Windows File Explorer might not synchronize that change either.
That is because the Refresh option in this case does not pull the changes directly from the container, but from the cache. If the option –dir-cache-time is not specified during mounting, the cache is automatically synchronized every 5 minutes. Therefore, if you for example change a name of the folder on your other device, you will be able to pull that change after up to about 5 minutes.
Specifying this option overwrites this default value of 5 minutes. In this example, the automatic refresh of cache was set to 1 minute (1m0s). It is also possible to set this value to for example 1 second (0m1s). You can replace 1m0s in the command above with the value of your choice.
You can now perform a few test mounts as explained in Step 3 and find the –dir-cache-time value that suits you.
Step 5: Configure automatic mounting of your container
Open PowerShell as Administrator and execute the following command there to navigate to the C:\rclone directory:
cd C:\rclone
Now, start the service installer of NSSM with the following command:
.\nssm.exe install
You should get the following window:
Click the … button next to the Path: text field.
Choose the location of Rclone. If you followed this tutorial, this location is as follows:
C:\rclone\rclone.exe
In the Arguments text field enter the following code. Replace remote-config with the name of your connection, test-container with the name of your container, E: with the drive letter under which you wish to mount it and 1m0s with the value you chose in Step 3.
mount --config "C:\rclone\rclone.conf" --network-mode remote-config:test-container E: --vfs-cache-mode full --dir-cache-time 1m0s
Warning
Make sure that you do not have anything mounted under the drive letter you choose here.
In the text field Service name: enter the name for your mounting service. It can be different than the name of your connection you set in Step 1 and the name of your S3 container. In this example, the name mounting-service will be used.
Navigate to the Log on tab. Make sure that the option Local System account is selected.
Click Install service.
Restart your computer and check whether the drive gets automatically mounted in the This PC window. If it is, the service works as intended.
You should now be able to work with your files.
Add a separate service with a different name as described above for each object storage container you wish to have automatically mounted on your computer. Memorize or write somewhere down the names of those services to be able to easily stop and remove them in the future if needed.
If you find yourself unable to delete files or folders on the object storage, you can remove them from the Object Store -> Containers option in the Horizon dashboard:
Removing software responsible for automatic mounting of eodata
If you no longer want to access the eodata repository on a particular machine, you can remove the appropriate configuration and software. The whole procedure will be covered in this section:
Removal of automatic mounting created in NSSM
Removal of saved object storage in Rclone
Uninstalling WinFsp
Removal of C:\rclone folder containing Rclone and NSSM
Open PowerShell as administrator. Navigate to the C:rclone directory by executing the following command:
cd C:\rclone
To check the status of your automatic mounting service, execute the following command. In it, replace mounting-service with the name of your automatic mounting service you set previously.
.\nssm.exe status mounting-service
You should get the following output:
SERVICE_RUNNING
To stop the automatic mounting of your container, execute the command below. Replace mounting-service as previously.
.\nssm.exe stop mounting-service
Delete the service by executing the command below. Replace mounting-service as previously.
.\nssm.exe remove mounting-service confirm
You should now get the output similar to this:
Service "mounting-service" removed successfully.
Delete the C:rclone folder you created.
Click the Start menu and type control panel.
You should now see the following search result:
Click on it to enter the Control Panel.
Make sure that the drop-down menu View by is set to Category.
In the section Programs select Uninstall a program.
On the list which appeared find the WinFsp entry:
Right-click it and choose Uninstall:
You will now get the following question:
Click Yes. You should get the following window:
Close all your open program and File Explorer windows (for example a This PC or Documents window). Make sure that the option Automatically close applications and attempt to restart them after setup is complete. is selected and click OK.
You will now be prompted to reboot your computer:
Make sure that you saved your worked and closed the programs and click Yes.
Your computer should now get rebooted.
Once you have logged in again, open This PC window.
Enter your C: drive.
Remove the rclone folder in it - you might need administrative privileges for it.
Rclone and other software used for mounting object storage should now be removed.