Indexing your cloud storage using Rclone and NeoFinder

NeoFinder for macOS (there’s also a Windows version called abeMeda) is hands down one of the most important and useful tools in my daily workflow. It indexes the contents of all my external storage devices, and allows me to search across all of those indexed drives using complex search criteria, without ever having to mount those drives on my Mac. 

And it’s insanely fast.

My entire digital life is indexed in NeoFinder, so it makes sense that my cloud storage should be indexed in my NeoFinder database as well. But unfortunately, while it’s great for indexing locally attached storage devices, NeoFinder doesn’t have the native ability to index a cloud storage volume, unless you use utilities like CloudMounter, ExpanDrive or Mountain Duck to mount one of those volumes as a virtual local filesystem.

However, there are a few downsides in using these utilities for indexing purposes. Of course, the first downside is that most of them are fairly pricey ($40 for Mountain Duck and $50 for ExpanDrive) or are subscription based ($30/year for CloudMounter). 

The other downside is that some of these tools are extremely slow when indexing a mounted cloud filesystem. While Mountain Duck appears to support fast, recursive enumeration of remote files & directories for S3 buckets, other tools like CloudMounter do not.

So I turned to another one of my favorite cloud storage management tools—the totally lifesaving, free and open source tool Rclone—to provide the glue to bridge my S3, Google Drive and Dropbox accounts with NeoFinder.

After messing around with different ideas, I discovered that Rclone has an option to recursively list the contents of a remote cloud storage bucket, and then output that list to a CSV file. NeoFinder can’t import this CSV natively, but it can import CSV files created by other cataloguing apps. Because I already had my own CSV files exported from the YoYotta LTO backup tool, I used them as a reference for writing a Python script that converts the rclone CSV files into the same formatting that NeoFinder expects when importing a YoYotta CSV file.

The resulting utility is called rclone2neofinder, which you can download from my GitHub repository!