OOTB Reaction Commerce v3 provides two storage adapters:
By default GridFS is used and media uploaded via the API using Reaction Admin are stored in MongoDB in several tables starting with
cfs_gridfs. The DB-based file storage works well for small amounts of media and all the basics are there currently for managing product images.
For each product image uploaded via local disk the API kicks off a
saveImage/local job via the
createSaveImageJob utility function in the
api-plugin-files package. The job is added to the
Jobs collection in MongoDB and the
status is updated as soon as the job is processed.
image/local jobs are processed file workers are kicked off and 5 copies of various image quality are created using sharp. The same transformations occur for
image/remote jobs as well. This creates 5 images for every 1 uploaded.
Using MongoDB presents scalaing issues:
There are quite a few drawbacks to using blobs to store media. First, database storage is typically expensive. When your catalog grows, you don’t want to have 50GB of media in a MongoDB instance. It doesn’t make sense. Second, with a lot of traffic, it’ll clog up your app <-> database link because it’ll use so much bandwidth. Not a good thing either, and can cause some pretty serious issues in production during traffic spikes. These are both large-scale examples, but they’re definitely real.
—Loan Laux, Oct 29, 2019
Another scaling issue occurs in development when restoring database backups which slows down development and reduces a businesses ability to fail fast—especially under limited bandwidth.
To overcome this shortcoming of Reaction an Object Storage adapter was created by Loan:
The adapter was made possible as a result of the thoughtful abstractions in reaction-file-collections. As the S3 adapter was created some forks of
api-plugin-files have shown up using it. Based on the docs for the S3 adapter it should be fairly straight-forward to implement. However, it seems there are some issues which arise due to assumptions made related to use of GridFS including:
- Files from S3 may need to be streamed back through Reaction API instead of downloaded by users directly using a URL to, for example, CloudFlare, an S3 endpoint or a Min.io server.
READMEfor reaction-file-collections suggests multiple Stores can be be created to keep some images in the DB and offload others to S3 but provides no working implementation.
These limitations of scale become apparent fairly quickly in new Reaction Commerce apps. And it’s not clear what the best solutions is yet to handle the problem. Let’s discuss.