A while ago, I wrote about integrating Carrierwave into my blog so that I could upload images that could then be inserted into my blog articles. I ran tests, I created a simple to use admin back end so I could look at the images and get the proper urls for them, and then uploaded them to the live blog in the hopes of using it to make my articles slightly less monotonous where images would be useful.
This entire set up kept the images stored locally on the same server where my blog application lives. For a small application using minimal photos this is a great idea, in theory, but, as it turns out for me, the platform I’m using to host my application, Heroku, is set up in a way that makes local storage of uploads untenable. I don’t fully understand the why of it all, but suffice to say, due to the way Heroku functions (and it does make deploying a Rails application very simple in many ways) as a transient read only system, that, to the best of my understanding, means that every time I change the application on Heorku from code I wrote on my home computer, all uploaded images are erased from my Heroku storage. I didn’t know about this functionality when I was working on my CarrierWave set up and was pretty bummed to find out that while it would work on my home computer, the functionality I had built wouldn’t work on the live blog itself with out using some sort of cloud based remote image hosting.
No matter what level of user you are reading this blog, you have some understanding of what the cloud is. Perhaps you’ve heard of AWS, or you have a Google Drive, or you use Dropbox. The solution to how Heroku treats my uploaded images locally is solved by using a cloud service to host the images and making calls to the same service when using the images on the blog. There are of course Ruby gems, like fog, that simplify the connection to these services, but few of them are free, and I didn’t want to spend more money on the images if I didn’t have to, so most of the services would be out for that reason but Dropbox seemed like it might be a solution with a little more investigation.
Setting Up Dropbox Account to Use for Remote Image Storage
You’ve probably heard of Dropbox. At it’s core, it’s a nice simple remote file storage / backup system with an easy to use user interface design. One nice thing about it is that it has a free to use level that allows anyone up to 2GB of file storage before having to pay any fees. I have a personal Dropbox account that I’ve used for many years. That account is under my personal email. However, I do have a singular email address linked to the domain name of this blog and thus I could easily set up a clean Dropbox to use for this process, which was an easy first step, but not the only step.
To be able to link the blog functionality to my drop box, I would have to create a way to communicate with Dropbox from an external source, and be able to identify myself as me when contacting Dropbox so it would know which user information to allow me access to. To do this, I would have to create a new application via the Dropbox API. Using the Dropbox API option, I created an application named
mageeworld that would have access to the entire Dropbox. For now I’m only using it for images, but it might have other uses in the future so better to set it up for flexibility. Following these steps, I record the app_key, app_secret and generated access_token that were created.
The next step of course was integrating this application and access permissions with carrierwave.
Setting Up the Carrierwave Dropbox connection
After doing a bit of research I found that there was a gem whose purpose was to connect CarrierWave to Dropbox, and it was named, creative carrierwave-dropbox. This gem is a bit older and less maintained than most gems I use, and there also seemed to be some issues regarding its functionality with the current version (2.0) of the Dropbox API. However, perhaps it would still work if I only set it up properly, but that meant a little experimenting, and probably a small bit of frustration as the instructions written are a little unclear regarding what to do. Fortunately, for me, I had a simple little place to do all this experimenting without messing with my primary application.
As I’ve mentioned before, I often create tinyapps to test out new functionalities. A default rails app with a basic scaffold model generated can serve a variety of test purposes. Though I’d never write any application I intended to publish this way, scaffoliding out your basic minimal needs to test a functionality while not worrying about any tests themselves can also be quite useful. And so it was when I was testing uploading gems. Before deciding to implement Carrierwave I did create two tinyapps, one for testing implementation and usage of CarrierWave and one for Paperclip. Though I had made my decision on Carrierwave, both tinyappls were saved, and as such, I could work with the already extant Carrierwave tinyapp and test this gem for Dropbox just by making some changes.
It took me a few tries but after finally figuring out the setup instructions, I was able to successfully connect the test application to the Dropbox account and upload a photo. Additionally (thankfully) the post processing work I had tested on this tinyapp to create both a thumbnail and display resizing of an original photo were also maintained.
So, one step in the process of being able to upload into Dropbox from my blog was accomplished. I knew how to set up the gem functionality, but it’s not the only step required before it could be made live. A couple other issues were still going to have to be dealt with.
The information that Dropbox provides to me to link my blog to the Dropbox folder isn’t exactly information you want shared around. Anytime an application sets up to connect to some sort of service, (like when you sign up for a new site and it allows you to use Facebook to sign up), the service has provided unique information to the application so that it can be identified every time it connects. This isn’t information you want to share, for a variety of reasons obviously. As such, you need to have your application be able to send this information when needed but you don’t want to hard code it into the code of your application because, especially when it’s an open source project, that means it’s visible to almost anyone. Obviously, this is something that you have to prevent, and there is a way you prevent it using Environment Variables
Environment Variables are not set in an application but within the machine/platform instance running the application. If you know how you can access the environment variables set to your log in on your compute and change them for the moment, for the session, or permanently. The Go Rails web site has a pretty good introduction to the concept of environment variables. One other thing that is great about environment variables is that you can set your own.
This ability to set your own environment variables helps maintain that security concept I mentioned above. Within your application server, you can set environment variables and your application can access them directly. If you look at the instructions for configuring the carrierwave-dropbox gem, you’ll see reference to a variety of things that start off ENV followed by a phrase in brackets (). This is how a rails application accesses environment variables.
Now, environment variables are another one of those things that I’ve been wary of getting too deep into up until this point because, if you mess them up, your app can be severely broken and even compromised to any various intrusions. Rails does have a few built in ways of helping you deal with environment variables, but it is in a bigger ecosystem of deployment that I haven’t fully gotten to yet, however, I do have Heroku which handles a variety of things for you, including setting up this variables.
Heroku has a pretty thorough developer center that provides instructions on doing a variety of things including setting up environment variables for your application. Now, while this isn’t how it’s done on most hosting sites, this is one of the benefits of Heorku. I can easily set my environment variables securely. So, thus, another (and more worrisome actually) step in setting up this functionality was made easier by Heorku.
Putting the Functionality in the Live Application
Though there is one more issue that might arise, I felt ready now to move the functionality into my live application. Following the steps from the tinyapp and the Heroku instructions for environment variables, I set up the connection between Carrierwave and Dropbox in the real code for this blog, and then I ran my tests. Unsurprisingly, all my tests related to image processing failed.
The new functionality I had just set up made no distinction between the environment being used. Thus the test environment was following the same steps production would be following. As such, it was looking for all those environment variables set up locally on my computer. It is possible to set them up easily on my home machine, and doing that is a partial solution, but it also slows down the testing. Every test involving image processing has to connect to the drop box, maybe even more than once, so that the tests can be completed. I know it takes a long time because I did it the long way just to see if it would work. My 81 tests took over 90 seconds to run (that’s a long time) because of the unnecessary connections to Dropbox itself to test functionality that was independent of the Dropbox application. (All the tests did pass if that means anything). That’s not really a great way to run your tests but fortunately, I have a way around that built-in to Rails and its use of environments.
As this is an issue of the testing environment, it’s a simple task to tell the application to do things on way when testing, and another way when not.
Within the uploader file, I write
if Rails.env.test? storage :file else storage :dropbox
Within the initializer, I encompass all the config commands within the simple:
if !Rails.env.test? Carrierwave.config do |config| #config information follows end end
Reminder, the ! before the conditional gives the opposite answer, so in the first example if it’s test, we use the file storage method which is good for testing purposes, and in the second example, the configuring is ignored in the testing environment. I left it on in the production environment for now, which may cause issues in the future, but we’ll see.
So the final step would, hopefully, be relatively simple. Just set up the variables on Heroku as instructed and then push the changes, and away we would go.
And that seems to have worked as you will see below the first, rather large, image posted in my blog of bucky the badger made of snow. It’s a pretty awesome image to leave you with I think.