Monday, April 09, 2007

Rails Fixture Tips

Here is how to impose ordering on fixture loads to handle foreign keys and how to load binary data in YAML fixtures!

Rails has tests baked into it which is nice. While learning Rails, I've been reading the "Agile Web Development with Rails" ebook which is okay, and of course it shows how to use tests and fixtures. But it leaves out some problems you will meet when doing real development:
  • YAML fixture data files will contain data with foreign key references to data in other YAML files, which requires an order on the load sequence of fixtures
  • A model might contain a self-reference, expressed through a foreign key constraint to itself, which requires an order on the load sequence of individual records inside the YAML file
  • The YAML file format is great for loading data that can easily be represented in readable text format, but how are binary data like images loaded?
YAML file load order
You can load all fixtures by doing "rake db:fixtures:load", which will load fixture files in alphabetical order. This is bad when there are FK constraints between the models.

You can solve this by simply doing "rake db:fixtures:load FIXTURES=parent,child", which will ensure fixtures are loaded in given order. That is ok, but requires us to put it on the command line each time.

Taking advantage of rails environments, you can put "ENV['FIXTURES'] ||= 'parent,child'" in the "environment.rb" file, which gives you have a predefined ordering for loading in all environments. You can even override this on the command line.

For a more "advanced" solution (have not tried it myself), you might want to have a look at this blog entry.

YAML file delete order
There is one problem with the above solution. When reloading fixtures into tables which already have data in them, Rails starts by emptying out the table with a delete statement. Rails does this in the order of inserts, which breaks FK references in the delete statements. There are two solutions to this, both represented through a new rake task.

db:fixtures:delete
You can make an explicit task to empty out fixture data from tables. This needs to be done in reverse order of the inserts. Place a ".rake" file in "lib/tasks" with this content:
namespace :db do
namespace :fixtures do
desc "Deletes all fixture tables mentioned in FIXTURES environment in reverse order to avoid constraint problems"
task :delete => :environment do
require 'active_record/fixtures'
ActiveRecord::Base.establish_connection(RAILS_ENV.to_sym)
ENV['FIXTURES'].split(',').reverse.each do |fixture_name|
ActiveRecord::Base.connection.update "DELETE FROM #{fixture_name}"
end
end
end
end
Now, you can simply do "rake db:fixtures:delete" before you do "rake db:fixtures:load".

load_fixtures_without_constraints
The above solution is nice and simple, but you will have to remember to explicitly empty tables before loading. On the Rails wiki I found a solution where FK constraints are disabled before load and enabled after load. Naturally, this is quite database specific and will not work on databases not supporting such (weird, some might say) functionality.

Individual record load order
You can also pose an ordering on the individual records in a YAML file. This is actually quite simple, as you can use the Ordered YAML format. Here is an example:
--- !omap
- parent:
id: 1
name: I am the parent
- child:
id: 2
name: I am the child
parent_id: 1
Normally, Rails would load the "child" YAML entry before "parent" as it comes first in the alphabet. Using the ordered mapping format, we ensure that parent is loaded before child.

Loading binary data into fixtures
When you want to load binary data, YAML can be a bit irritating. For me, it has mostly been loading images into MySQL, so I do not know if it works on other databases. Full credit should go to this blog poster. My example here is based on that a lot.

YAML supports binary data in one of two ways: "Canonical" or "generic". Someone out there seems to have had problems with the "canonical" format and this example is using the "generic" format. Basically, in YAML you can write
entry: !binary |
...here comes base64 encoded data...
And this rails code inside the YAML file can load a binary file, BASE64 encode it and substitute some spaces into it to make it YAML formatted.
<%
def binary_fixture_data(name)
filename = "#{RAILS_ROOT}/test/fixtures/binaries/#{name}"
data = File.open(filename,'rb').read
"!binary | #{[data].pack('m').gsub(/\n/,"\n ")}\n"
end
%>
picture_data_1:
id: 1
blob_data: <%= binary_fixture_data('test1.jpg') %>
There are problems in this, I think. But it works. The problems I see are:
  1. The definition of "binary_fixture_data" is inside the YAML file, which means it must be duplicated in other fixtures that needs binaries (tried putting in in "test_helper.rb" to no luck--any other good place to put such code?)
  2. It can easily break, as it substitutes a number of indent spaces after newlines, which must conform to the level of indention where the binary data are used in the YAML file
But again, it works. And I cannot find a better solution. Again, full credit shall go to Peter Donald for this solution.

2 comments:

Brian said...

You can put the method in fixture_helper.rb and then reference it in the fixture ERB with

FixtureHelper::binary_fixture_data

Per Olesen said...

Cool, thanks.