Importing Data into the CMS

This article illustrates how data can be imported into the CMS. Maybe you exported the content from your previous CMS and would like to import it into Scrivito. Or maybe you have a different data source. One approach is to use a migration for changing CMS content programmatically.

Product data format

Let's assume that we exported product data from a source system in order to import it into Scrivito. We exported the data to a file named productlist.json in the root directory of our Rails application. The importer will load this JSON file, iterate over the entries, and access its various keys in order to populate a CMS page with objects based on this data. This is what the file might look like:

Copy
[
  {
    "title": "Teddy the Bear",    
    "description": "<p>Teddy the Bear</p> ...",
    "image": "product_images/teddy.jpg"
  },  
  ...
]

The title key is a string, the description contains HTML markup. The value of an image key is a path to an image file in a folder named product_images located in the root directory of the Rails application.

Creating the product page and widget types

In this example, the products stored in the JSON file are imported into a single CMS “ProductListPage” object. Every instance of the “ProductListPage” object class has a widgetlist attribute named “products.” We want to import every product into a dedicated widget and add these widgets to the “products” widget of the page.

First, we'll create the page type:

rails g scrivito:page product_list_page

Open the resulting model file and add the “products” attribute to it:

Copy
# app/models/product_list_page.rb

class ProductListPage < Obj
  attribute :products, :widgetlist
end

Next, we generate the new widget type for the individual products and name it ProductWidget (the generator adds the “Widget” part if it's missing):

rails g scrivito:widget product

We'll provide this widget type with a widgetlist attribute named “content.” The product properties will be added to this widget list.

Copy
# app/models/product_widget.rb

class ProductWidget < Widget
  attribute :content, :widgetlist
end

Writing the import migration

Now, let's generate the migration intended for populating our page with products:

rails g scrivito:migration import_products

Open this file in your editor and replace the contents of up with the migration code below.

Copy
class ImportProductsMigration < Scrivito::Migration
  def up
    # Load the products
    productlist = MultiJson.load(File.read(Rails.root + 'productlist.json'))

    # Create the product list page, adding a permalink to it
    # and filling the "products" widgetlist with "ProductWidget"s
    ProductListPage.create(_permalink: 'products', products: productlist.map { |product|
      ProductWidget.new(content: [
        HeadlineWidget.new(headline: product['title']),
        TextWidget.new(text: product['description']),
        ImageWidget.new(image: Image.create(blob: File.new(Rails.root + product['image'])))
      ])
    })
  end
end

As mentioned, the code iterates over the product list, creating a “ProductWidget” for every product and adding this widget to the “products” widget list of the page. The properties of a product are imported into widgets of the “HeadlineWidget,” “TextWidget,” and “ImageWidget” types we assume to be present in the application. 

Running the migration

Now, run the migration to get the job done.

bundle exec rake scrivito:migrate

== 20130718065315 ImportProductsMigration: migrating =======================================
rake aborted!
Errno::ENOENT: No such file or directory - /Users/Bob/Code/toy_store/product_images/racing_car.jpg

Migrations only affect a dedicated working copy. If you made a mistake, don't worry, just abort the migration. The migration working copy is deleted automatically, similar to a transaction rollback. You can now fix the error by providing the missing image and rerun the migration:

bundle exec rake scrivito:migrate:abort

Migration workspace deleted.

bundle exec rake scrivito:migrate

== 20130718065315 ImportProductsMigration: migrating =======================================
== 20130718065315 ImportProductsMigration: migrated (8.0968s) ==============================

Once you are satisfied with the result, publish the migration working copy. Only then, the imported data is visible to website visitors.

bundle exec rake scrivito:migrate:publish