Test Enterprise-Class Web CMS Scrivito Free for 30 Days
Test Scrivito Free for 30 Days

Using Scrivito with Node.js

Server-side prerendering and content retrieval

BETA

Using Scrivito with Node.js

Usually, Scrivito applications run in a web browser, but you can also access Scrivito content on a server. To this end, the Scrivito SDK supports Node.js, the most popular JavaScript runtime for backends.

In this article, we will illustrate three typical use cases for the Scrivito SDK under Node.js. After outlining the steps needed to render pages on a server, we’ll explain how to generate a sitemap and an RSS feed.

Prerendering with Node.js

Prerendering a Scrivito-based website is a three-step process:

  1. Fetch the pages to be rendered.
  2. Render each page to a string.
  3. Store the prerendered HTML files and the content dumps.

The first two steps can be accomplished with a web build. However, using a browser to store data in the file system requires an unnecessarily complex setup.

With a Node.js build on the other hand, all files can be generated in a straightforward manner by a single prerendering script.

To render a page, the Scrivito SDK must be provided with

  • The Scrivito configuration (tenant, URL mapping, etc.)
  • Page and widget definitions
  • All the React components used by the app or any page or widget
  • Referenced assets must be transpiled to some form of valid JavaScript code

The web build already meets these requirements. Thus, the easiest way to create a Node.js build is to start with the web build configuration. From there, we should try to keep the changes as small as possible. This way, the Node.js target has access to all the required source files, and the resulting asset paths and hashes will match.

The Node.js build configuration

For configuring a web and a Node.js version side-by-side, webpack supports multi-compiler builds. In addition to the web build, we can set up a Node.js-specific build:

webpack.config.production.js
// Before: single target build: // module.exports = webpackConfig; // After: multi target build: function webpackConfigPrerendering(env = {}) { return webpackConfig({ ...env, isPrerendering: true }); } module.exports = [webpackConfig, webpackConfigPrerendering];

The webpack configuration can now contain separate values for the web and prerendering builds.

Next to the mandatory Node.js target setting, we specify separate output paths for each target:

webpack.config.production.js
target: env.isPrerendering ? "node" : ["web", "es5"], output: { path: path.join( __dirname, env.isPrerendering ? "buildPrerendering" : "build" ), // ... },

These are the main differences the resulting web and Node.js configurations will or may have:

  • Target (mandatory)
  • Separate build directories (highly recommended)
  • Separate caching directories or keys (recommended)
  • No development server for the Node.js build (recommended)
  • Separate transpiler options, e.g. no browser polyfills with Node.js
  • Optimization, like code splitting and minification, as well as performance hints, can be disabled for the Node.js build
  • Source map support for Node.js requires a dedicated tool like source-map-support

Using node as the target for Babel preset-env is not necessary. Sticking with the browser options may actually save us from the hassle with finding the right options, for example to avoid “TypeError: Class constructor cannot be invoked without 'new'”.

After these changes to the build configuration, executing npm start will indicate whether we have to take care of other issues after switching to the multi-compiler setup.

The prerender script

If the setup runs smoothly, we can configure and create the entry point for the prerendering routine:

webpack.config.production.js
entry: env.isPrerendering ? { prerender_content: "./prerender_content.js" } : { … }, // regular web entry points

The core of our prerender script looks like this:

src/prerender_content.js
import "./Objs"; import "./Widgets"; import { configure } from "./config"; import App from "./App"; configure({ priority: "background", origin: process.env.SCRIVITO_ORIGIN }); async function prerenderContent() { const objs = await Scrivito.load(() => Scrivito.Obj.onSite("default") .where("_objClass", "equals", ["Homepage", "Page"]) .toArray() ); const prerenderResults = await Promise.all( objs.map(async (obj) => { const { result, preloadDump } = await Scrivito.renderPage(obj, () => { const bodyContent = ReactDOMServer.renderToString(<App />); return { objUrl: Scrivito.urlFor(obj), bodyContent, }; }); }) ); // … now generate and store an HTML file for each prerenderResults item // … and copy assets from the web build to the prerendering target directory } prerenderContent();

Why use Scrivito.load?

Due to the lazy-loading nature of the SDK, we need to wrap the query inside a Scrivito.load call to prevent “Content not yet loaded” errors.

Why use Obj.onSite?

Scrivito supports multiple websites. Since there is no such thing as a “current website” in a non-browser environment, we have to be specific and use either Obj.onSite or Obj.onAllSites.

The script renders the app once for each page. Let’s compile the script and run it with Node.js:

export SCRIVITO_ORIGIN=http://example.com npm run build node buildPrerendering/prerender_content.js

The origin configuration is mandatory if content URLs are to be computed. Unlike in a browser environment, the SDK cannot default to the current window location.

Making the app Node.js compatible

At this point, there may be errors like “ReferenceError: window is not defined”.

These are indicators that the app contains calls to browser-specific APIs or packages. The code needs to be adjusted accordingly to be compatible with a Node.js environment.

In simple cases, a code path can be made conditional by checking for the presence of the browser-specific API, e.g. by means of typeof window === 'undefined'.

If the logic in question is required for correct rendering, it must be converted to a Node.js compatible alternative. We may have to add a Node.js-only package that has no browser fallback. In such cases we should make sure to include it conditionally, for example by using webpack module aliases or browser specific replacements:

package.json
"dependencies":{ "some-node-only-package":"~1.0.0" }, "browser":{ "some-node-only-package":false },

Otherwise, the web bundle may become bigger for no reason.

In cases where the code cannot be provided in a Node.js compatible way, we can postpone running the code in an Effect Hook, i.e. we wholly exclude it from the prerendered content.

Please keep in mind that the DOM and the state of a component rendered on a server should be identical to the output of the initial client-side render call. Otherwise, there is a risk of running into subtle inconsistencies after hydration.

Generating a sitemap

With the setup for prerendering with Node.js in place, generating additional files for deployment on a server is straightforward.

First, we are going to collect some data for our sitemap. Here’s how to generate the list of the URLs and the last-changed dates of all “Page” objects.

objs = await Scrivito.load(() => Scrivito.Obj.onSite("default").where("_objClass", "equals", "Page").toArray() ); await Scrivito.load(() => objs.map((o) => [Scrivito.urlFor(o), o.lastChanged()]) );

As you can see, the resulting data is sufficient for building an XML sitemap. Please have a look at the Scrivito Example App for details on how to generate the sitemap.xml file.

Generating an RSS feed

The following example illustrates how to create an Atom feed for all the blog posts in the content.

Since a blog post may have an image attached, we need to provide the class definitions for blog posts and images. If we reuse the initialization of the prerender script, the line

import "./Objs";

will include the required definitions, which could look like:

Scrivito.provideObjClass("BlogPost", { attributes: { title: "string", publishedAt: "date", titleImage: "reference", }, }); Scrivito.provideObjClass("Image", { attributes: { blob: "binary", }, });

We can now fetch the content needed to build our blog feed:

function feedData() { const blogPosts = Scrivito.Obj.onAllSites().where( "_objClass", "equals", "BlogPost" ); return blogPosts.toArray().map((o) => ({ title: o.get("title"), publishedAt: o.get("publishedAt"), imageUrl: o.get("titleImage") && Scrivito.urlFor(o.get("titleImage")), url: Scrivito.urlFor(o), })); } await Scrivito.load(feedData);

Based on this data, we can create the Atom feed XML file.

Just try it out and feel free to experiment with it. Also, check out the SDK Cheat Sheet for API usage examples, or the SDK API documentation for details!