If you cannot mock or do not wish to, it is recommended to set up additional Scrivito CMS instances to be used as testing environments. They would be used to create a working copy, add the content, publish and eventually delete everything with each run of the test. Keep in mind, an instance would only be able to deal with a single run at a time. So, if a developer wants to run such a test locally, they would have to have their own CMS instance. Otherwise, if there’s another developer running these tests in parallel against the same CMS instance, they will override each other’s data, resulting in non-predictable errors. Also, if there are CI machines running integration tests with publishes, each of these machines needs its own erasable CMS instance.
Creating the test tenant content is best done either in the test or via seed files. We advise to not copy anything from the production instance and in general completely isolate these two data sources in order to keep the production tenant as far from danger zones as possible. Also, in order to keep the tests as fast as possible, the seed data should to be as tiny as possible.
At Infopark, we use the following test workflow:
- Reset a CMS instance at the very beginning of the test suite.
- Run tests making publishes.
Note, we do not reset the tenant on each run, since it’s a relatively expensive and long-running operation.