Upgrading to Magnolia 5.7.x

Before starting the upgrade, we recommend you read:

• Release notes for the version you are updating to as well as for all intermediate versions
• How to update 
• Certified stack

If you are migrating from Magnolia 4.4/4.5, read the migration documentation first. You can also contact us for migration support. 

What to update

Add privacy modules

If you have an Enterprise license, add the new privacy modules that help you make your websites GDPR-compliant:.

<dependency>
  <groupId>info.magnolia.privacy</groupId>
  <artifactId>magnolia-privacy-cookie-manager</artifactId>
</dependency>

<dependency>
  <groupId>info.magnolia.privacy</groupId>
  <artifactId>magnolia-privacy-visitor-manager</artifactId>
</dependency>

<dependency>
  <groupId>info.magnolia.privacy</groupId>
  <artifactId>magnolia-privacy-ui</artifactId>
</dependency>

<dependency>
  <groupId>info.magnolia.privacy</groupId>
  <artifactId>magnolia-privacy-sample</artifactId>
</dependency>

Changes on add-on modules 

Add-on modules are official Magnolia modules but which are not bundled. We release them with the artifact magnolia-enterprise-addons-bundle.

Updated add on modules

With Magnolia 5.7 we also have updated the Backup module to version 2.3. If you want to use the backup module, you must upgrade it to 2.3.

Outdated add-on modules

In Magnolia 5.7 we removed several add-on modules (not bundled). We recommend you remove them too. 

This is the list of the add-on modules removed:

• magnolia-jndi-1.0.2
• magnolia-module-newsletter-bundle-2.2
• magnolia-module-webdav-2.1.3
• magnolia-templating-samples-5.4.1

See also  MGNLEE-533 - Getting issue details... STATUS .

WebDAV

You must remove the magnolia-module-webdav module.

Jackrabbit 2.16 removed the HttpClient3 based WebDAV API and introduced a completely new WebDAV API.

You can do many common WebDAV tasks with light development. You can store web resources such as CSS and JS files, template scripts and many more items in a light module and edit them comfortably as local files with your favorite editor. When you are done with the changes, push and commit the files to Git and configure Magnolia to watch for changes in the remote Git directory. Magnolia will register changes in light modules instantly.

Check custom fields extending or decorating Magnolia fields

We have changed some names for field definitions. The name of a field definition is the node name if defined via JCR in the folder modules/some-module/fieldTypes or the file name of a YAML based definition. Field definitions which have been defined via JCR and which we have renamed now are defined with YAML files. This was part of the initiative to reference fields by its "short" name with the property fieldType (see Referencing fields). 

If you have custom fields which reuse an existing Magnolia configuration via YAML include, YAML inherit, JCR extends or by decoration, your custom definition may be broken now if you reference to a field which has changed its name.

Use the Definitions app to check for problems.

Check the magnolia.properties file

 Between Magnolia 5.6.6 and Magnolia 5.7, the magnolia.properties file has not changed.

Below are the latest versions of the magnolia.properties files for the Magnolia 5.7.x series: 5.7.11

Magnolia CE / Magnolia EE Standard *

#--------------------------------------------
# Here we define some properties not
# configured in the config repository.
# They are used in common before the initialization
# of the repositories.
#
# WARNING: on Windows systems, either use the /
# to separate path elements, or escape the \ with
# another \, i.e C:\\magnolia\\data\\repositories
# or c:/magnolia/data/repositories
#--------------------------------------------
magnolia.home=${magnolia.app.rootdir}
# The directory to expose file system resources from
magnolia.resources.dir=${magnolia.home}/modules
# Pattern to define which resources should be observed by ClasspathScanner
magnolia.resources.classpath.observation.pattern=.*\\.(ftl|yaml)$
# Directories relative to magnolia.resources.dir which will be excluded from FileResourceOrigin observation.
# Also see info.magnolia.resourceloader.file.FileSystemResourceOrigin#EXCLUDED_DIRECTORIES
#magnolia.resources.filesystem.observation.excludedDirectories=META-INF,WEB-INF,cache,docroot,logs,repositories,tmp
magnolia.cache.startdir=${magnolia.home}/cache
magnolia.upload.tmpdir=${magnolia.home}/tmp
magnolia.exchange.history=${magnolia.home}/history
magnolia.repositories.config=WEB-INF/config/default/repositories.xml
magnolia.repositories.home=${magnolia.home}/repositories
magnolia.repositories.jackrabbit.config=WEB-INF/config/repo-conf/jackrabbit-bundle-h2-search.xml

log4j.config=WEB-INF/config/default/log4j2.xml
magnolia.logs.dir=${magnolia.home}/logs

# The directories in which the bootstrap files are searched
magnolia.bootstrap.dir=WEB-INF/bootstrap/author WEB-INF/bootstrap/common

# This is only used for the initial installation afterward the configuration in the config repository is used
# The value is saved in /server/admin
magnolia.bootstrap.authorInstance=true

# Some modules contain optional sample content. They will check this property to decide if they should install
# the sample data
magnolia.bootstrap.samples=true

# Activate UTF-8 support to pages
magnolia.utf8.enabled=false

# Switch to false to enhance the performance of the javascript generation and similar
magnolia.develop=false

# Change to point at your custom Vaadin widgetset and theme
# Your widgetset should always inherit magnolia's default widgetset (info.magnolia.widgetset.MagnoliaWidgetSet)
# Your theme should always include magnolia's default theme (admincentral)
magnolia.ui.vaadin.widgetset=info.magnolia.widgetset.MagnoliaWidgetSet
magnolia.ui.vaadin.theme=admincentral

# Contact details displayed in the footer of the login form
#magnolia.service.contact=

#--------------------------------------------
# Repository connection
#--------------------------------------------
magnolia.connection.jcr.userId = admin
magnolia.connection.jcr.password = admin

# Set it to true if bootstrapping/update should be performed automatically
magnolia.update.auto=false

# Location of the file containing both the private and the public keys used to verify authenticity of activation requests
# This file is generated if not present
magnolia.author.key.location=${magnolia.home}/WEB-INF/config/default/magnolia-activation-keypair.properties

Magnolia EE Pro

#--------------------------------------------
# Enterprise Pro webapp properties, currently only difference is Pro Widgetset.
#
# Here we define some properties not
# configured in the config repository.
# They are used in common before the initialization
# of the repositories.
#
# WARNING: on Windows systems, either use the /
# to separate path elements, or escape the \ with
# another \, i.e C:\\magnolia\\data\\repositories
# or c:/magnolia/data/repositories
#--------------------------------------------
magnolia.home=${magnolia.app.rootdir}
# The directory to expose file system resources from
magnolia.resources.dir=${magnolia.home}/modules
# Pattern to define which resources should be observed by ClasspathScanner
magnolia.resources.classpath.observation.pattern=.*\\.(ftl|yaml)$
# Directories relative to magnolia.resources.dir which will be excluded from FileResourceOrigin observation.
# Also see info.magnolia.resourceloader.file.FileSystemResourceOrigin#EXCLUDED_DIRECTORIES
#magnolia.resources.filesystem.observation.excludedDirectories=META-INF,WEB-INF,cache,docroot,logs,repositories,tmp
magnolia.cache.startdir=${magnolia.home}/cache
magnolia.upload.tmpdir=${magnolia.home}/tmp
magnolia.exchange.history=${magnolia.home}/history
magnolia.repositories.config=WEB-INF/config/default/repositories.xml
magnolia.repositories.home=${magnolia.home}/repositories
magnolia.repositories.jackrabbit.config=WEB-INF/config/repo-conf/jackrabbit-bundle-h2-search.xml

log4j.config=WEB-INF/config/default/log4j2.xml
magnolia.logs.dir=${magnolia.home}/logs

# The directories in which the bootstrap files are searched
magnolia.bootstrap.dir=WEB-INF/bootstrap/author WEB-INF/bootstrap/common

# This is only used for the initial installation afterward the configuration in the config repository is used
# The value is saved in /server/admin
magnolia.bootstrap.authorInstance=true

# Some modules contain optional sample content. They will check this property to decide if they should install
# the sample data
magnolia.bootstrap.samples=true

# Activate UTF-8 support to pages
magnolia.utf8.enabled=false

# Switch to false to enhance the performance of the javascript generation and similar
magnolia.develop=false

# Change to point at your custom Vaadin widgetset and theme
# Your widgetset should always inherit magnolia's Pro widgetset (info.magnolia.widgetset.MagnoliaProWidgetSet)
magnolia.ui.vaadin.widgetset=info.magnolia.widgetset.MagnoliaProWidgetSet
# Your theme should always include magnolia's default theme (admincentral)
magnolia.ui.vaadin.theme=admincentral

# Contact details displayed in the footer of the login form
#magnolia.service.contact=

#--------------------------------------------
# Repository connection
#--------------------------------------------
magnolia.connection.jcr.userId = admin
magnolia.connection.jcr.password = admin

# Set it to true if bootstrapping/update should be performed automatically
magnolia.update.auto=false

# Location of the file containing both the private and the public keys used to verify authenticity of activation requests
# This file is generated if not present
magnolia.author.key.location=${magnolia.home}/WEB-INF/config/default/magnolia-activation-keypair.properties

Apache Tomcat 9.0.8

If you use Apache Tomcat: we have upgraded to Apache Tomcat 9.0.8.

If you use another servlet container, check the certified stack for supported versions.

Third-party libraries

Jackrabbit 2.16.1

If you want to run Magnolia on Java 9 or 10, you must use Jackrabbit 2.16.1.

• jackrabbit-api-2.16.1
• jackrabbit-core-2.16.1
• jackrabbit-data-2.16.1
• jackrabbit-jcr-commons-2.16.1
• jackrabbit-spi-2.16.1
• jackrabbit-spi-commons-2.16.1

Vaadin 8.4.2

If you are migrating from Magnolia 5.5.x or lower, read the instructions for upgrading to Magnolia 5.6.x for custom modules and the widgetset.

• vaadin-compatibility-aceeditor-1.0
• vaadin-compatibility-ckeditor-1.0
• vaadin-compatibility-client-8.4.2
• vaadin-compatibility-context-menu-1.0
• vaadin-compatibility-expandingtextarea-1.0
• vaadin-compatibility-server-8.4.2
• vaadin-compatibility-shared-8.4.2
• vaadin-compatibility-themes-8.4.2
• vaadin-compatibility-tokenfield-1.0
• vaadin-server-8.4.2
• vaadin-shared-8.4.2
• validation-api-1.1.0.Final

Groovy 2.5

Magnolia 5.7 updates magnolia-module-groovy to version 2.7, which ships the latest stable version of Apache Groovy 2.5.

<dependency>
  <groupId>org.codehaus.groovy</groupId>
  <artifactId>groovy</artifactId>
  <version>2.5.0</version>
</dependency>
<dependency>
  <groupId>org.codehaus.groovy</groupId>
  <artifactId>groovy-xml</artifactId>
  <version>2.5.0</version>
</dependency>

Other third-party libraries

• commons-lang3-3.7
• freemarker-2.3.38
• gson-2.2.2
• guice-4.2.0
• guice-multibindings-4.2.0
• jackson-databind-2.9.5
• jsoup-1.8.3
• mycila-guice-closeable-4.0
• mycila-guice-injection-4.0
• mycila-guice-jsr-4.0
• snakeyaml-1.21

We removed Apache Xerces (xercesImpl) due to the fact that Java (since Java SE 7) already contains Java API for XML Processing (JAXP).

How to update

Recommendations

• Update to the latest minor release version first before upgrading to the most recent major release.
  A June 2018 example: a customer is considering an upgrade from 5.5.8 to 5.7. The correct upgrade sequence is: 5.5.8 → 5.5.16 → 5.6.14 → 5.7.x

• Update all external libraries required by the Magnolia release to which you intend to upgrade.
• Because the upgrade process takes time and is vulnerable to incidents, we recommend you minimize the risk by cleaning up your system, removing unused data, reindexing everything and doing a full backup first.

Updating manually

1. Stop the application server.
2. Extract the new Magnolia bundle.
3. Replace JAR files in the WEB-INF/lib folder of your old Magnolia instances with new JARs from the bundle. Also replace modules coming from the add-ons bundle (magnolia-enterprise-addons-bundle), in case you are using any of these.
    Magnolia 5.7 updates numerous 3rd-party libraries, see Java 8, 9 and 10 runtime compatibility and library updates.
4. Remove any module JARs you had previously removed from your instances. Add any modules you may have previously added.
    We have removed several add-on modules. You must also remove some. See Remove outdated add-on modules.
5. Add new Magnolia modules. See Add the privacy modules.
6. Optional: Delete all indexes to give them a boost. Delete the index folder under each workspace directory: repositories/magnolia/workspaces/<workspace>/index. Indexes are recreated on startup, which might take a while depending on the size of your repository.
7. If you customized your magnolia.properties files, compare the changes to the file in the new bundle. Properties may have been added or removed.
8. Read the release notes for all intermediate versions for any additional update tasks.
9. Restart the application server.
10. In your browser, go to the Magnolia instances and run the web update.

Updating Maven managed webapps

In this section we assume that you use Maven to manage your (custom) webapp. How you do this depends on how you have organized your pom files.

Upgrading the versions of inherited BOM files

The most typical use case is that your custom webapp is based on one of Magnolias preconfigured webapps. The structure of the Maven project that manages your webapp may look like this:

custom-ee/
├── custom-ee-webapp
│   ├── pom.xml
│   └── src
└── pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>com.example</groupId>
    <artifactId>custom-ee</artifactId>
    <version>1.0-SNAPSHOT</version>
    <relativePath>../pom.xml</relativePath>
  </parent>
  <artifactId>custom-ee-webapp</artifactId>
  <name>custom-ee: webapp</name>
  <packaging>war</packaging>

  <dependencies>
    <dependency>
      <groupId>info.magnolia.eebundle</groupId>
      <artifactId>magnolia-enterprise-pro-webapp</artifactId>
      <type>war</type>
    </dependency>
    <dependency>
      <groupId>info.magnolia.eebundle</groupId>
      <artifactId>magnolia-enterprise-pro-webapp</artifactId>
      <type>pom</type>
    </dependency>
    <!-- More custom modules here -->
    <dependency>
      <groupId>com.example</groupId>
      <artifactId>foobar-module</artifactId>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-war-plugin</artifactId>
        <configuration>
          <!-- exclude jars copied "physically" from the webapp overlay - so we only get those resolved by Maven's dependency management -->
          <dependentWarExcludes>WEB-INF/lib/*.jar</dependentWarExcludes>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Line 5: custom-ee/pom.xml is the parent pom file of your custom webapp. This file manages the dependencies and their versions. 

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.example</groupId>
  <artifactId>custom-ee</artifactId>
  <name>custom-ee (parent pom)</name>
  <version>1.0-SNAPSHOT</version>
  <packaging>pom</packaging>

  <properties>
    <magnoliaBundleVersion>5.6.6</magnoliaBundleVersion>
    <foobarModuleVersion>1.2</foobarModuleVersion>
    <javaVersion>1.8</javaVersion>
  </properties>

  <!-- Fill the following in, so you can use the release plugin -->
  <scm>
    <connection/>
    <developerConnection/>
    <url/>
  </scm>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>info.magnolia.eebundle</groupId>
        <artifactId>magnolia-enterprise-bundle-parent</artifactId>
        <version>${magnoliaBundleVersion}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
      <!-- More dependencies for your custom modules here -->
      <dependency>
        <groupId>com.example</groupId>
        <artifactId>foobar-module</artifactId>
        <version>${foobarModuleVersion}</version>
      </dependency>
    </dependencies>
  </dependencyManagement>


  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.7.0</version>
        <configuration>
          <source>${javaVersion}</source>
          <target>${javaVersion}</target>
        </configuration>
      </plugin>
    </plugins>

    <!-- default resources configuration which will filter your module descriptors -->
    <resources>
      <resource>
        <directory>src/main/resources</directory>
        <includes>
          <include>**/*</include>
        </includes>
      </resource>
      <resource>
        <filtering>true</filtering>
        <directory>src/main/resources</directory>
        <includes>
          <include>META-INF/magnolia/*</include>
        </includes>
      </resource>
    </resources>
  </build>

  <repositories>
    <repository>
      <id>magnolia.public</id>
      <url>https://nexus.magnolia-cms.com/content/groups/public</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
    </repository>
    <repository>
      <id>magnolia.enterprise.releases</id>
      <url>https://nexus.magnolia-cms.com/content/repositories/magnolia.enterprise.releases</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
    </repository>
    <repository>
      <id>vaadin-addons</id>
      <url>https://maven.vaadin.com/vaadin-addons</url>
    </repository>
  </repositories>

  <modules>
    <module>custom-ee-webapp</module>
  </modules>
</project>

Note that the parent pom (custom-ee/pom.xml) manages all versions for Magnolia modules as well as for third-party libraries.

It imports info.magnolia.eebundle:magnolia-enterprise-bundle-parent, which manages Magnolia Enterprise module versions and imports the following:

• info.magnolia.bundle:magnolia-bundle-parent - manages Magnolia CE module versions.
• info.magnolia.boms:magnolia-external-dependencies - manages third-party library versions.

If the pom files for your custom webapp are organized as shown in this example, you only need to change one property. In the <properties> section of your parent pom, change the version of the magnoliaBundleVersion property:

<properties>
  <magnoliaBundleVersion>5.7.1</magnoliaBundleVersion>
  <foobarModuleVersion>1.2</foobarModuleVersion>
  <javaVersion>1.8</javaVersion>
</properties>

Checking the Maven dependency tree

Regardless of how your Maven project is organized, building the Maven dependency tree helps you analyze the versions of all the Magnolia modules and all the third-party libraries of your custom webapp.

Open a shell, go to the directory of your webapp and execute the Maven command for the dependency tree:

cd /Users/jdoe/dev/repo/custom-mgnl-webapps/custom-ee/custom-ee-webapp
mvn dependency:tree

Known issues

Tomcat 8 using BCEL may throw class format exception

On Tomcat 8.5 or lower, you may encounter a class format exception.

SEVERE [localhost-startStop-1] org.apache.catalina.startup.ContextConfig.processAnnotationsJar Unable to process Jar entry [module-info.class] from Jar [file:/home/magnolia/dev/magnolia-5.6.6/apache-tomcat-8.5.5/webapps/magnoliaPublic/WEB-INF/lib/javax.json-api-1.1.jar] for annotations
 org.apache.tomcat.util.bcel.classfile.ClassFormatException: Invalid byte tag in constant pool: 19

Reason:

Apache Commons BCEL is used by Tomcat to scan for annotations. Some Java EE 8 APIs were produced from JDK9 as Multi-Release JARs. This means they're compatible with Java 8, but may also contain some Java 9 classes, including the module-info.class. Older Commons BCEL versions did not know how to read such classes.

Tomcat 8.5.12 and 9.0.0 updated Commons BCEL to support this. See the extract below from the Tomcat changelog.

60688: Update the internal fork of Apache Commons BCEL to r1782855 to add early access Java 9 support to the annotation scanning code. (markt)

Solution:

Use Tomcat 8.5.12 or higher.

If you encounter other problems, check the Known issues page.

Virtual URI mappings not working if too many are configured

To mitigate an issue caused by having more than 500 configured virtual URI mappings in light modules, a WARN-level message is now logged when a DirectoryWatcher overflow occurs (MAGNOLIA-7762). We recommend to keep the number of files in a single folder below 100 and to use folder hierarchies whenever possible. For the upcoming fix, see MAGNOLIA-7798.