Data module

The purpose of the Magnolia Data module is to manage structured data (as opposed to unstructured or semi-structured data commonly used for web pages). The Data module stores content independently of page structure. To place content data stored in the Data module on a page, the content for the paragraph or page template can be retrieved. In that regard, the Data module is similar to the DMS, which allows you to store documents independently of the page structure.

The Data module also provides functionality for automated import of external data which is especially useful for exposing data from other internal systems such as warehousing or CRM solutions to Magnolia. Using the Data module as intermediary storage for the imported data results in a single point of entry for such data which can then be used by the web pages in an uniform way, similar to other Magnolia content. The Data module also provides for post-import editing of the content so that automatically imported data can be modified for suitability for publishing.

Download

Download the Data module from Magnolia Store or Nexus repository.

Installing

Data is a community module included in the Standard Templating Kit bundle and is typically automatically installed. Go to Magnolia Store > Installed modules in AdminCentral to verify. To install the module individually, please see the general module installation instructions.

Uninstalling

See the general module uninstalling instructions and advice.

Usage

The Data module can be used similarly to DMS, except that Data module is designed to deal with content rather than with files (images, movies, documents). Following are suggestions on how the Data module can be used in projects:

• Boiler Plates: Define multiple text modules which can be referenced either within the template script or through a simple "Reference Paragraph".
• Look-up Tables: An example is to define a folder "US States" and for each state a node with properties like "abbreviation", "state name", "size of state" and more.
• Client references. The list of references on the Magnolia International website is driven by the Data module. By storing client information in the Data module you can generate different types of lists and choose the amount of detail displayed from same set of data.
• Contact database. Store contacts and display them on Web pages. For example, display the author's bio on any article they wrote.
• Product or manual catalogues for e-commerce solutions
• Categories and taxonomies. Create categories to characterize the site content and use them to tag articles. You can create an ad hoc taxonomy or import a third party classification scheme. Once an article is tagged the tag behaves as a navigation mechanism for finding related content.
• Other Ideas: The list of Magnolia References on the Magnolia website is driven from the Data module. By storing the client information in the Data module Magnolia can generate different types of lists with different amount of details displayed from the same set of data.

The Data module introduces the concept of node types. A node type is a tag for a node. Node type makes it easier to search "for nodes of type 'myNodeType'". Typically you start using the Data module by creating a custom node type for the items you are storing. Node types have to be registered before they can used. This occurs automatically when a new Data module entry is created in AdminCentral. When installing a custom module (using the Data module), the custom node types need to be bootstrap prior to bootstrapping config and content nodes. The Data module provides a special install task to perform this function and is explained further below.

Creating a data type

Everything in the Data module starts with the definition of the type. Here are the existing types in Data > Types.

To create a new Data type:

1. Click New Type.
2. Fill in the fields.
   
   Fields:
   
   • Name. The name of the type definition. This is used internally by the system, not displayed to users. The name will be a primary type of the content if the data is exported. Do not create a type named config. This is a reserved name.
   • Title is the name displayed to users in AdminCentral. It is used in the Data menu if no Menu title is entered.
   • Menu title is used in the Data menu.
   • Icon file. An image of size 16x16 pixels is recommended. The icon is used to represent the instances of the type in the Data menu tree as well as in the Data menu.
   • Root Path under which nodes of this type will be stored in the data workspace. The same path can be used for different types, however, if huge amounts of content will be imported for a type and search queries are executed on the type's content then a unique path should be specified so that search queries can be limited to a sub-path in order to simplify the search. Starts with a forwards slash, for example /contacts.
   • JCR Node Type is optional information for the type. Use JSR-283 specification syntax. The information can constrain the type to certain conditions or define auto-created properties or subnodes. Most types will not need this information.
   • Allow Folders. Check the box if you want to allow creation of folders under the type.
   • Sort by Name. Check to sort data of this type in alphabetical order. To see the data items for this type, clicked the type in the Data menu.
3. Save.

The system creates the following items:

• config:/modules/adminInterface/config/menu/data/<type>. Menu item. Refresh AdminCentral to see it.
• config:/modules/data/config/types/<type>
• config:/modules/data/dialogs/<type>
• config:/modules/data/trees/<type>
• data:<rootPath>. As defined when creating the type. Starts with "/".

When you delete the data type you may need to remove some of these items manually.

    @Override
    protected List getBasicInstallTasks(InstallContext installContext) {
        final List<Task> installTasks = new ArrayList<Task>();
        // make sure we register the type before doing anything else
        installTasks.add(new RegisterNodeTypeTask("RSSAggegator"));
        installTasks.addAll(super.getBasicInstallTasks(installContext));
        return installTasks;
    }

Editing the type dialog

When a type is created, a dialog is automatically created that will be used to create and edit data for the type. The dialog will be given an input only for a Name field upon creation but can be modified to add additional controls. For the following procedures the specific type for which dialog is being modified will be referred to as type.

To modify a dialog for a type:

1. Click Data > Config (Dialogs).
   
2. Click type under modules/data/dialogs to open to see tabs in dialog (the Contact type is opened in the example below).
   

To create a new tab:

1. Select type.
2. Click New content node in toolbar at bottom of the page or in the context menu (right-click popup). The new content node is the new tab.
3. Double click name field of new tab to open field for entry and enter a name for the new tab.
   
4. Select new tab.
5. click New node data in toolbar at bottom of the page or in the context menu (right-click popup).
6. Double click name field of new node and set to controlType.
7. Double click Value field of new node and set to tab.
8. Double click Type field of new node and set to String. See result of adding below.
9. Select new tab.
10. Click New node data in toolbar at bottom of the page or in the context menu (right-click popup).
11. Repeat steps for setting new node so name is label, Value is a string (label of tab), and Type is String. See result of adding below.
    

To add a control to a tab (for input on the tab):

1. Select tab.
2. Click New content node in toolbar at bottom of the page or in the context menu (right-click popup).
3. Double click name field of new control on tab to open field for entry and enter a name (same as changing name of tab above).
4. Select new control and click New node data in toolbar at bottom of the page or in the context menu (right-click popup).
5. Repeat steps for setting new node so name is controlType, Value is a type of control (see Controls for list of control types), and Type is a type of input value. See result of adding below.
   
6. Repeat steps for adding new node so name is label, Value is a string (label of control), and Type is String. See result of adding above.

Other settings may be necessary for controls on a tab (e.g. rows, boxType, etc.) depending on the type and usage of the control. See Controls for information on additional settings as well as possible input values.

Custom controls can also be created for Data type dialogs. A custom control should not add complexity to the structure under the type as this can result in problems with activating the Node Types.

See Dialogs for more information on dialogs.

Creating data

Once a type has appeared in the Data menu, data that has been created for that type can be listed by clicking the type in the Data menu. The Data > Category page in the demo author instance is below as an example:

To create data for a type (the specific type for which data is being created will be referred to as type:

1. Click Data > type.
2. Click New Item button in the toolbar at the bottom of the page or New Item in the context menu (right-click popup). The dialog for the Data type will appear. The dialog that appears when creating data on Data > Contact is below.
   
3. Enter data as needed in the dialog.

Importing data

The diagram below illustrates how data is imported from an external system to Magnolia and integrated into Web content. The example uses product information. The same mechanism applies to other kinds of data.

Data can be imported into Magnolia with import handlers that are defined for the data type. Import handlers are special tasks that import content into the repository and can be invoked periodically to synchronize data in the Data module repository with external data storage. Import handlers can also be configured to activate all the changes to public instances after finishing the import.

The import handlers are listed at Data > Config (Dialogs)/modules/data/config/importers as shown below.

Configuration for import handlers are shown below:

Configuration properties:

Property	Description
targets	Collection of targets into which imported data should be saved. In most cases the single target main should be enough.
targets/main/class	Name of the class transforming imported data into a format suitable for data storage. The info.magnolia.module.data.importer.SimpleImportTarget is suitable for most of the cases.
targets/main/targetPath	The path for storing the content by this target. In most cases this path should match the root path specified for the Node Type in the Node Type definition.
automatedExecution/cron	Cron job pattern for scheduling automatic execution of the ImportHandler. See Quartz Cron Pattern for more details on supported patterns.
automatedExecution/enabled	Set to true to enable automatic execution of the ImportHandler based on the specified cron pattern.
activateImport	Set to true when imported content should be automatically activated after successful import.
backupPath	Sets the backup location. In previous versions this is hard coded. The default path is /<CATALINA_HOME>/webapps/<contextPath>/backup.  4.5.10+
class	The import Handler definition. The class extending abstract info.magnolia.module.data.importer.ImportHandler. This class will be called by the ImportCommand to execute the import.
deleteBackupAfterImport	Allows to automatically delete backup after successful import. The default for this property is false.  4.5.10+
deleteOldData	Set to true when the data not found in the external system should be automatically deleted (and deactivated).
lockDuringImport	Controls whether exclusive lock should be issued on the parent content to prevent any other updates during run of the importer. This property provides the ability to define the degree of control for long running and often executed imports. In case the first import is not finished, the lock will prevent any other imports from running. The lock will also prevent any manual editing of the content while the import is running. If the import handler fails to connect to the external data source and doesn't timeout or get otherwise locked when communicating with the external system, the lock will prevent any further updates to the affected Type. The only workaround in such case is to export the content, delete it and re-import. Use the content locking only when sure that your ImportHandler will not end up hanging during the import process while communicating with the external system.
repository	Target repository for imported data. By default the value should be data, but in case it is necessary for the import handler to import data into another repository, it can be changed here. Only one repository can be specified at a time.
fileURL	URL of the file to import. This can be a path starting with // or a path relative to magnolia.app.rootdir. If the data importer cannot resolve the path, it returns an empty string. Likewise, if you set fileURL to /myFile.csv it looks for the file in disk root directory.
zipFileURL	Zip file containing additional files such as images or documents.

Hierarchical data

Hierarchical structures for Data types was added with Data module 1.4 in order to provide realistic and representative relationships between data that have relationships of a hierarchical nature. This structure allows Data types to "parent" other Data types. The "child" Data types are referred to as subtypes.

An example of this structure is provided on the demo author instance with the company Data type. The structure, which can be accessed at Data > Types/company is shown below:

The company type has the types department and employees as "child" types. Likewise, the type employees has three "child" types: director, manager, developer. This structure represents a company that is comprised of departments and employees with three types of employees: directors, managers, developers.

Stored data that corresponds to this hierarchical types structure on the demo author instance at Data > Company/Big-Flower-Inc is shown below:

The company in this case is, of course, Big-Flower-Inc. Big-Flower-Inc has two department}}s, Publishing and Debt-Extraction. Big-Flower-Inc has two groups of {{employees, Permanent-Employees and Temporary-Workers. Permanent-Employees has one manager (Jack-Middleman), one developer (Joe-Workhard), and one director (Jimmy-Important). Temporary-Workers has two developers, Tim-Admin and Randy-Coder.

To determine the Data type of stored data:

1. Examine the icon of the data. Icons are specified in the data type dialog (see description for creating a new data type).
2. Edit the data by double clicking the icon. The dialog should provide identification of the Data type.

To create a subtype:

1. Select the "parent" type.
2. Click New Type button in the toolbar at the bottom of the page or New Type in the context menu (right-click popup).

See the description for creating a new data type for the remaining steps in creating the subtype. Note that for a subtype, Root Path, Allow Folders, and Sort by Name cannot be set as shown in the subtype dialog below:

Types and subtypes are separate content types in the underlying repository. Since the repository requires type names to be unique, subtypes of the same name cannot exist under more than one type.

To move a subtype to another parent type:

1. Select subtype to move.
2. Click Move in context menu (right-click popup).
3. Move subtype to new parent type. Location is indicated as a line below the parent. In image below the director subtype is being moved to under the department type.
   

When activating a type, all of the subtypes can also be activated by clicking *Activate incl. sub nodes" in the (right-click popup) context menu.

Commands

The Data module provides commands upon installation. The commands are used internally and are available for free use to anyone.

Installed commands are divided into two catalogues. The dataType catalogue contains commands specific to the Node Types definitions and the data catalogue contains commands common to the all defined types. The two catalogues found at Data > Config (Dialogs)/modules/data/commands are shown below:

The data catalogue commands are:

• activate activates content from the Data repository. The node of the given custom type and all it's sub-nodes of type dataItemNode are configured as one piece of content and will be activated together. Sub nodes of any other types will not be activated together with the content.
• deactivate deactivates specified content and it's sub-nodes in the Data repository removing them from the public instance.
• import will invoke the default ImportHandler. This command will try to find the importer definition matching the name of the type at Data > Config (Dialogs)/modules/data/config/importers. If a matching importer is found, it will be invoked. If not, the command will delegate to the importData command.
• deleteAll deactivates and deletes all content in a specified path. This command may be optionally called by ImportHandler.
• importData allows for manual invocation of ImportHandler directly from the Import data button in the toolbar at the bottom of the Data > type page. The ImportHandler can be specified in the product/importer entry of the command. The value specified here has to match one of the ImportHandlers defined at Data > Config (Dialogs)/modules/data/config/importers. When called, command will try to invoke the importer specified in its sub-node product/importer.
• activateAll activates all the content including all the sub content. This command may be optionally called by ImportHandler.
• deactivateAll deactivates all the content in given path. This command may be optionally called by ImportHandler.

The dataType catalogue commands are:

• delete ensures that all the structures defined for the type (dialog, tree, sub menu entry) are deleted with the type.
• importData is identical to the importData command in the data catalogue.
• activate is identical to the activate command in the data catalogue.

Examples

See Using the Data module to import data.

The modules  RSS Aggregator and Categorization module use the Data module for data storage. See the demo author instance to see the data types for those modules.