Similar content

Loading

Powered by Canoo FindIT.

Data

The purpose of the Magnolia Data module is to manage structured data (as opposed to unstructured or semi-structured data commonly used for web pages). The Data module stores content independently of page structure. To place content data stored in the Data module on a page, the content for the paragraph or page template can be retrieved. In that regard, the Data module is similar to the DMS, which allows you to store documents independently of the page structure.

The Data module also provides functionality for automated import of external data which is especially useful for exposing data from other internal systems such as warehousing or CRM solutions to Magnolia. Using the Data module as intermediary storage for the imported data results in a single point of entry for such data which can then be used by the web pages in an uniform way, similar to other Magnolia content. The Data module also provides for post-import editing of the content so that automatically imported data can be modified for suitability for publishing.

Download

Download the Data module from Magnolia Store or Nexus repository.

Installing

Data is a community module included in the Standard Templating Kit bundle and is typically automatically installed. Go to Magnolia Store > Installed modules in AdminCentral to verify. To install the module individually, please see the general module installation instructions.

Uninstalling

See the general module uninstalling instructions and advice.

Usage

As mentioned above, the Data module can be used similarly to DMS, except that Data module is designed to deal with content rather than with files (images, movies, documents). Following are suggestions on how the Data module can be used in projects:

  • Boiler Plates: Define multiple text modules which can be referenced either within the template script or through a simple "Reference Paragraph".
  • Look-up Tables: An example is to define a folder "US States" and for each state a node with properties like "abbreviation", "state name", "size of state" and more.
  • Product or manual catalogues for e-commerce solutions
  • Other Ideas: The list of Magnolia References on Magnolia's website is driven from the Data module. By storing the client information in the Data module Magnolia can generate different types of lists with different amount of details displayed from the same set of data.
The Data module introduces users to the concept of Node Types. Node Types are nothing new and underpin the whole working of the repositories. For further reading it would be beneficial to get familiar with the concept, for example by reading the JSR-283 specification for more details. Node Types are referred to as Types (or type) in AdminCentral.

A Node Type is (very) simply speaking, a tag for a node. Node Type makes it easier to search "for nodes of type 'myNodeType'". The concept of starting a Data module by creating a custom Node Type is fundamental to understanding Data modules. Note that Node Types have to be registered before they can used. This occurs automatically when a new Data module entry is created in AdminCentral. When installing a custom module (using the Data module), the custom Node Types will need to be bootstrap before the config and content nodes are bootstrap. The module provides a special install task to perform this function and is explained further below.

Creating a type

Everything in the Data module starts with the definition of the type. In the image below you can see the Data > Types page listing all the existing types.

To create a new Data type:

  1. Click the New Type button in the toolbar at the bottom of the page or New Type in the context menu (right-click popup).

  2. Fill in the fields:
    • Name. The name of the type definition. This is used internally by the system, not displayed to users. The name will be a primary type of the content if the data is exported. Important! Do not create a type named config. This is a reserved name already used elsewhere.
    • Title is the name displayed to users in AdminCentral who work with this data type. It will be used in the Data menu if no Menu title is given.
    • Menu title is used in the Data menu instead of Title.
    • Icon file. An image of size 16x16 pixels is recommended. The icon is used to represent the instances of the type in the Data menu tree as well as in the Data menu.
    • Root Path under which nodes of this type will be stored in the data workspace. The same path can be used for different types, however, if huge amounts of content will be imported for a type and search queries are executed on the type's content then a unique path should be specified so that search queries can be limited to a sub-path in order to simplify the search. Starts with a forwards slash, for example /contacts.
    • JCR Node Type is optional information for the type. Use JSR-283 specification syntax. The information can constrain the type to certain conditions or define auto-created properties or subnodes. Most types will not need this information.
    • Allow Folders. Check the box if you want to allow creation of folders under the type.
    • Sort by Name. Check to sort data of this type in alphabetical order. To see the data items for this type, clicked the type in the Data menu.
  3. Save.
The system creates the following items:
  • config:/modules/adminInterface/config/menu/data/<type>. Menu item. Refresh AdminCentral to see it.
  • config:/modules/data/config/types/<type>
  • config:/modules/data/dialogs/<type>
  • config:/modules/data/trees/<type>
  • data:<rootPath>. As defined when creating the type. Starts with "/".
When you delete the data type you may need to remove some of these items manually.

Tip

Node types can also be created programmatically (e.g. bootstrapping types and their data from a custom module) by using info.magnolia.module.data.setup.RegisterNodeTypeTask. This task needs to be added at the beginning of the basicInstallTasks of your VersionHandler to ensure that when bootstrapping content that is using the node type, the type is already registered. An example of this task is:

    @Override
    protected List getBasicInstallTasks(InstallContext installContext) {
        final List<Task> installTasks = new ArrayList<Task>();
        // make sure we register the type before doing anything else
        installTasks.add(new RegisterNodeTypeTask("RSSAggegator"));
        installTasks.addAll(super.getBasicInstallTasks(installContext));
        return installTasks;
    }

Modifying type dialog

When a type is created, a dialog is automatically created that will be used to create and edit data for the type. The dialog will be given an input only for a Name field upon creation but can be modified to add additional controls.

For the following procedures the specific type for which dialog is being modified will be referred to as type.

To modify a dialog for a type:

  1. Click Data > Config (Dialogs).
  2. Click type under modules/data/dialogs to open to see tabs in dialog (the Contact type is opened in the example below).
To create a new tab:
  1. Select type.
  2. Click New content node in toolbar at bottom of the page or in the context menu (right-click popup). The new content node is the new tab.
  3. Double click name field of new tab to open field for entry and enter a name for the new tab.
  4. Select new tab.
  5. click New node data in toolbar at bottom of the page or in the context menu (right-click popup).
  6. Double click name field of new node and set to controlType.
  7. Double click Value field of new node and set to tab.
  8. Double click Type field of new node and set to String. See result of adding below.
  9. Select new tab.
  10. Click New node data in toolbar at bottom of the page or in the context menu (right-click popup).
  11. Repeat steps for setting new node so name is label, Value is a string (label of tab), and Type is String. See result of adding below.
To add a control to a tab (for input on the tab):
  1. Select tab.
  2. Click New content node in toolbar at bottom of the page or in the context menu (right-click popup).
  3. Double click name field of new control on tab to open field for entry and enter a name (same as changing name of tab above).
  4. Select new control and click New node data in toolbar at bottom of the page or in the context menu (right-click popup).
  5. Repeat steps for setting new node so name is controlType, Value is a type of control (see Controls for list of control types), and Type is a type of input value. See result of adding below.
  6. Repeat steps for adding new node so name is label, Value is a string (label of control), and Type is String. See result of adding above.
Other settings may be necessary for controls on a tab (e.g. rows, boxType, etc.) depending on the type and usage of the control. See Controls for information on additional settings as well as possible input values.

Custom controls can also be created for Data type dialogs. A custom control should not add complexity to the structure under the type as this can result in problems with activating the Node Types.

See Dialogs for more information on dialogs.

Creating data

Once a type has appeared in the Data menu, data that has been created for that type can be listed by clicking the type in the Data menu. The Data > Category page in the demo author instance is below as an example:

To create data for a type (the specific type for which data is being created will be referred to as type):

  1. Click Data > type.
  2. Click New Item button in the toolbar at the bottom of the page or New Item in the context menu (right-click popup). The dialog for the Data type will appear. The dialog that appears when creating data on Data > Contact is below.
  3. Enter data as needed in the dialog.

Importing data

The diagram below illustrates how data is imported form an external system to Magnolia CMS and integrated into Web content. The example uses product information. The same mechanism applies to other kinds of data.

Data can be imported into Magnolia with import handlers that are defined for the data type. Import handlers are special tasks that import content into the repository and can be invoked periodically to synchronize data in the Data module repository with external data storage. Import handlers can also be configured to activate all the changes to public instances after finishing the import.

The import handlers are listed at Data > Config (Dialogs)/modules/data/config/importers as shown below.

Configuration for import handlers are shown below:

Configuration properties:

Property Description
targets Collection of targets into which imported data should be saved. In most cases the single target main should be enough.
targets/main/class Name of the class transforming imported data into a format suitable for data storage. The info.magnolia.module.data.importer.SimpleImportTarget is suitable for most of the cases.
targets/main/targetPath The path for storing the content by this target. In most cases this path should match the root path specified for the Node Type in the Node Type definition.
automatedExecution/cron Cron job pattern for scheduling automatic execution of the ImportHandler. See Quartz Cron Pattern for more details on supported patterns.
automatedExecution/enabled Set to true to enable automatic execution of the ImportHandler based on the specified cron pattern.
activateImport Set to true when imported content should be automatically activated after successful import.
class The import Handler definition. The class extending abstract info.magnolia.module.data.importer.ImportHandler. This class will be called by the ImportCommand to execute the import.
deleteOldData Set to true when the data not found in the external system should be automatically deleted (and deactivated).
lockDuringImport Controls whether exclusive lock should be issued on the parent content to prevent any other updates during run of the importer. This property provides the ability to define the degree of control for long running and often executed imports. In case the first import is not finished, the lock will prevent any other imports from running. The lock will also prevent any manual editing of the content while the import is running. If the import handler fails to connect to the external data source and doesn't timeout or get otherwise locked when communicating with the external system, the lock will prevent any further updates to the affected Type. The only workaround in such case is to export the content, delete it and re-import. Use the content locking only when sure that your ImportHandler will not end up hanging during the import process while communicating with the external system.
repository Target repository for imported data. By default the value should be data, but in case it is necessary for the import handler to import data into another repository, it can be changed here. Only one repository can be specified at a time.
fileURL URL of the file to import. This can be a path starting with // or a path relative to magnolia.app.rootdir. If the data importer cannot resolve the path, it returns an empty string. Likewise, if you set fileURL to /myFile.csv it looks for the file in disk root directory.
zipFileURL Zip file containing additional files such as images or documents.

Hierarchical types/data

Hierarchical structures for Data types was added with Data module 1.4 in order to provide realistic and representative relationships between data that have relationships of a hierarchical nature. This structure allows Data types to "parent" other Data types. The "child" Data types are referred to as subtypes.

An example of this structure is provided on the demo author instance with the company Data type. The structure, which can be accessed at Data > Types/company is shown below:

The company type has the types department and employees as "child" types. Likewise, the type employees has three "child" types: director, manager, developer. This structure represents a company that is comprised of departments and employees with three types of employees: directors, managers, developers.

Stored data that corresponds to this hierarchical types structure on the demo author instance at Data > Company/Big-Flower-Inc is shown below:

The company in this case is, of course, Big-Flower-Inc. Big-Flower-Inc has two departments, Publishing and Debt-Extraction. Big-Flower-Inc has two groups of employees, Permanent-Employees and Temporary-Workers. Permanent-Employees has one manager (Jack-Middleman), one developer (Joe-Workhard), and one director (Jimmy-Important). Temporary-Workers has two developers, Tim-Admin and Randy-Coder.

To determine the Data type of stored data:

  1. Examine the icon of the data. Icons are specified in the data type dialog (see description for creating a new data type).
  2. Edit the data by double clicking the icon. The dialog should provide identification of the Data type.
To create a subtype:
  1. Select the "parent" type.
  2. Click New Type button in the toolbar at the bottom of the page or New Type in the context menu (right-click popup).
See the description for creating a new data type for the remaining steps in creating the subtype. Note that for a subtype, Root Path, Allow Folders, and Sort by Name cannot be set as shown in the subtype dialog below:

Types and subtypes are separate content types in the underlying repository. Since the repository requires type names to be unique, subtypes of the same name cannot exist under more than one type.

To move a subtype to another parent type:

  1. Select subtype to move.
  2. Click Move in context menu (right-click popup).
  3. Move subtype to new parent type. Location is indicated as a line below the parent. In image below the director subtype is being moved to under the department type.
When activating a type, all of the subtypes can also be activated by clicking *Activate incl. sub nodes" in the (right-click popup) context menu.

Commands

The Data module provides commands upon installation. The commands are used internally and are available for free use to anyone.

Installed commands are divided into two catalogues. The dataType catalogue contains commands specific to the Node Types definitions and the data catalogue contains commands common to the all defined types. The two catalogues found at Data > Config (Dialogs)/modules/data/commands are shown below:

The data catalogue commands are:

  • activate activates content from the Data repository. The node of the given custom type and all it's sub-nodes of type dataItemNode are configured as one piece of content and will be activated together. Sub nodes of any other types will not be activated together with the content.
  • deactivate deactivates specified content and it's sub-nodes in the Data repository removing them from the public instance.
  • import will invoke the default ImportHandler. This command will try to find the importer definition matching the name of the type at Data > Config (Dialogs)/modules/data/config/importers. If a matching importer is found, it will be invoked. If not, the command will delegate to the importData command.
  • deleteAll deactivates and deletes all content in a specified path. This command may be optionally called by ImportHandler.
  • importData allows for manual invocation of ImportHandler directly from the Import data button in the toolbar at the bottom of the Data > type page. The ImportHandler can be specified in the product/importer entry of the command. The value specified here has to match one of the ImportHandlers defined at Data > Config (Dialogs)/modules/data/config/importers. When called, command will try to invoke the importer specified in its sub-node product/importer.
  • activateAll activates all the content including all the sub content. This command may be optionally called by ImportHandler.
  • deactivateAll deactivates all the content in given path. This command may be optionally called by ImportHandler.
The dataType catalogue commands are:
  • delete ensures that all the structures defined for the type (dialog, tree, sub menu entry) are deleted with the type.
  • importData is identical to the importData command in the data catalogue.
  • activate is identical to the activate command in the data catalogue.
Tip

Each of the activation commands allow customization with respect to determining the portion of the content to include during activation and the portion that should be left out. By default, commands are configured to include anything of type dataItemNode as part of the content and to activate it with the parent type. Other types could be added by changing the value of the itemTypes property of the activation command. The itemTypes being added to be activated with the parent type, need to be specified in a comma separated list. If updating this entry, verify that all the activation commands defined by the Data module are updated to contain the same values.

Examples

See Using the Data module to import data.

The modules RSS Aggregator and Categorization use the Data module for data storage. See the demo author instance to see the data types for those modules.