Magnolia 5.4 reached end of life on November 15, 2018. This branch is no longer supported, see End-of-life policy.
The RSS Aggregator module displays external feed content on a Magnolia page (feed aggregation) and generates feeds from Magnolia content (feed syndication).
Planet feeds enhance standard aggregation feeds by collecting additional data and generating feed statics.
A feed aggregator retrieves external feeds and displays the content on a Magnolia page.
Planet feeds have all the features of standard feeds but can generate additional data and statistics that is displayed in planet components. See Feed components and Generating planet data for more.
In this example we collect three individual RSS feeds into one aggregate feed:
These have a common topic – celebrity gossip – so they are good candidates for aggregation. A reader interested in one feed is likely to be interested in the others too.
To create a RSS aggregator, in the Feeds app:
CelebrityGossip
. This is the internal name of the aggregator.Feeds can be marked as a planet feed at any time. When you change a standard feed to a planet feed, re-import the feed data before generating planet data
To import data:
See also: Scheduling automated feed imports.
The module includes components that display external feeds and generate feeds from internal content.
Content will only display in the feed components after it has been imported. To display feed content on the public instance publish the feed in the Feeds app. This will also publish the feed data.
The feed components are configured in /modules/rssaggregator/templates/components
.
Node name | Value |
---|---|
rssaggregator | |
templates | |
components | |
combinedFeedsParagraph | |
feedListParagraph | |
planetFeeds | |
feedSyndication | |
feedStatistics |
combinedFeedsParagraph
| Combines all feeds in a channel and displays them sequentially. Editors can set the number, sort order and character limit of the entries. Use with standard or planet feeds. | |
feedListParagraph
| Displays a defined number of entries for each feed in the channel. The external feed title is used if no internal title is set. Editors can set the number, sort order and character limit of the entries. Use with standard or planet feeds. | |
planetFeeds
| Displays the entries of all feeds in a channel with the latest appearing first. Uses the HTML of the source entry. This means that images and links are preserved and displayed. Allows pagination. Use with planet feeds only. | |
feedStatistics
| Uses planet statistics data to display a list of authors with feed subscription icons and links to the external website. Author list is ordered by post frequency with top contributors appearing first. Editors can set a title and subtitle, define the number of authors to include, and whether links and the post count should be included. Use with planet feeds only. | |
feedSyndication
| Displays Atom and RSS subscription icons and links that allow users to subscribe to the feed on your site. Editors can add a title and select a planet feed. Use with planet feeds only. |
The model class and template script are the important properties that determine content in the different components. All RSS model classes extend
AbstractFeedModel
that provides the business logic to retrieve a defined feed and it's data from the rss
workspace and supports:
Entry sorting by title or publication data.
Ascending and descending entry sorting.
Maximum results property. Default is 20
.
Search capabilities
Any custom model class should extend AbstractFeedModel
.
You can add the feed components to any template.
Example: Template definition with all feed components in main
area.
templateScript: /my-module/templates/pages/my-script.ftl renderType: freemarker visible: true title: My template areas: main: availableComponents: combinedFeeds: id: rssaggregator:components/combinedFeedsParagraph feedList: id: rssaggregator:components/feedListParagraph planetFeeds: id: rssaggregator:components/planetFeeds feedStatistics: id: rssaggregator:components/feedStatistics feedSyndication: id: rssaggregator:components/feedSyndication
Node name | Value |
---|---|
myTemplate | |
areas | |
main | |
availableComponents | |
combinedFeeds | |
id | rssaggregator:components/combinedFeedsParagraph |
feedList | |
id | rssaggregator:components/feedListParagraph |
planetFeeds | |
id | rssaggregator:components/planetFeeds |
feedStatistics | |
id | rssaggregator:components/feedStatistics |
feedSyndication | |
id | rssaggregator:components/feedSyndication |
templateScript | /my-module/templates/pages/my-script.ftl |
renderType | freemarker |
visible | true |
title | My Template |
Importing feed data manually is not a long-term solution. You should configure an automated import schedule. Choose the update frequency depending on how often feeds have new content. For example, if a blog gets one post a day then importing once a day at 6 a.m. is enough.
You can schedule automated imports for all feeds in the Configuration app. The global settings can be overridden for single feeds.
0 0 6 * * *
imports data daily at 6 a.m. See Cron Maker for help.The input in the app is stored in the module configuration in /modules/rssaggregator/config/
.
Node name | Value |
---|---|
rssaggregator | |
config | |
planetOptions | |
feedGenerators | |
automatedImport | true |
cron | 0 0 0/1 1/1 * ? * |
fetcherClass | info.magnolia.module.rssaggregator.importhandler.FastRSSFeedFetcher |
importTimingSetter | CronMaker |
Properties:
config | required Module configuration node. |
| required, default is Enables and disables automated imports. |
| required The cron pattern for scheduled imports. |
| required FastRSSFeedFetcher retrieves RSS feed content over HTTP. Fetches all feed channels defined in an aggregate feed. You can use SimpleRSSFeedFetcher as an alternative. This is a simple, single-threaded fetcher that reduces server load and is suitable when you don't have many feed channels and you don't import often. |
| required Utility used to build the cron expression. |
You can configure automated import settings for each feed in the Import Settings tab of the edit dialog. These setting override the global settings.
0 0 6 * * *
imports data daily at 6 a.m. See Cron Maker for help.Feed data is stored in the rss
workspace.
Example: CelebrityGossip feed in the JCR Browser app.
Node name | Value |
---|---|
CelebrityGossip | |
data | |
channel-02 | |
entry-0 | |
categories | |
author | Perez Hilton |
channelTitle | Perez Hilton |
content | <p><strong></p><p>Yolanda Foster</strong> has been <a href="http://perezhilton.com/2015-12-09-yo... |
description | Yolanda Foster has been coming to terms with her divorce from David Foster ever since announcing it... |
link | http://perezhilton.com/2015-12-10-yolanda-foster-divorce-david-foster-focus-on-her-health-on-the-mend |
pubDate | 1,449,759,236,000 |
title | Yolanda Foster Thinks Divorce Will Help Her Fight With Chronic Lyme Disease — Here's How! |
entry-1 | |
entry-2 | |
.... | |
description | Perez Hilton dishes up the juiciest celebrity gossip on all your favorite stars, from Justin Bieber to Kim Kardas.. |
link | http://perezhilton.com |
rss | http://i.perezhilton.com/?feed=rss2 |
title | PerezHilton |
type | rss_2.0 |
channel-00 | |
channel-01 | |
automatedImport | true |
cron | 0 0/15 * 1/1 * ? * |
importTimerSetter | bgvCronMaker |
name | CelebrityGossip |
overrideDefault | false |
title | Celebrity Gossip |
Structure:
<RSS aggregator name>
| |
| Data retrieved from the Internet. The content of this folder is that same for standard and planet feeds. This content is used for standard feed components |
| A channel is created for each RSS feed. |
| One entry in the feed such as one blog post. |
| |
| |
| |
.... | |
| Feed properties retrieved from the internet. |
| |
.... | |
| Internal feed properties. |
| |
.... |
The module includes two custom commands to generate planet data.
The commands are configured in /modules/rssaggregator/commands/planet/
.
Node name | Value |
---|---|
rssaggregator | |
commands | |
planet | |
generatePlanetData | |
class | info.magnolia.module.rssaggregator.generator.PlanetDataGenerator |
collectPlanetStatistics | |
class | info.magnolia.module.rssaggregator.generator.CollectStatisticsCommand |
planet | required Planet commands node. |
| required Generate planet data command node. |
| required PlanetDataGenerator :
|
| required Collect planet statistics command node. |
| required CollectStatisticsCommand :
|
Use the Scheduler module to execute the planet commands to generate planet data and schedule regular updates.
Two jobs are preconfigured in /modules/scheduler/config/jobs
:
Node name | Value |
---|---|
scheduler | |
config | |
jobs | |
generatePlanetData | |
params | |
repository | rss |
active | true |
catalog | planet |
command | generatePlanetData |
cron | 0 0/10 * 1/1 * ? * |
description | generate data for RSS planet |
collectPlanetStatistics | |
params | |
repository | rss |
active | true |
catalog | planet |
command | collectPlanetStatistics |
cron | 0 0/18 * 1/1 * ? * |
description | generate statistics for RSS planet |
Properties:
jobs | required Node for scheduled jobs |
| required Name of job. |
| required/optional Parameters passed to the command. |
| required Workspace where the content item resides. |
| required Enables and disables the job. Set to |
| required Name of the catalog where the command resides |
| required Name of the command. |
| required CRON expression that sets the scheduled execution time. You can use the pattern |
| optional Description of the job. |
Planet data is used in planet components. The data is stored in the planetData
folder in the rss
workspace. View the data in your custom JCR browser.
Example: CelebrityGossip feed after marking it as a planet feed, re-importing the data, and running the generatePlanetData
job.
Node name | Value |
---|---|
CelebrityGossip | |
data | |
planetData | |
posts-00000 | |
entry-1 | |
author | Perez Hilton |
authorLink | http://perezhilton.com |
channelTitle | PerezHilton |
checksum1 | 1f271e498d7aa6352d38a1d8ae6707bb |
checksum2 | 6d2abcefcb5e196c76494f0f4b7a1d04 |
description | <p><strong></p><p>Yolanda Foster</strong> has been <a href="http://perezhilton.com/2015-12-09-yolanda-... |
hidden | false |
link | http://perezhilton.com/2015-12-10-yolanda-foster-divorce-david-foster-focus-on-her-health-on-the-mend |
pubDate | 1,449,759,236,000 |
rssLink | http://i.perezhilton.com/?feed=rss2 |
title | Yolanda Foster Thinks Divorce Will Help Her Fight With Chronic Lyme Disease — Here's How! |
entry-2 | |
entry-3 | |
.... |
Structure:
<aggregator name>
| |
| The GeneratePlanetData command reads feed data and stores it here. |
| All posts from all feeds. First all entries from the first feed, newest entry first. Then all posts from the second feed, and so on. |
| One entry in the feed such as one blog post. |
| |
| |
| |
| The module uses checksums to handle duplicate entries. Because feed data is deleted and recreated on every run, there is a high probability that a subsequent run will include entries that were contained in a previous run. Some entries may only have changed slightly, for example a different publication date. To avoid duplicates, the PlanetDataGenerator command uses checksums for each entry. Two checksum properties are generated. If an archive node with one of the checksums exists, no data is stored for the new item and an INFO level entry is written in the logs. |
| |
| |
| |
| |
| |
|
Planet statistics are generated from planet data and stored in the statistics
folder in the rss
workspace. View the data in your custom JCR browser.
Example: CelebrityGossip feed after running the generatePlanetStatistics
job.
Node name | Value |
---|---|
CelebrityGossip | |
data | |
planetData | |
statistics | |
authors | |
author-0 | |
author-1 | |
author-2 | |
countedPosts | |
af0a8534-d0eb-4bd4-8c03-c1051438468a | |
f4c3ca83-f766-4ae1-bb33-97761916fd7d | |
.... | |
author | Perez Hilton |
blogLink | http://perezhilton.com |
feedLink | http://i.perezhilton.com/?feed=rss2 |
postCount | 20 |
author-3 | |
.... |
Structure:
<aggregator name>
| |
| The CollectPlanetStatistics command extracts statistics from the /planetData node. This node is deleted and recreated on every run of the command. |
| All authors from all feeds. |
| Each author is allocated a number. |
| Each child node is a reference to a post in the feed. |
| |
.... | |
| |
| |
| |
| Number of posts by this author in the aggregate feed. |
Planet data is generated and stored for the last 3 months by default. You can configure the time period for which the data is retained in /modules/rssaggregator/config/planetOptions
.
Node name | Value |
---|---|
rssaggregator | |
config | |
planetOptions | |
lastMonthsIncluded | 3 |
Properties:
planetOptions
| required Planet options |
| required, default is Number of months. |
Feed generators generate RSS feeds from Magnolia content and imported content stored in the rss
workspace.
Four feed generators are registered in /modules/rssaggregator/config/feedGenerators
.
Node name | Value |
---|---|
modules | |
rssaggregator | |
config | |
feedGenerators | |
rss | |
class | info.magnolia.module.rssaggregator.generator.RSSModuleFeedGenerator |
planet | |
class | info.magnolia.module.rssaggregator.generator.PlanetFeedGenerator |
category | |
class | info.magnolia.module.categorization.syndication.CategorySyndicator |
templateContent | |
class | info.magnolia.module.rssaggregator.generator.PageSyndicator |
Properties:
feedGenerators | required Feed generators node. |
| required Generator name. |
| required Generator class:
|
Custom generators should extend the convenience base class
AbstractSyndFeedGenerator
. Subclasses need to implement the template methods loadFeedEntries()
and setFeedInfo(SyndFeed)
.
The feed syndication servlet writes an XML feed to the response.
The servlet is registered in /server/filters/servlets/FeedSyndicationServlet
.
Node name | Value |
---|---|
server | |
filters | |
context | |
.... | |
servlets | |
ClasspathSpoolServlet | |
.... | |
FeedSyndicationServlet | |
mappings | |
-rss-- | |
pattern | /rss/* |
parameters | |
class | info.magnolia.cms.filters.ServletDispatchingFilter |
comment | Responsible for RSS Feed syndication |
enabled | true |
servletClass | info.magnolia.module.rssaggregator.servlet.FeedSyndicationServlet |
servletName | FeedSyndicationServlet |
Properties:
FeedSyndicationServlet | required Feed syndication servlet node. |
| required FeedSyndicationServlet writes an XML feed to the response. Based on the request parameters, thefeedGenerators configuration is resolved and used to generate the XML feed. The content of the feed is written to the response with the appropriate character encoding. |
Syndication components use virtual URI mappings to redirect generated feeds. The mappings use regular expressions and are called by the feed generator classes to render appropriate content in the XML feed.
Mappings are configured in /modules/rssaggregator/virtualURIMappings
.
Node name | Value |
---|---|
modules | |
rssaggregator | |
virtualURIMapping | |
rssFeeds | |
class | info.magnolia.cms.beans.config.RegexpVirtualURIMapping |
fromURI | /rssFeeds/(.*) |
toURI | redirect:/rss/?feedPath=/$1 |
planetFeeds | |
class | info.magnolia.cms.beans.config.RegexpVirtualURIMapping |
fromURI | /planetFeeds/(.*) |
toURI | redirect:/rss/?feedPath=/$1&generatorName=planet |
categoryFeeds | |
class | info.magnolia.cms.beans.config.RegexpVirtualURIMapping |
fromURI | /categoryFeeds/([a-zA-Z0-9,-]*)/(.*) |
toURI | redirect:/rss/?generatorName=category&categories=$1&siteRoot=/$2 |
Properties:
virtualURIMapping | required Virtual URI mapping node |
| required Name of mapping. |
| required RegexpVirtualURIMapping allows you to specify a regular expression pattern that matches a sequence of characters. |
| required Pattern to match in the requested URI. |
| required Concrete URI where the request is mapped to. |
The anonymous role does not have permissions to the rss
workspace on the author or public instance. Public users cannot see the feed content by default.
The RSS Aggregator module installs the rss-aggregator-base
role that provides read permissions to the rss
workspace.
Workspace | Permission | Scope | Path |
Rss | Read only | Selected and sub nodes | / |
To provide public access to feed content, assign the rss-aggregator-base
role to the anonymous
systems user on the public instance.
By default only superuser
can access the Feeds app and work with feeds because the superuser
role includes read/write permissions to the rss
and config
workspaces.
Here's how to grant permissions for various feed tasks:.
rss-aggregator-base
role to give read only access to feed content. This allows the user to view the list of feeds in the app, select feeds in components and view feed content on pages. rss
workspace and assign it to the user. This allows the user to create new feeds in the Feeds app, but they cannot access the Configuration subapp.config
workspace to /modules/rssaggregator
to allow the user to schedule automated feed imports in the Configuration subapp.