Magnolia 5.3 reached end of life on June 30, 2017. This branch is no longer supported, see End-of-life policy.
The RSS Aggregator module displays external feed content on a Magnolia page (feed aggregation) and generates feeds from Magnolia content (feed syndication). Displaying aggregated feeds provides continuous fresh content on your site, which encourages regular crawling by search engines. RSS feeds can also add a sense of timeliness and community involvement to your site. Feed syndication increases your content exposure. This in turn generates more traffic and backlinks that improve your site rank.
Planet feeds enhance standard feeds by collecting feed statics. Planet Magnolia is an example site that uses the module to collect blog posts about Magnolia.
A feed aggregator displays external feed content on a Magnolia page. This provides continuous fresh content and encourages regular crawling by search engines.
In this example we collect three individual RSS feeds into one aggregate feed:
These have a common topic – search engine optimization (SEO) – so they are good candidates for aggregation. A reader interested in one feed is likely to be interested in the others too.
To create a RSS aggregator, in the Feeds app:
SEOBlogs
. This is the internal name of the aggregator.To import data for:
See also: Scheduling an automatic feed import.
To display the aggregated feed on the site:
/demo-features/aggregation-paragraphs/rss-aggregation
.content
area.Importing feed data manually is not a long-term solution. You should configure an automatic import schedule. Choose the update frequency depending on how often feeds have new content. For example, if a blog gets one post a day then importing once a day at 6 a.m. is enough.
You can schedule automatic imports for all feeds in the Configuration subapp:
0 0 6 * * *
imports data daily at 6 a.m. See Cron Maker for help.The input in the app is stored in property nodes in the module configuration in /modules/rssaggregator/config/automatedImport
and /cron
.
Node name | Value |
---|---|
modules | |
rssaggregator | |
config | |
fetcher | |
planetOptions | |
feedGenerators | |
automatedImport | true |
cron | 0 0 6 * * * |
Magnolia 5.3+ / RSS Aggregator 2.3+
You can configure automatic import settings for each feed in the Import Settings tab of the edit dialog:.
Feeds are imported by a fetcher class configured in /modules/rssaggregator/config/fetcher
.
Node name | Value |
---|---|
modules | |
rssaggregator | |
config | |
fetcher | |
class | info.magnolia.module.rssaggregator.importhandler.FastRSSFeedFetcher |
Properties:
class
:
FastRSSFeedFetcher
retrieves RSS feed content over HTTP. Fetches all feed channels defined in an aggregate feed.
You can use
SimpleRSSFeedFetcher
as an alternative to FastRSSFeedFetcher
. This simple, single-threaded fetcher reduces server load and is suitable when you don't have many feed channels and you don't import often.
Feed data is stored in the rss
workspace. Configure a custom JCR browser to view the content in the repository. You can download the feedsJCRBrowser configuration XML files if you prefer not to configure from scratch.
Here's the SEOBlogs example data in the JCR.
Node name | Value |
---|---|
SEOBlogs | |
data | |
channel-00 | |
entry-0 | |
categories | |
author | Marieke van de Rakt |
channelTitle | Yoast |
content | <p><img class="nopadding alignright wp-image-276935 size-medium" src="https://cdn.yoast.com/wp-content/uploads/2014/12/... |
description | <p>A few weeks ago, I wrote a post about obtaining an attractive writing style. I gave some practical tips to make your blogs ... |
link | https://yoast.com/writing-blog-creating-clear-blog-post-structure/ |
pubDate | 1,418,224,075,000 |
title | Writing a blog: creating a clear blog post structure |
entry-1 | |
entry-2 | |
.... | |
description | The Art & Science of Website Optimization |
link | https://yoast.com |
rss | http://feeds.yoast.com/yoast |
title | Yoast |
type | rss_2.0 |
channel-02 | |
channel-01 | |
automatedImport | true |
cron | 0 0 8 * * * |
name | SEOBlogs |
overrideDefault | true |
title | SEO Blogs |
Structure:
<RSS aggregator name>
data
: Data that was retrieved from the Internet. channel-<number>
: A channel is created for each RSS feed.entry-<number>
: One entry in the feed such as one blog post.author
channelTitle
content
description
: Feed properties retrieved from the internet.link
automatedImport
: Internal feed properties.cron
:Content will only display in the feed components after it has been imported. To display feed content on the public instance publish the feed in the Feeds app. This will also publish the feed data.
There are two standard feed components:
STKCombinedFeed
: Combines all feeds in a channel and renders them sequentially. You can set the number, sort order and character limit of the entries.STKSingleFeed
: Displays a defined number of entries for each feed in the channel. The external feed title is used if no internal title is set. You can set the number, sort order and character limit of the entries.For non-STK users, the RSS Aggregator module provides equivalent components, CombinedFeedParagraoh
and feedListParagraph
. The two sets of components are essentially identical.
The components definitions are configured in STK > Template Definitions /components/teasers/stkCombinedFeed
and /stkSingleFeed
.
Here's the stkCombinedFeed
definition:
Node name | Value |
---|---|
components | |
teasers | |
stkCombinedFeed | |
parameters | |
description | rssaggregator.paragraph.combined.description |
dialog | rssaggregator:combinedFeed |
i18nBasename | info.magnolia.module.rssaggregator.messages |
modelClass | info.magnolia.module.rssaggregator.templates.components.CombinedFeedModel |
renderType | stk |
templateScript | /templating-kit/components/teasers/combinedFeed.ftl |
title | rssaggregator.paragraph.combined.title |
The model class and template script are the important properties that determine content in the different components. All RSS model classes extend
AbstractFeedModel
that provides the business logic to retrieve a defined feed and it's data from the rss
workspace and supports:
Entry sorting by title or publication data.
Ascending and descending entry sorting.
Maximum results property. Default is 20.
Search capabilities
Any custom model class should extend AbstractFeedModel
.
Planet feeds have all the features of standard RSS feeds but also store additional data that is used to create feed statistics. For example, a planet feed will tell you the number of posts by an author. You can use this information in a Planet Statistics component.
To create a planet feed check the Planet Feed box in the feeds dialog when creating or editing a feed. Feeds can be marked as a planet feed at any time. When you change a standard feed to a planet feed, re-import the feed data before generating the additional planet data (described below).
The module includes two custom commands that generate planet data. These are configured in /modules/rssaggregator/commands/planet/generatePlanetData
and /collectPlanetStatistics
.
PlanetDataGenerator command is executed when planet data needs to be updated. Here's what the command does:
author
, channelTitle
, title
, content
and description
properties. Posts without these properties are not stored in the archiveCollectStatisticsCommand command is executed when planet statistics need to be updated. Here's what the command does:
Node name | Value |
---|---|
modules | |
rssaggregator | |
commands | |
planet | |
generatePlanetData | |
class | info.magnolia.module.rssaggregator.generator.PlanetDataGenerator |
collectPlanetStatistics | |
class | info.magnolia.module.rssaggregator.generator.CollectStatisticsCommand |
Use the Scheduler module to schedule planet updates. The update schedules are stored as a standard scheduler job in /modules/scheduler/config/jobs/generatePlanetData
and /collectPlanetStatistics
. The jobs execute the planet commands that do the actual work.
To schedule a planet update:
active
property to true
.0 0 6 * * *
imports data daily at 6 a.m. See Cron Maker for help. There is no way to manually generate the planet data. You can use the pattern
0 0/5 * 1/1 * ? *
to generate the data five minutes after changing the settings.
Node name | Value |
---|---|
modules | |
scheduler | |
config | |
jobs | |
generatePlanetData | |
params | |
repository | rss |
active | true |
catalog | planet |
command | generatePlanetData |
cron | 0 0/10 * 1/1 * ? * |
description | generate data for RSS planet |
collectPlanetStatistics | |
params | |
repository | rss |
active | true |
catalog | planet |
command | collectPlanetStatistics |
cron | 0 0/18 * 1/1 * ? * |
description | generate statistics for RSS planet |
To display planet components on a site:
Make the components available in any area. You can make them available in a specific page template or globally for all pages in the template prototype. The planet components are not available by default.
In the Pages app edit a page where the components are available.
Add the components to the page.
Here are the planet components using our SEOBlogs example data.
Planet Feed: Displays feed posts, including images. Editors can define the length of the text, and pagination is available.
Planet Statistics: Shows a list of authors with statistics (number of posts).
Planet data is used to generate content for planet components. The data is stored in the rss
workspace. View in the data in your custom JCR browser.
Here's the SEOBlogs example after marking it as a planet feed, re-importing the data, and running the planet commands.
Node name | Value |
---|---|
SEOBlogs | |
data | |
planetData | |
posts-00000 | |
entry-1 | |
author | Barry Schwartz |
authorLink | http://searchengineland.com |
channelTitle | Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing |
checksum1 | b703958ac530e66da288f0801965e15b |
checksum2 | 391df9cc772ecd286f92e2ea66cccde6 |
description | Google takes Google Now to the next level but integrating your home appliances directly into voice search and predictive search behavior.. |
hidden | false |
link | http://feeds.searchengineland.com/~r/searchengineland/~3/93zppBI3FaY/nest-google-now-googles-now-predicting-home-run-211006 |
pubDate | 1,418,738,867,000 |
rssLink | http://feeds.searchengineland.com/searchengineland?format=xml |
title | Nest + Google Now = Google’s Now Predicting How Your Home Should Run |
entry-2 | |
entry-3 | |
.... |
Properties:
<aggregator name>
planetData
: The GeneratePlanetData
command reads feed data and stores it here.posts-<number>
: All posts from all feeds. First all entries from the first feed, newest entry first. Then all posts from the second feed, and so on.entry-<number>
: One entry in the feed such as one blog post.author
authorLink
channelTitle
checksum1/2
: The module uses checksums to handle duplicate entries. Because feed data is deleted and recreated on every run, there is a high probability that a subsequent run will include entries that were contained in a previous run. Some entries may only have changed slightly, for example a different publication date. To avoid duplicates, the PlanetDataGenerator
command uses checksums for each entry. Two checksum
properties are generated. If an archive node with one of the checksums exists, no data is stored for the new item and an INFO level entry is written in the logs.description
hidden
: Set to true
to hide the entry from the planet feed. Useful for hiding spam entries.link
pubDate
rssLink
title
Planet statistics are generated from planet data and stored in the rss
workspace. View the data in your custom JCR browser.
Here's the data for the SEOBlogs example.
Node name | Value |
---|---|
SEOBlogs | |
data | |
planetData | |
planetStatistics | |
authors | |
author-0 | |
author-1 | |
author-2 | |
countedPosts | |
5ff40975-00e4-4004-8ee3-980ee4a5d6af | |
a8d760ca-f42c-48fe-8962-97922eae0242 | |
cbeef72c-74f4-4c29-8a12-57b64b94c96c | |
author | Greg Sterling |
blogLink | http://searchengineland.com |
feedLink | http://feeds.searchengineland.com/searchengineland?format=xml |
postCount | 3 |
author-3 | |
.... |
Properties:
<aggregator name>
statistics
: The CollectPlanetStatistics
command extracts statistics from the /planetData
node. This node is deleted and recreated on every run of the command.authors
: All authors from all feeds.author-<number>
: Each author is allocated a number. counted-posts
: Each child node is a reference to a post in the feed.<post UUID>
author
blogLink
feedLink
postCount
: Number of posts by this author in the aggregate feed.Planet data is generated and stored for the last 3 months by default. You can configure the time period for which the data is retained in /modules/rssaggregator/config/planetOptions/lastMonthsIncluded
.
Node name | Value |
---|---|
modules | |
rssaggregator | |
config | |
fetcher | |
planetOptions | |
lastMonthsIncluded | 3 |
Properties:
planetOptions
lastMonthsIncluded
: Number of months. Default is 3.Planet component definitions are configured in /modules/rssaggregator/templates/components/
.
The Planet Feed component is an enhanced version of the STK Combined Feed component and can only be used with planet feeds.
The component:
The component definition is in /modules/rssaggregator/templates/components/planetFeeds
.
Node name | Value |
---|---|
modules | |
rssaggregator | |
templates | |
components | |
planetFeeds | |
description | rssaggregator.component.planet.description |
dialog | rssaggregator:planetFeeds |
i18nBasename | info.magnolia.module.rssaggregator.messages |
modelClass | info.magnolia.module.rssaggregator.templates.components.PlanetFeedModel |
renderType | stk |
templateScript | /rssaggregator/components/planetFeed.ftl |
title | rssaggregator.component.planet.title |
The Planet Statistics component uses the planet statistics data. The component:
The component definition is in /modules/rssaggregator/templates/components/feedStatistics
.
Node name | Value |
---|---|
modules | |
rssaggregator | |
templates | |
components | |
feedStatistics | |
description | rssaggregator.component.statistics.description |
dialog | rssaggregator:feedStatistics |
i18nBasename | info.magnolia.module.rssaggregator.messages |
modelClass | info.magnolia.module.rssaggregator.templates.components.FeedStatisticsModel |
renderType | stk |
templateScript | /rssaggregator/components/feedStatistics.ftl |
title | rssaggregator.component.statistics.title |
The Planet Authors component allows users to subscribe directly to the external feeds on a Magnolia page. The links take the user to the external sites.
The component definition is in /modules/rssaggregator/templates/components/feedSubscriptions
.
Node name | Value |
---|---|
modules | |
rssaggregator | |
templates | |
components | |
feedSubscriptions | |
description | rssaggregator.component.subscriptions.description |
dialog | rssaggregator:feedSubscriptions |
i18nBasename | info.magnolia.module.rssaggregator.messages |
modelClass | info.magnolia.module.rssaggregator.templates.components.FeedSubscriptionsModel |
renderType | stk |
templateScript | /rssaggregator/components/feedSubscriptions.ftl |
title | rssaggregator.component.subscriptions.title |
Feed syndication increases your content exposure. This in turn generates more traffic and backlinks that improve your site rank.
Feed generators generate RSS feeds from Magnolia content and imported content stored in the rss
workspace.
Four feed generators are registered in /modules/rssaggregator/config/feedGenerators
.
rss
:
RSSModuleFeedGenerator
generates a SyndFeed
based on standard aggregation feeds.
planet
:
PlanetFeedGenerator
generates a SyndFeed
based on planet aggregation feeds.
category
:
CategorySyndicator
is registered by the Categorization module. It generates a SyndFeed
based on articles tagged with categories.
templateContent
:
PageSyndicator
is registered by the Standard Templating Kit module. It generates a SyndFeed
based on template categories and subcategories.
Node name | Value |
---|---|
modules | |
rssaggregator | |
config | |
feedGenerators | |
rss | |
class | info.magnolia.module.rssaggregator.generator.RSSModuleFeedGenerator |
planet | |
class | info.magnolia.module.rssaggregator.generator.PlanetFeedGenerator |
category | |
class | info.magnolia.module.categorization.syndication.CategorySyndicator |
templateContent | |
class | info.magnolia.module.templatingkit.syndication.PageSyndicator |
Custom generators should extend the convenience base class
AbstractSyndFeedGenerator
. Subclasses need to implement the template methods loadFeedEntries()
and setFeedInfo(SyndFeed)
.
FeedSyndicationServlet
writes an XML feed to the response. Based on the request parameters, the feedGenerators
configuration (above) is resolved and used to generate the XML feed. The content of the feed is written to the response with the appropriate character encoding.
The servlet is registered in /server/filters/servlets/FeedSyndicationServlet
.
Node name | Value |
---|---|
server | |
filters | |
context | |
.... | |
servlets | |
ClasspathSpoolServlet | |
.... | |
FeedSyndicationServlet | |
mappings | |
-rss-- | |
pattern | /rss/* |
parameters | |
class | info.magnolia.cms.filters.ServletDispatchingFilter |
comment | Responsible for RSS Feed syndication |
enabled | true |
servletClass | info.magnolia.module.rssaggregator.servlet.FeedSyndicationServlet |
servletName | FeedSyndicationServlet |
The syndication components use virtual URI mappings to redirect the generated feeds. The mappings use regular expressions and are called by the feed generator classes to render appropriate content in the XML feed.
Three mappings are configured in the RSS Aggregator module in /modules/rssaggregator/virtualURIMappings
.
Node name | Value |
---|---|
modules | |
rssaggregator | |
virtualURIMapping | |
rssFeeds | |
class | info.magnolia.cms.beans.config.RegexpVirtualURIMapping |
fromURI | /rssFeeds/(.*) |
toURI | redirect:/rss/?feedPath=/$1 |
planetFeeds | |
class | info.magnolia.cms.beans.config.RegexpVirtualURIMapping |
fromURI | /planetFeeds/(.*) |
toURI | redirect:/rss/?feedPath=/$1&generatorName=planet |
categoryFeeds | |
class | info.magnolia.cms.beans.config.RegexpVirtualURIMapping |
fromURI | /categoryFeeds/([a-zA-Z0-9,-]*)/(.*) |
toURI | redirect:/rss/?generatorName=category&categories=$1&siteRoot=/$2 |
Standard Templating Kit module includes the contentFeeds
mapping in
./modules/standard-templating-kit/virtualURIMapping/contentFeeds
Node name | Value |
---|---|
modules | |
standard-templating-kit | |
virtualURIMapping | |
contentFeeds | |
class | info.magnolia.cms.beans.config.RegexpVirtualURIMapping |
fromURI | /contentFeeds/([a-zA-Z0-9]*)/([a-zA-Z0-9]*)/(.*) |
toURI | redirect:/rss/?generatorName=templateContent&category=$1&subCategory=$2&siteRoot=/$3 |
Three modules provide syndication components: Standand Templating Kit, Categorization and RSS Aggregator. All components rely of the functionality of the RSS Aggregator module.
The STK module includes the stkExtrasContentTypeRSSFeed
component. The component renders an RSS subscription icon on the page. Editors can define feeds that aggregate pages based on the Article, News or Events templates, and select a parent page.
Here's the component definition is in STK > Template Definitions /components/extras/stkContentTypeRSSFeed
:
Node name | Value |
---|---|
components | |
extras | |
stkExtrasContentTypeRSSFeed | |
parameters | |
description | paragraphs.extras.stkExtrasContentTypeRSSFeed.description |
dialog | standard-templating-kit:components/extras/stkExtrasContentTypeSyndicate |
i18nBasename | info.magnolia.module.templatingkit.messages |
modelClass | info.magnolia.module.templatingkit.templates.components.ContentTypeSyndicateModel |
renderType | stk |
templateScript | /templating-kit/components/teasers/syndicate.ftl |
title | paragraphs.extras.stkExtrasContentTypeRSSFeed.title |
Properties:
modelClass
:
ContentTypeSyndicateModel
creates a STK renderable definition and returns the appropriate content.
templateScript
: syndicate.ftl
(GIT) renders the RSS icon in the component and itemLinks
provided by the model class.
The contentFeeds
URI mapping calls the templateContent
feed generator class (
PageSyndicator
) that generates the XML feed for content based on templates with a category
property equal to content
, and a subCategory
property equal to the selection in the dialog.
Here's the generated feed for all article pages in the demo-project/about/subsection-articles
section.
The Categorization module includes the categoryRSSFeed
component. The component renders an RSS subscription icon on the page. The generated feed includes all pages tagged with specified categories. Editors can define the categories and root page for the feed.
The component definition is in /modules/categorization/templates/components/categoryRSSFeed
.
Node name | Value |
---|---|
modules | |
categorization | |
templates | |
components | |
categoryRSSFeed | |
parameters | |
description | paragraphs.categoryRSSFeed.description |
dialog | categorization:categoryRSSFeed |
i18nBasename | info.magnolia.module.categorization.messages |
modelClass | info.magnolia.module.categorization.model.CategorySyndicateModel |
renderType | stk |
templateScript | /categorization/paragraphs/syndicate.ftl |
title | paragraphs.categoryRSSFeed.title |
Properties:
modelClass
:
CategorySyndicateModel
model class provides the business logic to select the relevant entries for the feed.
templateScript
: syndicate.ftl
(Git) renders the RSS icon in the component and itemLinks
provided by the model class.
The categoryFeeds
URI mapping calls the category
feed generator class (
CategorySyndicator
) that generates the XML feed for content based on the categories and root page selected in the dialog. The URL for the generated feed of content in the demo-project/about
section, tagged with the family
category, is similar to http://localhost:8080/magnoliaAuthor/rss/?generatorName=category&categories=ab9437db-ab2c-4df5-bb41-87e55409e8e1&siteRoot=/demo-project/about/subsection-articles
. Compare the feed URL to the categoryFeeds
Virtual URI mapping. The long number sequence is the UUID of the family
category set up in Categories app.
The RSS Aggregator module includes the planet feedSyndication
component that renders subscription Atom and RSS icons and links. Editors can add a title and select a planet feed.
The component definition is in /modules/rssaggregator/templates/components/feedSyndication
.
Node name | Value |
---|---|
modules | |
rssaggregator | |
templates | |
components | |
feedSyndication | |
description | rssaggregator.component.syndication.description |
dialog | rssaggregator:feedSyndication |
i18nBasename | info.magnolia.module.rssaggregator.messages |
renderType | stk |
templateScript | /rssaggregator/components/feedSyndication.ftl |
title | rssaggregator.component.syndication.title |
Properties:
templateScript
: The feedSyndication.ftl
(Git) script renders the icons and links, and the
PlanetFeedGenerator
generator class renders the feed.The anonymous role does not have permissions to the rss
workspace on the author or public instance. Public users cannot see the feed component content by default. You can check this by logging out of the public instance.
The RSS Aggregator module installs the rss-aggregator-base
role that provides read permissions to the rss
workspace.
Workspace | Permission | Scope | Path |
---|---|---|---|
Rss | Read only | Selected and sub nodes | / |
To provide public access to the feed content, assign the rss-aggregator-base
role to the anonymous
systems user on the public instance.
Now when you log out on the page, the feed content displays.
By default, all users have app launcher access to the Feeds app, but only superuser
can work with feeds. The superuser
role includes the following relevant permissions.
Workspace | Permission | Scope | Path |
---|---|---|---|
Rss | Read/Write | Selected and sub nodes | / |
Config | Read/Write | Selected and sub nodes | / |
These permissions allow superuser
to:
You can provide editors read only permissions to feed content by assigning the rss-aggregator-base
role to the editor or group. This will allow them to:
To provide editors write permissions to the Feeds app, create a new role with the following permissions and assign it to the editor or group.
Workspace | Permission | Scope | Path |
---|---|---|---|
Rss | Read/Write | Selected and sub nodes | / |
This will allow them to:
Users with write permissions to the rss
workspace cannot view content in the Configuration subapp . The
RSSAggregatorApp
app class requires users to have write permissions to the rssaggregator
module in the config
workspace.
To provide access to the Configuration subapp, create a new role with the following permissions and assign it to the user or group.
Workspace | Permission | Scope | Path |
---|---|---|---|
Config | Read/Write | Selected and sub nodes | /modules/rssaggregator |