Knowledge Base / Appnel Solutions 

Feeds.App 2 Manual

This documentation is for version 2.01 of the plugin.

Feeds.App is a plugin for publishing syndication feed content into Movable Type templates.

REQUIREMENTS

Feeds.App requires Movable Type 3.0D and the following modules:

• XML::Parser (version >= 2)
• Class::XPath (version >= 1.41)[1]
• XML::Elemental (version >= 1.01)
• XML::RAI (version >= 1.01)
• XML::Atom::Syndication (version >= 0.091)
• Storable (version >= 2)
• Date::Parse (version >= 2.6)
• Getopt::Long (version >= 2.33

All of these modules are included with the Feeds.App distribution except XML::Parser and Storable which require compilation. These two modules can be obtained from CPAN.

[1] NOTE: Because of a bug in Class::XPath that has yet to be posted to CPAN, only use the include version of the module. See "CAVEATS AND KNOWN BUGS" for more.

INSTALLATION

• 1 Download and unpack the latest Feeds.App distribution file.
• 2 Move the files into place.

  Where mt is Movable Type application directory, transfer the contents of the following distribution directories to

   extlib -> mt/extlib 
   plugins -> mt/plugins
   tmpl ->mt/tmpl
   docs -> path/to/mt-static/docs**

  ** - You may need to create the docs directory in your mt-static directory.

  DO NOT replace these directories in MT as they are likely to contain other files that your system and other plugins need.

• 3 Set permissions

  Once these files are in place, go into the plugins/feeds directory and set the permissions of all of the CGI scripts (files whose names end in .cgi) and the agent file to 755.

  Permissions set to 755 means that all users on the system have Read and Execute permissions; only the owner should have Write permissions.

  If you are setting permissions through a Unix shell, simply use the command

   $ chmod 755 *.cgi agent

• 4 Setup the cache directory

  Create a cache directory for your syndication feeds. This directory will be used to store copies of syndication data that has been retrieved from its source. Caching improves the speed of page rebuilds using syndication feeds and reduces the amount of bandwidth consumed. Similar to your weblog directories, if you are not running Movable Type under cgiwrap or suexec, set the permissions of the directories you just created to 777 (see the MT installation documentation for more info).

  Remember the full path name of this directory for later in the installation.

INITIALIZATION

• 1 Sign in to MT

  With your web browser sign in to the Movable Type application. If properly installed, you will see Feeds.App listed on the main menu page under the heading Configure Active Plugins.

• 2 Configure Feeds.App

  Click on the Feeds.App link to launch the configuration application. Modify the default configuration as appropriate. The following configuration settings are available:

  • Cache Directory.

  The is the full path name of the directory that will be used to write local copies of syndication feeds. If you are using a file based storage system the field will default to the path defined by DataSource in mt.cfg configuration file.

  • Background Processing.

  The plugin can be configured to rebuild with cached information only and allow for the feed agent to be run as another process (most likely as a regular cron job) in the background. This dramatically reduces the delay of building templates with syndication data by making the most of the cache. This is particularly helpful if an extensive number of feeds are being rendered. The default is to use background processing.

• 3 Click Save Configuration.

  Your plugin is now configured for use.

BACKGROUND PROCESSING WITH THE AGENT

If you have the ability to schedule execution of scripts on your server, you should setup the Feeds.App agent to run periodically and refresh the cache with the latest syndication feed information you will need to generate you pages.

For example, if you have rights to create cron job on a unix system you would use the following from the Unix shell:

 $ crontab -e

This will open a file with your existing cron jobs (it may be blank if this is your first). To run the agent every 30 minutes for the feeds contained in a file called mysubs.txt you would add a line something like this and save it.

 */30 * * * * /path/to/mt/plugins/feed/agent --blog_id=2
--tmpl=Feeds path/to/data/mysubs.txt

mysubs.txt is assumed to be of type list -- one syndication feed URL per line. After checking the feeds and updating the cache, the agent script will rebuild the index template named Feeds in the blog with an id of 2.

Here is an example of using the agent script to build a template using just the ID and a list in Comma Seperate Value (CSV) format.

/path/to/mt/plugins/feed/agent --tmpl_id=40 -c path/to/data/mysubs.csv

The agent script has more options that are defined below.

     --help     -?  The help screen
     --version      The version number of the script
 
     MODE
     --silent   -s  No status messages
     --quiet    -q  Minmal status messages (Default)
     --verbose  -v  extensive status messages and tracer report
 
     TYPE
     --list     -l  Use the list reader on the collection(s)
     --opml     -o  Use the OPML reader on the collection(s)
     --csv      -c  Use the CSV reader on the collection(s)
     --type         A argument for stating type by key
 
     --tmpl_id      The template ID to rebuild after caching
     --tmpl         The template name to rebuild after caching
     --blog_id      The blog ID the named template (tmpl) is in

Save the file and quit. The agent will now run in the background at half-past and on the hour.

INTEGRATING SYNDICATION DATA IN TEMPLATES

The Basics

The following bit of MT template markup will insert the title of the feed, wrapped in h3 tags, and output an unordered list of hyperlinked entry titles.

<$MTFeedsInclude uri=http://www.timaoutloud.org/xml/index.rdf lastn=5$>

Yes, that's it. This tag with these parameters essential create the following layout and build it.

     <MTFeeds uri=http://www.timaoutloud.org/xml/index.rdf>
     <h3><$MTFeedTitle$></h3>
     <ul><MTFeedEntries lastn=5>
     <li><a href=<$MTFeedEntryLink$>><$MTFeedEntryTitle$></a></li>
     </MTFeedEntries></ul>
     <$MTFeedVersionSlug$>
     </MTFeeds>

Multiple Feeds

Assume we have created a file, mysubs.txt, with a list of syndication feed URLs like so:

http://www.timaoutloud.org/xml/atom.xml http://www.movabletype.org/index.xml http://www.sixapart.com/log/index.rdf http://sixapart.com/corner/index.rdf http://www.sixapart.com/pronet/index.rdf

To generate the last 5 entries from these feeds into a template insert this bit of markup.

 <$MTFeedsInclude collection=mysubs.txt lastn=5$>

This tag essential ouput content with the following layout:

 <MTFeeds collection=mysubs.txt type=list>
 <h3><$MTFeedTitle$></h3>
 <ul><MTFeedEntries lastn=5>
 <li><a href=<$MTFeedEntryLink$>><$MTFeedEntryTitle$></a></li>
 </MTFeedEntries></ul>
 <$MTFeedVersionSlug$>
 </MTFeeds>

NOTE: type=list is the default if not specified. Other valid types include opml and csv. See the template tag documentation for more.

Here we output the title of the feed in h3 tags and a hyperlinked unordered list of the last 5 entry titles for each feed listed in mysubs.txt.

Latest Feeds

Syndication feeds are updated at differing frequencies and intervals. If the intent of your page is to track or display the latest news or posts across a collection of feeds, the previous example will not do. The Feeds.App plugin can accomadate a last entries list with the following bit of markup.

 <$MTFeedsIncludeLatest collection=mysubs.txt lastn=25
title=Latest Posts$>

Which produces output like this template:

 <MTFeedsLatest collection=mysubs.txt type=list>
 <h3>Latest Posts</h3>
 <ul><MTFeedEntries lastn=25>
 <li><a href=<$MTFeedEntryLink$>><$MTFeedEntryTitle$></a> (<$MTFeedEntrySourceName$>)</li>
 </MTFeedEntries></ul>
 </MTFeedsLatest>

Here we use a static title of Latest Posts and output the latest 25 entries from all of the feeds listed in mysubs.txt as a single unordered list of hyperlinked titles along with their source feed title in parenthesis.

These examples are only scratching the surface of the many ways and means that syndicated data can be used. See the plugin tag documentation for more options.

TEMPLATE TAGS

The Feeds.App plugin makes the following template tags available.

• MTFeeds uri=
• MTFeeds collection= [type=]

A container tag representing one or more syndication feeds.

Either the uri or collection argument must be specified.

When inserting one feed, use the uri argument to specify the location of the feed. Supported protocols are http://, https:// and file://.

When inserting multiple feeds, use the collection argument to specify the path of a file relative to the blogs root directory. The type argument is optional and specifies the collection source type.

The list type reader will process a plain text file containing a URI on each line. This is the default when no type is specified.

The opml type reader uses an XML parser to find all <outline> tags and extracts URIs from either the xmlUrl or xmlurl attributes. Outline tags with either of these are skipped.

The csv type reader will process a plain text file of comma seperated values extracting URIs.The first row in these files are assumed to be field names. Links are extracted from the 'link' or 'url' field in each row.

• MTFeedLatest collection= [type=]

Creates a virtual feed of the latest entries from the specified collection. This tag is an alternate to the Feeds container.

The collection argument is the path of a file relative to the blogs root directory.

The type argument is optional and specifies the collection source type. The default is 'list'.

• MTFeedTitle

The title of the syndication feed.

• MTFeedLink [rel= any=0|1]

The URI of the syndication feed. The optional rel attribute specifies a certain link relationship as found in Atom feeds. The default is alternate. This attribute has no real effect on RSS feeds. The optional any attribute takes a boolean value. If set to true (1) the tag will return the first link of any relationship. The default is (0) false.

• MTFeedCreatorName

The name of the syndication feed.

• MTFeedCreatorEmail

The email address of the syndication feed.

• MTFeedCreatorURL

The URL of the syndication feed.

• MTFeedDescription [type=]

An abstract or tagline of the syndication feed.

Using the optional argument of type will only return content if a description/abstract is available with the specified MIME type.

• MTFeedRights

The rights or license of the syndication feed's content.

• MTFeedLanguage

The language of the syndication feed's content.

• MTFeedIdentifier

A unique identifier for the feed.

• MTFeedContributors

A container tag that loops through each contributor to the syndication feed (if any) and creates the necessary context.

• MTFeedContributorName

The name of a contributor to the syndication feed. Must be used within a MTFeedContributors tag context.

• MTFeedContributorEmail

The email address of a contributor to a syndication feed. Must be used within a MTFeedContributors tag context.

• MTFeedContributorLink

The URL of a contributor to a syndication feed. Must be a used within a MTFeedContributors tag context.

• MTFeedModified [format=]

The last modified timestamp of the syndication feed.

The optional format argument can be used to format the timestamp using the same specifiers as other MT timestamp tags such as MTDate. The default is W3C Date Time Format (W3CDTF aka ISO 8601) using UTC (Coordinated Universal Time).

• MTFeedImageURL

The image URL associated to the syndication feed.

• MTFeedImageHeight

The height of the image associated to the syndication feed.

• MTFeedImageWidth

The width of the image associated to the syndication feed.

• MTFeedEntries [offset= lastn=]

A container tag that will loop through the entries of the feed in context. Like MTEntries this tag supports optional arguments of limit and offset to control the number entries and starting point to be insert. Entries are sorted in reverse chronological order.

• MTFeedEntryCount

The total number of entries in the syndication feed.

• MTFeedEntryTitle

The title of the syndication feed entry in context.

• MTFeedEntryLink [rel= any=0|1]

The hyperlink to the syndication feed entry in context. The optional rel attribute specifies a certain link relationship as found in Atom feeds. The default is alternate. This attribute has no real effect on RSS feeds. The optional any attribute takes a boolean value. If set to true (1) the tag will return the first link of any relationship. The default is (0) false.

• MTFeedEntryContributors

A container tag that loops through each contributor to the syndication feed entry (if any) and creates the necessary context.

• MTFeedEntryContributorName

The name of a contributor to the syndication feed entry. Must be used within a MTFeedEntryContributors tag context.

• MTFeedEntryContributorEmail

The email address of a contributor to a syndication feed entry. Must be used within a MTFeedEntryContributors tag context.

• MTFeedEntryContributorLink

The URL of a contributor to a syndication feed entry. Must be a used within a MTFeedEntryContributors tag context.

• MTFeedEntryDescription [type=]

An abstract or summary of the syndication feed entry.

Using the optional argument of type will only return content if a description/abstract is available with the specified MIME type.

Due to the varying nature of RSS format feeds this may not return what you expect. This tag will only return content if a description tag exists in the syndication feed along with a full content tag such as a XHTML body or content:encoded tagset.

• MTFeedEntryLanguage

The language of the syndication feed entry.

• MTFeedEntryIdentifier

A unique identifier to the syndication feed entry.

• MTFeedEntryPing

The Trackback URL to ping the syndication feed entry with.

• MTFeedEntryPinged

A container tag that loops through each Trackback URL pinged by the syndication feed entry.

• MTFeedEntryPingedURL

The URL of the resource that was pinged by this syndication feed entry. Must be used within a MTFeedEntryPinged tag context.

• MTFeedEntrySubjects

A container tag that loops through each subject the syndication feed entry is associated with.

• MTFeedEntrySubjectLabel

A subject label the syndication feed entry has been associated with. Must be used within the context of a MTFeedEntrySubjects tag context.

• MTFeedEntryContent [type=]

The content of the syndication feed entry.

Using the optional argument of type will only return content if it matches the specified MIME type.

• MTFeedEntryModified [format=]

The last modified timestamp of the syndication feed entry.

The optional format argument can be used to format the timestamp using the same specifiers as other MT timestamp tags such as MTDate. The default is W3C Date Time Format (W3CDTF aka ISO 8601) using UTC (Coordinated Universal Time).

• MTFeedEntryIssued [format=]

The issued timestamp of the syndication feed.

The optional format argument can be used to format the timestamp using the same specifiers as other MT timestamp tags such as MTDate. The default is W3C Date Time Format (W3CDTF aka ISO 8601) using UTC (Coordinated Universal Time).

• MTFeedEntryCreated [format=]

The created on timestamp of the syndication feed.

The optional format argument can be used to format the timestamp using the same specifiers as other MT timestamp tags such as MTDate. The default is W3C Date Time Format (W3CDTF aka ISO 8601) using UTC (Coordinated Universal Time).

• MTFeedEntrySourceName

The name (title) of the feed that the entry to pulled from. This tag on returns a value when you in a latest feed.

• MTFeedEntrySourceLink

The primary link of the feed that the entry to pulled from. This tag on returns a value when you in a latest feed.

• MTFeedIf name= [type=]

A simple and generic container tag for inserting conditional logic into templates depending upon on the content of the syndication feed.

The required name argument is the plugin tag (sans the MTFeed prefix -- MTFeedLink is 'Link' and so on) whose content will be tested. If the associated tag has content the contents of MTFeedIf will be inserted. Otherwise it will be skipped unless the standard MTElse tagset is used to render a different result.

An optional argument of type can also be used to specify a specific MIME type for the condition. This is only useful when working with content tags such as MTFeedEntryContent and MTFeedEntryDescription.

• MTFeedsVersion

The version number of the Feeds.App plugin.

• MTFeedsVersionSlug

Returns Gathered by Feeds.App with the version number.

• MTFeedsInclude uri= [lastn= description=none|tooltip|paragraph]
• MTFeedsInclude collection= [type= lastn= description=none|tooltip|paragraph]

A tag to insert syndication content with a generic unordered list layout.

• MTFeedsIncludeLatest collection= [lastn= description=none|tooltip|paragraph title=]

A tag to insert a latest syndication content with a generic unordered list layout.

USAGE WITH A PROXY SERVER

The Feeds.App agent will use the standard MT configuration parameters of PingTimeout, PingProxy and PingNoProxy if they have been set.

If you are running mt in an environment that requires the use of a proxy server to connect outside of your network, these parameter must be set. Chances are you may have already set them if you are using outbound Trackbacks.

SUPPORT AND MORE

Users of Feeds.App are encouraged to join the mt-feeds-announce group at http://groups.yahoo.com/group/mt-feeds-announce/. This is a low-volume mailing list that will let you keep up with the latest news and releases of Feeds.App and related activites.

Technical support is not provided for the software. You may find the mt-feeds-discuss user-to-user mailing list and archive at http://groups.yahoo.com/group/mt-feeds-discuss/ helpful in addressing any issues.

TO DO

• API Clean-up. Lots of refactoring.
• Enhanced tracer reports and management.
• Configurable and additional CacheMgr drivers
• Integrate XML::Atom
• Add callback hooks.

CAVEATS AND KNOWN BUGS

Class::XPath 1.4 chokes on feeds that use in their identifier string which is most a URI uses this character. A patch has been submitted to the author of the module, but it has not been posted for some time. In the meanwhile, I am including my patched version 1.41.

There is a known bug effecting MT 3.0D using SQLite. It was fixed by Six Apart with the release of 3.1.

Atom feeds using relative links are not supported yet.

Error handling needs to be refined through out. For instance MT::CacheMgr-expire()> does not pass success or failure indicator back.

Using the optional argument of type will only return content if a description/abstract is available with the specified MIME type.