vB5 ATOM to RSS filter.

ven0m · Oct 4, 2016

Heisenberg submitted a new resource:

You must be registered for see links

- ATOM to RSS filter.

vB Version: 5.x.x

Part 1 of 3.

While this topic is specific to vBulletin, due to the fact that it affects RSS aggregators in general, I thought it would be good information to re-post here.

I've been pulling hair out for some time now trying to figure out why sometimes my RSS aggregator works fine and other times is crashes.
I finally pulled all of the logs and sorted out each and every RSS feed request until I came up with a list of broken...

You must be registered for see links

ven0m · Oct 4, 2016

Part 2 of 3.

Code:

<?php if (!defined('VB_ENTRY')) die('Access denied.');
/*========================================================================*\
|| ###################################################################### ||
|| # vBulletin 5.2.1# ||
|| # ------------------------------------------------------------------ # ||
|| # Copyright 2000-2016 vBulletin Solutions Inc. All Rights Reserved.  # ||
|| # This file may not be redistributed in whole or significant part.   # ||
|| # ----------------- VBULLETIN IS NOT FREE SOFTWARE ----------------- # ||
|| # http://www.vbulletin.com | http://www.vbulletin.com/license.html   # ||
|| ###################################################################### ||
\*========================================================================*/

class vB_External_Export_Rss extends vB_External_Export
{
    // RSS information needed to fill format fields
    protected $rssinfo = array();

    // default language information
    protected $defaultLang = array();

    // assertor
    protected $assertor;

    protected function __construct()
    {
        parent::__construct();
        $this->loadDefLanguage();
        $this->assertor = vB::getDbAssertor();
    }

    protected function buildOutputFromItems($items, $options)
    {
        $this->loadRssInfo($options);
    }

    /**
     * Loads default language data needed for RSS output
     */
    protected function loadDefLanguage()
    {
        $langid = vB::getDatastore()->getOption('languageid');
        $languages = vB_Api::instanceInternal('language')->fetchAll();
        $this->defaultLang = $languages[$langid];
    }

    /**
     * Loads information needed for RSS output
     *
     * @param     array     Options to be considered for feed.
     */
    protected function loadRssInfo($options)
    {
        $description = $this->getPhraseFromGuid(vB_Page::PAGE_HOME, 'metadesc');

        $stylevars = vB_Api::instanceInternal('style')->fetchStylevars(array(vB::getDatastore()->getOption('styleid')));
        $imgdir = (!empty($stylevars['imgdir_misc']) AND !empty($stylevars['imgdir_misc']['imagedir'])) ? $stylevars['imgdir_misc']['imagedir'] : '';
        $this->rssinfo = array(
            'title' => vB::getDatastore()->getOption('bbtitle'),
            'link' => vB::getDatastore()->getOption('frontendurl'),
            'icon' => $imgdir . '/rss.png',
            'description' => $description,
            'ttl' => 60
        );

        $this->rssinfo = $this->applyRssOptions($options, $this->rssinfo);
    }


    /**
     *
     * Gather needed channel information for RSS items.
     * Like htmltitle which is a clean version of channel title.
     *
     * @param     Array     List of items to fetch channel information for.
     *
     * @return     Array     Array containing the needed channels information.
     */
    protected function getItemsChannelInfo($items)
    {
        $info = array();
        foreach ($items AS $id => $item)
        {
            if (!isset($info[$item['content']['channelid']]))
            {
                $info[$item['content']['channelid']] = vB_Library::instance('node')->getNodeBare($item['content']['channelid']);
            }
        }

        return $info;
    }

    /**
     * Builds description tag content used in RSS outputs.
     *
     *     @param         String     Text to build description from.
     *    @param         Array     Options to consider building description.
     *
     *     @return     String     Description.
     *
     */
    protected function getItemDescription($text, $options)
    {
        // @TODO VBV-11108 description should be plain text only, replace this to use plain text parser when implemented
        if (!empty($options['fulldesc']))
        {
            $description = vB_String::htmlSpecialCharsUni(
                vB_String::fetchCensoredText(
                    vB_String::stripBbcode($text, true, false, true, true)
                )
            );
        }
        else
        {
            $description = vB_String::htmlSpecialCharsUni(
                vB_String::fetchCensoredText(
                    vB_String::fetchTrimmedTitle(
                        vB_String::stripBbcode($text, true, false, true, true), vB::getDatastore()->getOption('threadpreview')
                    )
                )
            );
        }

        return $description;
    }

    /**
     * Modifies RSS information used for output from given options
     *
     * @param     array     Options.
     */
    private function applyRssOptions($options, $info)
    {
        foreach ($this->options AS $name => $val)
        {
            if (isset($options[$name]))
            {
                switch ($name)
                {
                    case 'nodeid':
                        if (sizeof($options[$name]) == 1)
                        {
                            $channel = $this->assertor->getRow('vBForum:getPageInfoFromChannelId', array(
                                'nodeid' => $options[$name]
                            ));

                            $info['title'] = vB_Phrase::fetchSinglePhrase('external_x_hyphen_y', array($info['title'], $this->getPhraseFromGuid($channel['guid'], 'title')));
                            $info['description'] = $this->getPhraseFromGuid($channel['guid'], 'metadesc');
                        }
                        else
                        {
                            $info['title'] = vB_Phrase::fetchSinglePhrase('external_x_hyphen_y', array($info['title'], implode(', ', $options[$name])));
                        }
                        break;
                    default:
                        break;
                }
            }
        }

        return $info;
    }

    /**
     * Get metadescription phrase from a given page.guid
     *
     * @param     string     GUID
     * @param     string     Field to render phrase (title, metadesc)
     *
     * @return     string     Phrase
     */
    private function getPhraseFromGuid($guid, $phrase)
    {
        $guidforphrase = vB_Library::instance('phrase')->cleanGuidForPhrase($guid);
        $rows = $this->assertor->getRows('vBForum:phrase', array('languageid' => array($this->defaultLang['languageid'], 0, -1),
            'varname' => ('page_' . $guidforphrase . '_' . $phrase)
        ));

        $description = '';
        if (!empty($rows) AND is_array($rows) AND !isset($rows['errors']))
        {
            foreach ($rows AS $row)
            {
                // get default lang phrase if possible
                if ($row['languageid'] == $this->defaultLang['languageid'])
                {
                    $description = $row['text'];
                }
                // default install set lang -1 for page phrases which change to the right langid or 0 on page edit/save.
                else if (in_array($row['languageid'], array(0, -1)))
                {
                    $description = $row['text'];
                }
            }
        }
        else
        {
            $page = $this->assertor->getRow('vBForum:page', array('guid' => $guid));
            $description = $page[($phrase == 'title' ? $phrase : 'metadescription')];
        }

        return $description;
    }
}

/*=========================================================================*\
|| #######################################################################
|| # Downloaded: 13:52, Sun Mar 13th 2016
|| # CVS: $RCSfile$ - $Revision: 85802 $
|| #######################################################################
\*=========================================================================*/

ven0m · Oct 4, 2016

Part 3 of 3.

hmmm, interesting...

In doing a little research and testing I have discovered something more about hxxp://feedmix.novaclic.com/atom2rss.php
They are doing far more than JUST an ATOM to RSS conversion.

I have a Celtic site to which I've long wanted to add a Tumbler blog, hxxp://loki-in-myth.tumblr.com/rss
Tumbler though is notorious for badly formatted RSS.
In this example one can see that the feed outputs RSS (not ATOM); that elements of RSS 1.1 and 2.0 are used; that multiple description entries are used; that the formatting is not proper for any version of RSS.

Code:

<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" version="2.0">
<channel>
<description>A blog detailing Loki and his roles and interactions in Norse Mythology
There is a lot of conflicting information in Norse mythology, so nothing can really be considered a perfect truth about Loki.  However, I aim to provide the most reasonable conclusions and explanations that I can, based on my research.
At times, I will speculate on implications of Loki’s character/role/history from literary, mythological, and anthropological standpoints.  I will attempt to give proper explanation for each perspective.
Father: Farbauti (or Bergelmir) (male)Mother: Laufey (or Nal) (female)Possible Siblings:
Odin (m) and Hoenir (m) - contested in favour of idea that Odin and Loki are merely brothers in blood oathor
Byleist (m) and Helblind (m) - most popular conception, but also most easily contested, as both these names are names used by Odinor
Hler (m) [water], Karl (m) [air], and Ran (f) [the sea] - also contested, but prominent among Rokkr beliefs of a pantheon of gods predating the AesirRacial Affiliation: AsSpecies: God (a term used here to include As, Van, and Jotunn, as all are of the same species)Home: AsgardSpouse: Glut (1), Sigyn (2)Children: by Glut (f) (mother) - Eisa (f), Einmyria (f)
by Angrboda (f) (father) - Fenrir (m), Midgardsormr (?), Hela (f)
by Svadilfari (m) (father) - Sleipnir (?)
by Sigyn (f) (mother) - Vali (m), Narfi (m)</description>
<title>Loki: A Humble Dose of Mythos</title>
<generator>Tumblr (3.0; @loki-in-myth)</generator>
<link>http://loki-in-myth.tumblr.com/</link>
<item>
<title>What evidence is there that intertwined snakes were used as a symbol of Loki?</title>
<description>&lt;p&gt;Sorry for the confusion.  There is a lot of archaeological evidence connecting the symbol to Loki, but I don’t have any specific articles to recommend for that, so I simply cited archaeological evidence in general.  My citation of the Rundkvist article was more secondary in nature, simply to identify the prolific appearances of the symbol.  I’m planning to completely reformat this blog, so it can be more helpful to people trying to do their own research.  At that time, I will have combed through all of my articles/papers/texts/sources, so I will assign a more proper source than “archaeological evidence”.  That won’t happen for another few months though, because my current studies take up all of my time.  I’m sorry for the lack of new posts as well, but it is for the same reason.  I am so incredibly thankful for your interest though, despite the current inactivity of this blog.  I’m very excited to finish my classes, so I can get back to this!&lt;/p&gt;</description>
<link>http://loki-in-myth.tumblr.com/post/42625087535</link>
<guid>http://loki-in-myth.tumblr.com/post/42625087535</guid>
<pubDate>Fri, 08 Feb 2013 17:14:03 -0800</pubDate>
<category>Loki Laufeyson</category>
</item>
</channel>
</rss>

I tested the feed in vB, and as expected is came back with an error
XML Error: Not well-formed (invalid token) at Line 2

On a whim, I filtered the feed through hxxp://feedmix.novaclic.com/atom2rss.php and to my surprise, IT WORKED.

So, what have I learned from this experiment?
A. Many websites use terrible feed formatting
B. vB is set up to ONLY process RSS 2.0 and has no error trapping for any other version or bad formatting (it will error and crash)
C. hxxp://feedmix.novaclic.com/atom2rss.php is doing more than just ATOM to RSS processing. It is testing for the presence of multiple RSS versions, even when ATOM is suppose to be passed to it; it is reorganizing the code; and it is outputting it into proper RSS 2.0 format which works in vB.

There has to be a way to rewrite the existing vB RSS.php to do the same thing.

Search

Search

vB5 ATOM to RSS filter.

Radio

ven0m

Administrator

ven0m

Administrator

ven0m

Administrator