rss feeds – how to get it wrong and how to get it right


Over at Freethought blogs a few of the folks there are a bit upset with a chap who is scraping all their postings via the RSS feed. It is a great example of how to be a complete douche.

– PZ writes about it here with his usual sprinkling of humor – “I guess this is Peter Grant’s blog now

– Ed Braytonalso writes about it in more detail here – “Peter Grant Steals from FTB

OK, so what it the problem here. Well, it turns out that Mr Grant runs a website that aggregates other blogs. Now perhaps you might be thinking, “Yea, but lots of sites do that, so what is the problem here?“. Well, the issue is that the way he does it is wrong for a lot of reasons.

The site is here, and at the time of writing (it might have changed by the time you read this), he is …

  • Literally copying every single post in its entirety
  • Does not attribute where any come from
  • He does Provide links back to the original, but even that is not a clue because he uses divr.it links

Somebody might wish to point him in the direction of what is and is not legal. Basically, it is fine to quote a short extract and then link back to the original, but for fracks sake at least acknowledge who wrote it and where it came from.

As an example of how to get it right, here is a site that also aggregates from other blogs, but look, “The Atheism News Magazine” contains a short excerpt from each post and a link to the rest of it, all with attribution. You can quite clearly see where it comes from and so that is perfectly fine, is well covered by fair use and so if Mr Grant did it like this, then he would be OK.

However, all these sites that suck up RSS feeds and serve up links are a tad redundant because that is not what RSS feeds are really for. If you are familiar with RSS and make regular use of an RSS reader, then you can stop reading now, the rest of this posting is not for you. However, if the term RSS is a bit of a mystery, well here now is a quick introduction.

What is RSS?

RSS stands for Really Simple Syndication. It is a standard by which sites that publish lots of stuff on a regular basis can announce new articles.

There are lots of websites out there and they all present information in lots of different layouts, so if you attempted to write some software to check all the sites you care about for updates, it would be a complete nightmare. Each and every site would need special software just to read its unique layout, and when the layout changed it would need to be changed as well. You just could not do it, and could never keep up.

Ah, but what if all such sites had a page that had exactly the same machine readable layout that was updated every time a new article appeared, would that solve the problem? Yep it would and so that is exactly what RSS is. Each and every site with an RSS feed is simply a site that has such a machine-readable page in this common format.

You can read up on RSS here.

OK, so all  sites that support RSS have a page in a common format, now what?

Well, the secret sauce for you as an individual is that you can use what is called an RSS Reader. Basically you collect up the RSS feeds for all the sites you are interested in, put them into your RSS reader and it then gives you a list of links to all the latest articles for all the sites you care about, along with a snippet of information regarding the article’s content.

This is very handy because all you need to do to check out what is new across hundreds of sites is to use your news reader, so in seconds you can see what is new and not waste your time scanning across hundreds of websites.

You can also collect different feeds into different categories, for example you might choose to collect links into categories such as

  • Skeptical
  • Science
  • General News
  • Atheism
  • Religion
  • etc…

This allows you to focus on a specific subset, or if you prefer, to look across them all in one go by clicking an “All Links” category.

Over time as you discover new sites you can add their RSS feed, and of course as your preferences, or for that matter as the sites themselves change, you can discard.

It is you own personal daily feed of information that you can tune and refine to contain exactly what you want.

What RSS Reader should I use?

You have lots of choice, but often a good place to start is by crowd-sourcing … in other words, ask yourself the question, “What are the most popular?”, then go check them out and see if you personally feel comfortable with it.

You have a choice of two basic types

  • Web-based : You can log in from anywhere, all you need is a web-browser
  • Client-based : An RSS Client program is installed on your PC

My personal preference is for web-based because I like the freedom of logging in from anywhere, but that is just me. Others might like a client on their desktop because they then have a bit more control over it.

OK, so I want to give it a go, where can I start?

Pick one and just dive in.If it is not for you, then try a few of the others out there.

To start you off, why not try Google Reader. I’m picking this because you can quickly and very easily use it to just give this a go.Just surf on over to Google, and sign in … if you do not have a google account, register, then sign in to get up and running.

Once you are in you might then wish to begin adding links. How? Well, Google makes it easy and they enable you to browse for collections of RSS links (just click the “Browse for Stuff” on the left hand side). As you search, you get an idea of how popular it is by seeing how many have subscribed.

There is of course lots more to learn, there always is, but hopefully the above will be enough to enable you to leap in a get going.

Enjoy.

Leave a Reply