Using Yahoo Pipes For Listening

11 03 2009

Are you interested in learning about a tool that can help you manage the massive amounts of information?  In this article I will share how I use Yahoo Pipes with Twitter for my personal use.  Perhaps something about my experience will encourage you to look at Pipes yourself.

Pre-requisites:

  • In order to use Pipes, you need a Yahoo! account.
  • We’ll be discussing subscribing to RSS feeds, so you may want to brush up on your understanding of that. I’ll show briefly feed results in Google Reader and Net
  • I’ll also be discussing my use of Twitter, so if you’re interested in replicating my example, you’ll need a Twitter account.

WHAT YAHOO PIPES IS

From the Yahoo! Pipes home page:

Yahoo! Pipes is a powerful composition tool to aggregate, manipulate, and mashup content from around the web.  Like Unix pipes, simple commands can be combined together to create output that meets your needs:

  • combine many feeds into one, then sort, filter, and translate it.
  • geocode your favorite feeds and browse items on an interactive map.
  • power widgets/badges on your web site.
  • grab the output of any Pipes as RSS, JSON, KML, and other formats.

It sounds pretty technical, and to some extent it is. But fortunately, there are many examples of pipes already in use, and the best way to learn how they work is to see some of them in action.

Sound like fun?  Let’s take a look at one of the pipes I’ve put together using Twitter.

BUILDING A LISTENING FEED FOR TWITTER

Twitter Search is a tremendously useful utility for listening.  One of the things I’ve noticed about the Twitter Search RSS feed, though, is that it only shows the twitter post.  In an RSS reader like Google Reader or NetVibes (I use both), you have to expand the item to see the author.  When rapidly scanning a list of items, this can become time consuming or at least take more time than I personally want to allow.  Wouldn’t it be great to see the author and the post at the same time? Fortunately, there are some things you can do in Pipes that can help with that.

GRAB TWITTER SEARCH FEED

First, let’s grab some Twitter Search feeds. Go to http://search.twitter.com and enter a search term.  When you’re finished, click Search. In this particular example, I entered my user name.

dj1

In the Twitter Search results screen, you want to grab the RSS feed link. Here I’ve right-clicked and selected the text that says, “Copy link address”.

dj2

Now, we’re ready for Yahoo! Pipes. Go to http://pipes.yahoo.com and log in.  Click on the link to Create a New Pipe.

dj3

The visual palette opens, and you’ll see different frames for modules, debugging, and the main layout.

FETCH THE FEED

Under Sources group of modules, select the FetchFeed module and drag it in to the workspace.  Paste the Twitter Search Feed URL into the box.

dj4

FILTER BY DATES

Next, we’ll set up a Filter to filter out only the most recent Twitter posts in the search feed.  To do this we first add the Filter module.dj5

Next, we add the Date Builder module.  Hook up the Filter to the FetchFeed module, and hook up the Date Builder module to the Filter module. Make sure the Filter settings are set up so that you are permitting all feed items greater than 7 days ago.  See the picture for more information.

dj6

SORT ON PUB DATE

After this, add the Sort module and set it up to sort descending on the Pub Date.

dj7

PRE-PEND THE AUTHOR TO THE POSTED ITEMS WITH LOOP AND STRING BUILDER

We could hook this up to the Pipe Output and be done.  But as I mentioned earlier, the results in the list would only show the posted items when we see them in the feed results; we need to pre-pend the authors to the posted items.  To do that, we add the Loop module and hook it up to the Sort module.

dj8

Next, drag the String Builder module inside the Loop module.  Add the item.author.name to the String Builder.

dj9

After this, add two more items in the String Builder module, placing a colon (:) in the second one and the item.title in the third.

dj10

After this, be sure to assign results to the item.title and hook up the Loop module to the Pipe Output.

dj11

TITLE AND SAVE

Until now, the pipe we’ve created has remained untitled, so let’s give it a title and click OK when we’re done.

dj12

We’re not ready to run it just yet. Click on Save.

dj13

RUN THE PIPE AND GIVE IT A DESCRIPTION

Once the Pipe has been saved, we’re ready to run it to see the results. Click on Run Pipe.

dj14

Add a Description if you want, and click Save.

dj15

ADD THE PIPES RSS FEED TO YOUR FEED READER

Notice the buttons that are available to get the RSS.  You can add this pipe to Google Reader or NetVibes and begin using it.

Google Reader

dj16

NetVibes Feed module

dj17

Yahoo! Pipes is a great tool for mashing up content on the web. I hope you’ll take a closer look and consider adding it to your toolbelt.  In this article we created a listening feed around a single Twitter keyword item and remixed the feed to include the author of the post in the feed.

 

djj-photoDaniel Johnson, Jr., from Cincinnati, Ohio, has been spending time in the new media playground for over seven years, working on various projects and connecting with others online and in-person. During the day he develops solutions that provide information to help people do their jobs and make better business decisions.  For more information, check out  http://danieljohnsonjr.com/main/about