Post by Peter Cooper on June 15th, 2006

Screen scraping with Ruby

Peter Szinek has announced he's going to write a series of articles on 'screen scraping' with Ruby (more accurately, extracting data from Web pages and other online sources) and has released the first article entitled "Data Extraction for Web 2.0: Screen scraping in Ruby/Rails". He covers four basic scraping techniques, first using regular expressions, then HTree and REXML, then RubyfulSoup, and finally WWW::Mechanize. If you need to process shaky HTML sources from Ruby, read on.

The font used on his blog may seem a little painful to some readers, but he covers these main libraries and techniques pretty well.

[Post to Twitter] Tweet This

This entry was posted on Thursday, June 15th, 2006 at 1:00 pm and is filed under Elsewhere, Ruby Tricks, Tutorials. Responses are currently closed, but you can trackback from your own site.

Previous Post: KRJS - Extending Rails views with automatic element AJAX events
Next Post: Local del.icio.us backup with Ruby and SQLite3

One Response to “Screen scraping with Ruby”

  1. #1
    Josh Warchol Says:
    June 15th, 2006 at 3:09 pm

    Great timing! The New Haven Ruby Brigade just talked about RubyfulSoup and Mechanize at our monthly meeting last night.