home / articles / cgi

Frame Search

Everybody knows how to search the web using Yahoo. Unfortunately, you may not like the results that you get from such a search. Often, there are too many choices that seem to have little relevance to what you asked for. Well, this week's article shows how to make it easier to dig through the results.

Using Perl and HTML Frames, this week's script will format the results from a search engine into a more usable format. Borrowing from Microsoft's Internet Explorer 4.0, I've designed a simple three-frame interface for performing a search. In the top-left frame, enter the topic you want to find. Hit the GO! button and the results will appear in the lower frame. Then, click on one of the links in the lower frame and it will appear in the large frame on the right.

Try It Out

How it Works

• fsindex.html - main .html file, contains the FRAMESET
• fsmain.html (main) - file for the main window
• fsresult.html (resultFrame) - file for the search results window
• fstype.html (typeFrame) - form that takes input and submits it to fscgi.pl

Here are the names for each of the frames in the FRAMESET:

When you point your web browser to fsindex.html, it loads the FRAMESET and displays the appropriate .html file in each frame. When you click GO! in the typeFrame, it submits your query to fscgi.pl. But, instead of having the results be displayed back into typeFrame, the output goes to resultFrame. How is this accomplished?

Here's the magic code, found in fsindex.html:

<form method=get action="/cgi-bin/fscgi.pl" target="resultFrame">

Having target= in the <FORM> tag makes this possible. It specifies the frame where the results will appear.

Next, consider the contents of the .html file returned by a query to Yahoo. It has HTML that looks something like the following:

<a href="http://www.spam.com">Hormel Foods Corporation</a>

If you clicked on a link like that in the resultFrame, the resulting URL will load into the resultFrame. Again, this is not what we want. Instead, we need to redirect the new URL into the main frame. This means that the HTML will need to be changed:

<a href="http://www.spam.com" target=main>Hormel Foods Corporation</a>

This is a trivial task in Perl:

s/(<a href=\".+?\")(.+?>)/$1 target=main $2/gi;

Also, you may have noticed that a few images are included in Yahoo's results. In the interest of keeping the results clean and simple, we can easily eliminate all the <IMG> tags with a little Perl code:

s/<img.+?src.+?>//gi;

Finally, using Perl to act as a a web browser or User Agent to submit the search query is interesting. To learn more, look at the file named fscgi.pl in the source file. It contains the Perl subroutines used to establish a socket connection and send an HTTP request. Using Perl to make HTTP requests is quite powerful. I'll cover more topics in this area in the weeks to come.