Related pages: Web site basics • Making your own web site

Search engines
making your site visible

Getting visitors

Visitors will only find a site by word of mouth (which includes email, blogs and forums these days), through a search engine or adverts. Generally speaking it’s better to get a good number of the right visitors than lots of random visitors. You’ve probably read about pumping up search rankings and other ways of blagging visitors—here’s what I set out to do:

Search engines are useful but probably won’t be enough on their own.

Web site design

This site is simple and fast, the message is clear and it’s easy to find things. There are no pop-up windows, animations, award logos, software badges, affiliate buttons, banner ads or contacts from a link exchange. The sound clips are MP3 and don’t need a special player. You don’t need to register anywhere to use the site.

Some “professional” sites are impossible to use because of all the junk. User interface experts say a third of visitors don’t wait more than 8 seconds for the first page.

Web site links

A link exchange won’t help unless it’s a high value site targeted at your audience. You don’t want to waste your visitors’ time sending them to a low value list.

I told my best contacts when I launched the site and word of mouth was far more powerful than I expected. These visitors are more likely to be tuned-in to what you’re doing. The word of mouth network is definitely worth spending some time on.

Web site statistics

I recommend using a web statistics tool to tell you who’s visiting, where they’re from and what they’re looking at. This helps you keep the site relevant. Most web hosts offer one and there are several freeware statistics sites. (I’ve used Matrix Stats which cost about £10 a year and works well, Urchin which is very good, AWStats which is OK and Webalizer which is poor. Urchin is used by Google Analytics.) Stats will show:

Search engine tips

Search engines will only find a site if you tell them it’s there or if other indexed sites link to it. It’s worth remembering the blindingly obvious, that searching always uses the words that are there, not what you meant to say, and not pictures, diagrams or logos. Some directories ask for keywords and you’ll probably use them in your page titles and in HTML headers anyway. It took me a while to get this right.

Search-friendly HTML

There’s a lot of folklore about search engine friendly HTML techniques and contradictory advice on just about every angle. I designed the site for users rather than search engines and I don’t think I’m missing any magic tricks that would get better rankings. Here are some common sense steps.

If trial searches for your own site fail, look at the HTML source for the top result pages and see if it gives any clues.

There are other ways to write your pages; this is just my best guess from the research I’ve done, and it works. I have many top page ranks and many top 10 page ranks on Google.

Meta name tags

Meta name tags are used by some search engines and not by others, you can safely forget about them. I have tried with and without—information in meta name tags is effectively ignored by all the big search engines.

Submitting this web site

None of the major search engines will index a site quickly unless you pay, and their free services vary widely. Things were changing rapidly as I went along and it’s not easy to make sense of the search engine network. In some cases it’s hard to work out how to submit a site and what the terms are. Some different nationality sites have different options, so it’s worth trying variations (e.g. and The trend is to charge for submissions but many are still free. I didn’t use a mass submission service; the free ones tend to be a front for junk mailers.

Free search engines I used

Search engineTypeHow submission works
GoogleindexIndexes a site from the main URL, also uses the Open Directory Project directory. Google shares its index with many other search engines.
All The WebindexIndexes a site from the main URL (also offers Lycos chargeable options).
AltaVistaindexSays it indexes specific pages (not everything under one main URL) and only 5 pages at a time. The AltaVista crawler (Scooter) visited after about two months and indexed the whole site, not just the pages I submitted.
Ask JeevesindexSimple submission by email —one main URL and a description of the site. You will get hits from anyway.
MusicDishdirectoryA specialist directory but they don’t change an entry once it’s up—you can link an MP3. No good for direct traffic but scores as a Google link.
Open Directory Project (dmoz)directoryThis is the directory used by Google and many others. It took nearly a year to get onto ODP.

Search engines I didn’t use

DogpileUses information from other search engines.
ExciteHas links to the Open Directory Project (so I assume it uses that directory) and a free option that indexes specific pages you submit. It says 1 page at a time but I couldn’t get it to work at all.
YahooAppears to be free but if you sell anything you must use the chargeable Yahoo Express service.
LycosI could only find free options from the UK site. The index option worked but the directory option didn’t. Otherwise Lycos is part of a (mostly chargeable) group including Overture, Infoseek, Infospace, MetaCrawler, WebCrawler, HotBot, InSite, DirectHit, Go and others, sharing a mix of indexes, directories, data and portals.

Although I didn’t submit to Yahoo, Excite and Lycos, their crawlers visited and I get searches referred by them and many others (e.g. MSN and AOL).

Web site submission lead time

This is the sequence of events with Google and ODP from July to November 2002, as an illustration of submission lead time. I also had a number of hits by FAST WebCrawler and other spiders but it’s difficult to track the source and timing because their search engine network is complex (Lycos above).

DateElapsed daysGooglebotPages
What happened
6 July0---0First URL submission
6 August31-10100Nothing searchable yet
25 August50--101Some home page words searchable
27 August520-101Submitted to the Open Directory Project (ODP)
1 September5745151More Googlebot visits
2 September58516311More Googlebot visits
11 September6714-311Subscribed to Google free web and site search
21 September7724-311Resubmitted URL today
23 September79261321Another Googlebot visit.
Cached page same as 25 August
24 September8027-322Cached page updated!
26 September82291331Still just the home page indexed
28 September84  3320Site indexed (Google says about 24 pages).
Checked the pages indexed, initially 20, then 24
6 October-39255824More Googlebot visits (regular update)
8 October-4126024More Googlebot visits
9 October-4216124More Googlebot visits
Oct/Nov   6132More pages indexed by Google
7 November-7116232More Googlebot visits (regular update)
8 November-726112332More Googlebot visits
9 November-73112432More Googlebot visits
23 November-87-12432Still not on ODP (but listed in July 2003)

My Google index was originally updated about once a calendar month: the crawler visited for a day or two and about 4-8 weeks later the index was updated. This meant the index could lag the site content by up to 3 months. These days updates are generally on Google within a week or so. Here are a few tips for checking your Google visibility.

The most useful online guide I found about search engines, submission, etc. is Robert Woodhead’s site. It has good advice and coverage (although I’m sceptical about search ranking tricks) and provides some useful tools.

Summary of web site publicity

Here’s a summary of what worked best.

By the end of 2002 my hit rate was good enough and although I keep tweaking I don’t plan any major work on HTML or search engines. It’s now simply a matter of keeping the site up-to-date.

Here’s the kind of growth rate you can expect on a popular site without advertising. Slow but steady.

Bemuso stats web site annual growth curve

Getting a foothold on the web can be a slow and frustrating process but you only have to do it once. The web crawling strategy of Google has changed considerably since 2002 so your experience may be completely different but the basic principles are still the same.

go to the top of this page  go to the main menu for this page  go to the home page

Related pages: Web site basics • Making your own web site

© Rob Cumberland 2002–2013, all rights reserved • This is a UK web site • About Bemuso