When most people think of site visitors, they think of people - people seeking information, people filling out forms, people shopping, hopefully buying, people returning to buy more.
Though most think of individual people when they think of site visitors, technically speaking, websites are not actually visited, even in an electronic form. In a sense, there is no such thing as a site visitor. Instead of saying a person visits a site, it is correct to say that a person's user agent requests files which are retrieved from a website and rendered at the person's personal computer, thus creating the website traffic. The traffic is your site being sent to the person's computer after being invited by a user agent.
User agents are basically different types of software that request files from your web server. A road is likely to be used by an automobile driven by a person on his or her way to work, by a cyclist out for exercise, by a person walking across the street, by a semi truck hauling goods, and even by the occassional opposum or deer crossing the street. Just like roads get used by lots of things for many different reasons, your website gets used by lots of things for many different reasons. The most common users of your website are web browsers operated by a person, but your site is most likely used by other agents, such as crawlers, robots or spiders.
NOTE: User agents don't actually crawl over and visit your site. They simply send a message to your server requesting that your server send them your web pages or other files. Even spiders don't really "crawl." They just make requests for file transfers from your server. In a sense, your website is the visitor. Your website visits the people you have been thinking of as visitors.
The World Wide Web Consortium states that "User agents include HTML browsers and other types of software that retrieve and render Web content." They further state that a user agent is "Any software that retrieves and renders Web content for users. This may include Web browsers, media players, plug-ins, and other programs � including assistive technologies -- that help in retrieving and rendering Web content."
- from W3C: User Agent Accessibility Guidelines 1.0.
Web Browsers
Instructions for tracking web browsers as well as browser reviews. Presents the best web browsers and links to free browser downloads.
Crawlers, Robots, or Spiders
Discussion of web crawlers, robots, or spiders. Describes the most common crawlers and provides links to excellent follow-up resources.
