A spider is a technique employed by search engines to go through the content of various websites it indexes in order to get a better idea of what are they all about. This is done whenever a site is included in the natural listings and its ranking is to be determined.