WWW.AU
The index is serving at least 2,000 users per month, with the top 1,000 of these hitting the index at least six times per month.
A key feature of the index has been the focus on keeping it small and accurate, partly by culling entries which effectively point at different parts of what can logically be considered one site. The accuracy comes partially from the authors themselves providing a succinct description of their site together with a definitive URL. However, all entries are moderated to achieve a level of consistency and uniformity.
A big advantage has been gained from storing the entries in a database rather than hard-coding HTML documents. Hard-coded documents are difficult to maintain and develop an inertia which can render them obsolete fairly rapidly. WWW.AU makes it relatively easy to add and modify entries, with the indexes adjusting themselves immediately and automatically. This occurs since the HTML for the indexes is generated on the fly when and as required by incoming requests.
A further advantage of storing the index as a database is that it can more easily be totally reorganised. For example, the look and feel can be readily experimented with by simply tweaking a few software parameters in the computer programs which generate the HTML responses from the database in real time. This would be much harder to achieve if the layout of the entries was tied down ahead of time by building it in to a large number of HTML documents.
On the search front, it would be convenient to make the string matching facility more powerful and general by adding a regular expression facility for specifying more complex matches.
A further area for improvement lies in the HTML forms used for updating the index. For example, it would be handy to automatically generate an update form whose default values are the existing values of the desired record in the database. At present, the forms begin with standard default values which often need to be replaced in their entirety.
Finally, in the area of deletions, one cannot in practice rely on the owners of a site to register a deletion when the site is dismantled or moved. More often than not, the first indication of this type of situation comes from user feedback. Users become concerned when clicking on an index entry no longer gets the user to the promised destination. A possible solution worth exploring is to automatically poll each URL in the database from time to time, maintaining statistics as to the outcome. If the site is unavailable sufficiently often, it could be scheduled for automatic deletion.
