Harry Anderson has written a thorough series of articles on the SecurityFocus web site called an Introduction to Nessus. He goes into detail about how the nessus client and server pieces work. He also gives some advice on interpreting Nessus results.
Nessus review by PC Magazine: "Based on a client/server architecture, Nessus lets users run the administrative console, which executes vulnerability scans and holds databases on a machine other than the ser".
I personally think PC Magazine was dismissive of Nessus. These magazines tend to be biased against the non-Windows platforms. I have not done so myself but one can write one's own security scans via the Nessus plug-in language (NASL) and there is at least on Perl module available to programmatically control Nessus. My employer has built a Linux, Apache, PostgreSQL, and Perl based vulnerability assessment application around the Nessus security scanner.
Renaud Deraison, the author of the popular open source security scanner, has announced the release of Nessus 2.0.10. This will be the last release of Nessus 2.0 as Renaud and others are working on the Nessus 2.1. This release fixes several problems with false positive and path detection and better service fingerprinting (I wonder if they could use NMAP for this).
Some new features are:
Service version fingerprinting
IP addresses are now sorted in EVERY report
Automagically rewrite banners to handle distributions which do back-porting of security fixes (ie: Debian)
Non-intrusive OS-fingerprinting (based on xprobe's techniques)