README for Spackle v0.3
Copyright 2002-3 Jeremy Weatherford, see COPYING for details.

Please see the file 'INSTALL' for info on getting up and running.

======= Spackle v0.3 
Misc. fixes and improvements, new MySQL layer, see CHANGES for
details.  Next version will likely include a rewrite of the client in
Perl.

Note that the interpretation of backslashes in target scripts has
changed since v0.2!  spackle.sh used to strip a layer of backslashes
in the process of reading from the network writing the script to disk,
and it no longer does so.

Feedback is what keeps this project alive... please send your
comments, whether good or bad, and any suggestions.  Thanks!

======= Spackle v0.2
This is the second public release of Spackle.  Spackle is intended to
be a flexible architecture for the gathering and displaying of
statistics, primarily suited to monitoring servers.  This release is
not expected to be bug-free by far, but is designed to stimulate
comments and suggestions from potential users of Spackle.  I'm
especially trying to find ways to streamline the installation process.

If you find this release to be useful for your needs, great... don't
expect it to be polished at this point, though.  Definitely expect a
new public release soon, hopefully within a month or two of this
initial release.  Keep an eye on http://spackle.xidus.net for updates.


======= Information about Spackle
The idea behind Spackle is to allow the monitoring of multiple systems,
generally servers, with a minimum of effort.  

The server side of the equation is simply a set of PHP scripts,
backed by a PostgreSQL database.  (support for MySQL probably coming
soon) This part can be located on any machine, including one being
monitored.  The server stores information about the targets to be
monitored in its database.  A target consists of basic information
about a statistic, such as the units to measure it in, and includes a
script to be run in order to produce the stat.  The server stores all
gathered statistics by using RRDtool
(http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/), which allows
for the archiving of a large amount of data in a minimal amount of
space.  (my local install has 211 stats with data stored for the past
year, taking up only 10megs of disk)

The client is a single bash shell script, intended to be run
periodically (generally every 5 minutes) on each system to be
monitored.  Generally speaking, the client does not need any special
permissions, and can be run by an ordinary user.  (A few of the
targets do require additional permissions)

When the client runs for the first time, it connects to the server via
a normal HTTP request.  The server determines which targets should be
monitored by that client, and transfers the necessary scripts, along
with any data marked for that specific client.  The client then runs
each script, collecting the output, and transfers the results to the
server via an HTTP POST request.  After the client has transferred the
needed data once, it simply checks a serial number on the server each
time to determine whether to update its data or used the cached data.

This centralization of the scripts to gather stats allows for the
clients to be set up once on each machine to be monitored, and then
left alone.  Need to add a new stat to be measured on each of 50
machines?  Set up the target on the server, increment the system-wide
serial number, and each client will update itself next time it runs.


======= Security concerns
Since a Spackle client executes content which is received over the
network, additional security risks are incurred.  As a result, I
strongly recommend running the Spackle client as a cronjob by a
NON-PRIVILEGED user.  Again, there's no need to run the Spackle client
as root.  For the few stats that may need additional privileges, an
alternate method is provided (look at client/spackle-root.sh).

The security of the client depends completely on the security of the
server.  If the web interface of the server is compromised and a
script is modified, it will be distributed to the clients using it,
and executed.  It is my belief that I've designed the authentication
system on Spackle to be completely secure against any kind of external
or network-sniffing attack.  However, it -is- vulnerable if the
database contents are accessible.  Please take care to secure your
database, especially the auth table, from any kind of public access.

Any advice regarding making Spackle even more secure would be greatly
appreciated.


======= Spackle details
Here's some more information about each component of Spackle.

==== Server
The server consists of a set of PHP scripts:
 add.php, admin.php, list.php, view.php -- administrator interface
 index.php -- graphing output, generally publicly accessible
 client.php -- handles requests from the Spackle client

The PHP include files:
 auth.inc       -- authorization library (see below for more info)
 client-rrd.inc -- RRDtool routines used by client.php
 rrd.inc        -- additional RRDtool routines, including graphing
 common.inc     -- misc
 db.inc         -- PostgreSQL database interface, not very abstracted yet
 html.inc       -- misc routines for dealing with HTML

==== Authorization:
Spackle uses a somewhat unique method for authorization.  The only
requirement is that the authenticating user's browser supports
Javascript, because it relies on client-side MD5 hashing.  The steps
taken for a user to log in are as follows:
 1) user connects to index.php
 2) init_auth generates a new, unique session hash, and sets a cookie
accordingly.  The session hash is also added to the 'session' table,
along with a 'client hash', incorporating data about the user's IP
address and User-Agent, to reduce the possibility of cookie-stealing
 3) The login box is shown, and a random challenge hash (also stored
in 'session') is sent over the wire to the user's browser
(incorporated in the Javascript)
 4) The user enters their login name and password, and hits the Login
button.  The client-side Javascript takes the MD5 hash of the
password, then takes the hash of the challenge plus the password hash,
effectively signing the challenge with the password hash, and sends
this as the response.  The password is never sent over the wire.
 5) The server looks up the session, verifies the clihash is constant,
and that the response matches the expected response for the username
indicated.  If so, the session table is updated with the user name,
and the user is considered logged in.

The only vulnerability I see in this is that the server's database is
vulnerable.  If its contents were to be read, specifically the
password hashes from the auth table, the password hash would, by
itself, be sufficient to forge a response to the server, without
knowing the actual password.  I don't see any way to rule out this
possibility without making other things less secure, though.

In any case, I believe this method to be very secure, and would like
to see something like it being used more often in web-apps, to prevent
against sniffing attacks, which are frequently trivial to implement.

==== Client
The shell-script client is very straightforward, depending on the
server for its intelligence.  It simply splits the server's response
into scripts to be run, and data files to be used by the scripts.  It
runs each script in turn, collecting the output and constructing a
POST request using it, then transfers the gathered stats by using that
request.

The client is intended to be run without special permissions, in order
to decrease the security risk of executing data received via the
network.  If you wish to monitor stats which require root privileges,
an alternate method should be used.  See client/spackle-root.sh for
details.


======= Comments?
All suggestions and feedback are very welcome... please feel to
contact me at xidus@xidus.net.  I'd love to see how you're using
Spackle, so if you'd like to drop me a link to your monitoring site,
I'd really enjoy taking a look at it.

Enjoy!

Jeremy Weatherford
xidus@xidus.net
http://xidus.net