qsf-1.2.7/ 0000755 0000764 0000764 00000000000 10665021416 010123 5 ustar aw aw qsf-1.2.7/README 0000644 0000764 0000764 00000006611 10664537024 011015 0 ustar aw aw Introduction
************
This is the README for `qsf', a quick spam filter. QSF uses a database of
"tokens" taken from previous emails to determine whether any given email is
likely to be spam or not. If an email is miscategorised, simply send the
email back to QSF with the appropriate parameters to modify the database and
make it more accurate.
Note that `qsf' works well with either its own backend databases, GDBM,
SQLite, or MySQL. GDBM and SQLite are fast but the builtin database reduces
dependencies on other libraries, whereas MySQL can be harder to set up.
If you use the MySQL backend, "make test" may fail. Read the documentation
in the "MYSQL BACKEND" part of the manual for information on how to use
`qsf' with MySQL. Also, it has been reported that MySQL 4.1.x works much
better with QSF than 4.0.x or the 3.23.x series.
Documentation
*************
A manual page is included in this distribution. If you prefer plain text,
then look at `doc/quickref.txt' for a text version.
Dependencies
************
The internal binary tree and list databases have no dependencies.
The GDBM backend requires GDBM: http://www.gnu.org/software/gdbm/
The MySQL backend requires MySQL: http://www.mysql.com/
The SQLite backend requires SQLite: http://www.sqlite.org/ (version 2.x).
Note that if you try to compile MySQL support but the MySQL libraries cannot
be found, make sure you have the MySQL development package installed.
You may also need to install the "zlib-devel" package.
Compilation
***********
To compile the package, type "sh ./configure", which should generate a
Makefile for your system. You may then type "make" to build everything.
You may need GNU `make' for compilation to work properly.
See the file `doc/INSTALL' for more about the `configure' script.
Developers note that you can do "./configure --enable-debugging" to cause
debugging support to be built in. Also note that doing "make index" will
generate an HTML code index (using `ctags' and `cproto'); this index lists
all files used, all functions defined, and all TODOs marked in the code.
Memory allocation tracing (MALLOC_TRACE) may cause a crash if the allocation
log file gets too large (>2GB usually). This is a bug in the C library;
unfortunately it cannot be fixed in `qsf'.
There are some extra shell scripts in the "extra/" directory, which can help
with generation of statistics. Read the scripts for full details.
Author
******
This package is copyright (C) 2007 Andrew Wood, and is being distributed
under the terms of the Artistic License 2.0. For more details, see the
file `doc/COPYING'.
You can contact me by email at andrew.wood@ivarch.com or by using the
contact form on my web page at http://www.ivarch.com/.
Suggestions and patches have been received from the following people:
Tom Parker
Dr Kelly A. Parker
Vesselin Mladenov
Glyn Faulkner
Mark Reynolds
Ondrej Suchy
Sam Roberts
Scott Allen
Karsten Kankowski
M. Kolbl
Micha Holzmann
Jef Poskanzer
Clemens Fischer
Nelson A. de Oliveira
Michal Vitecek
Tommy Pettersson
The `qsf' home page is at:
http://www.ivarch.com/programs/qsf/
The latest version can always be found here.
-----------------------------------------------------------------------------
qsf-1.2.7/src/ 0000755 0000764 0000764 00000000000 10665021416 010712 5 ustar aw aw qsf-1.2.7/src/include/ 0000755 0000764 0000764 00000000000 10665021416 012335 5 ustar aw aw qsf-1.2.7/src/include/message.h 0000644 0000764 0000764 00000004265 10554714162 014145 0 ustar aw aw /*
* Message handling prototypes and structures.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#ifndef _MESSAGE_H
#define _MESSAGE_H 1
#ifndef _OPTIONS_H
#include "options.h"
#endif
#define MAX_MESSAGE_SIZE 512*1024
struct msg_s;
typedef struct msg_s *msg_t;
struct msg_s { /* structure describing an email message */
char *original; /* the original message, in full */
long original_size; /* the length of the full original message */
char *sender; /* email address from "From:" header */
char *envsender; /* email address of envelope sender */
int num_headers; /* number of header lines */
char **header; /* the header lines */
char *body; /* the original message body */
long body_size; /* the size of the original message body */
char *content; /* decoded content of message */
long content_size; /* the size of the decoded content */
long content_alloced; /* size of block allocated */
char *textcontent; /* decoded content of message, no HTML */
long text_size; /* the size of the decoded non-HTML content */
long *wordpos; /* positions of words in decoded non-HTML */
int *wordlength; /* the length of each word */
long num_words; /* size of the above array (no. of words) */
int num_images; /* the number of image attachments found */
char *_bound[8]; /* content boundaries */
int _bdepth; /* current boundary nesting depth */
char _in_header; /* in message header */
char _encoding; /* current content encoding type */
char _nottext; /* set if current content is not text */
long _pos; /* decoding position */
};
msg_t msg_parse(opts_t);
void msg_spamsubject(msg_t, char *);
void msg_spamheader(msg_t, char *, double);
void msg_spamratingheader(msg_t, double, double);
void msg_spamlevelheader(msg_t, double, double);
void msg_dump(msg_t);
void msg_free(msg_t);
char *msg_from_base64(char *, long *);
char *msg_from_qp(char *, long *);
char *msg_decode_rfc2047(char *, long *);
int msg_addcontent(opts_t, msg_t, char *, long);
msg_t msg_alloc(opts_t);
int msg_read(opts_t, msg_t);
int msg_headers_store(opts_t, msg_t);
#endif /* _MESSAGE_H */
/* EOF */
qsf-1.2.7/src/include/md5.h 0000644 0000764 0000764 00000000630 10466704356 013204 0 ustar aw aw #ifndef MD5_H
#define MD5_H
#include
typedef uint32_t uint32;
struct MD5Context {
uint32 buf[4];
uint32 bits[2];
unsigned char in[64];
};
extern void MD5Init();
extern void MD5Update();
extern void MD5Final();
extern void MD5Transform();
/*
* This is needed to make RSAREF happy on some MS-DOS compilers.
*/
typedef struct MD5Context MD5_CTX;
#endif /* !MD5_H */
qsf-1.2.7/src/include/log.h 0000644 0000764 0000764 00000000450 10554714163 013273 0 ustar aw aw /*
* Functions for logging.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#ifndef _LOG_H
#define _LOG_H 1
void log_level(int);
void log_add(int, char *, ...);
void log_dump(char *);
void log_errdump(char *);
void log_free(void);
#endif /* _LOG_H */
/* EOF */
qsf-1.2.7/src/include/database.h 0000644 0000764 0000764 00000002054 10554714164 014261 0 ustar aw aw /*
* Database handling prototypes, structures, and constants.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#ifndef _DATABASE_H
#define _DATABASE_H 1
typedef enum {
QDB_NEW,
QDB_READONLY,
QDB_READWRITE
} qdb_open_t;
struct qdb_s;
typedef struct qdb_s *qdb_t;
struct qdbint_s;
typedef struct qdbint_s *qdbint_t;
typedef struct {
unsigned char *data;
int size;
} qdb_datum;
qdb_t qdb_open(const char *, qdb_open_t);
int qdb_fd(qdb_t);
char *qdb_type(qdb_t);
void qdb_close(qdb_t);
qdb_datum qdb_fetch(qdb_t, qdb_datum);
int qdb_store(qdb_t, qdb_datum, qdb_datum);
int qdb_delete(qdb_t, qdb_datum);
qdb_datum qdb_firstkey(qdb_t);
qdb_datum qdb_nextkey(qdb_t, qdb_datum);
void qdb_optimise(qdb_t);
char *qdb_error(void);
void qdb_unlock(qdb_t);
void qdb_relock(qdb_t);
void qdb_restore_start(qdb_t);
void qdb_restore_end(qdb_t);
/*
* Common library functions for backends to use.
*/
int qdb_int__lock(int, int, int *);
void qdb_int__sig_block(void);
void qdb_int__sig_unblock(void);
#endif /* _DATABASE_H */
/* EOF */
qsf-1.2.7/src/include/options.h 0000644 0000764 0000764 00000006177 10554714165 014223 0 ustar aw aw /*
* Global program option structure and the parsing function prototype.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#ifndef _OPTIONS_H
#define _OPTIONS_H 1
struct opts_s;
typedef struct opts_s *opts_t;
typedef enum {
ACTION_NONE,
ACTION_TEST,
ACTION_MARK_SPAM,
ACTION_MARK_NONSPAM,
ACTION_TRAIN,
ACTION_PRUNE,
ACTION_DUMP,
ACTION_RESTORE,
ACTION_TOKENS,
ACTION_BENCHMARK,
ACTION_MERGE,
ACTION__MAX
} action_t;
struct opts_s { /* structure describing run-time options */
char *program_name; /* name the program is running as */
char *database; /* location of the database file */
char *globaldb; /* location of global database */
char *globaldb2; /* location of 2nd global database */
char *plainmap; /* location of plaintext map */
void *dbr1; /* first db handle, if any */
void *dbr2; /* second db handle, if any */
void *dbr3; /* third db handle, if any */
void *plaindata; /* plaintext working data */
int db1weight; /* weighting multiplier for db 1 */
int db2weight; /* weighting multiplier for db 2 */
int db3weight; /* weighting multiplier for db 3 */
void *dbw; /* db handle to write to, if any */
void *inbuf; /* stdin replacement, if any */
long inbufsize; /* size of stdin replacement */
unsigned char modify_subject; /* whether to modify subject line */
unsigned char no_header; /* set if not to add an X-Spam line */
unsigned char add_rating; /* set if adding X-Spam-Rating line */
unsigned char add_stars; /* set if adding X-Spam-Level line */
unsigned char no_filter; /* set if we're not filtering */
double threshold; /* spam threshold (default 0.9) */
unsigned char allowlist; /* set if allow-list is enabled */
unsigned char denylist; /* set if deny-list is enabled */
unsigned char modifydenylist; /* set if acting on the deny-list */
unsigned int weight; /* weighting to use when marking */
unsigned char noautoprune; /* set if we've not to auto-prune */
unsigned char showprune; /* show verbose prune indicator */
unsigned int loglevel; /* logging level (default 0) */
unsigned int min_token_count; /* min tokens before giving a score */
unsigned long prune_max; /* max tokens to prune at once */
char *subject_marker; /* string to add to subject if spam */
char *header_marker; /* string to set X-Spam header to if spam */
char *mergefrom; /* database to merge data from */
char *emailonly; /* email address to use in -e mode */
char *emailonly2; /* extra email address for -e */
int argc; /* number of non-option arguments */
action_t action; /* what action we are to take */
char **argv; /* array of non-option arguments */
};
extern opts_t opts_parse(int, char **);
extern void opts_free(opts_t);
#endif /* _OPTIONS_H */
/* EOF */
qsf-1.2.7/src/include/spam.h 0000644 0000764 0000764 00000001641 10554714166 013460 0 ustar aw aw /*
* Spam handling prototypes, structures, and constants.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#ifndef _SPAM_H
#define _SPAM_H 1
#ifndef _OPTIONS_H
#include "options.h"
#endif
#ifndef _MESSAGE_H
#include "message.h"
#endif
enum {
SPAM,
NONSPAM
};
#define TOKEN_CHARS "0123456789" \
"abcdefghijklmnopqrstuvwxyz" \
"ABCDEFGHIJKLMNOPQRSTUVWXYZ" \
"_'.$£-!"
double spam_check(opts_t, msg_t);
int spam_update(opts_t, msg_t, int);
int spam_dumptokens(opts_t, msg_t);
int spam_db_dump(opts_t);
int spam_db_restore(opts_t);
int spam_db_prune(opts_t);
int spam_db_merge(opts_t);
int spam_train(opts_t);
int spam_benchmark(opts_t);
int spam_allowlist_manage(opts_t);
int spam_denylist_manage(opts_t);
void spam_plaintext_update(opts_t, char *, int, char *, int);
void spam_plaintext_free(opts_t);
#endif /* _SPAM_H */
/* EOF */
qsf-1.2.7/src/include/mailbox.h 0000644 0000764 0000764 00000000753 10554714167 014157 0 ustar aw aw /*
* Functions for reading messages from a mailbox.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#ifndef _MAILBOX_H
#define _MAILBOX_H 1
#ifndef _OPTIONS_H
#include "options.h"
#endif
#ifndef _STDIO_H
#include
#endif
struct mbox_s;
typedef struct mbox_s *mbox_t;
mbox_t mbox_scan(opts_t, FILE *);
void mbox_free(mbox_t mbox);
size_t mbox_count(mbox_t mbox);
int mbox_select(opts_t, mbox_t, FILE *, size_t);
#endif /* _MAILBOX_H */
/* EOF */
qsf-1.2.7/src/library.c 0000644 0000764 0000764 00000005632 10664772303 012537 0 ustar aw aw /*
* Some small library functions.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include
#include
#include
/*
* Find the start of the first occurrence of the substring "needle" of
* length "needlelen" in the memory area "haystack" of length "haystacklen".
* Returns a pointer to the start of the beginning of the substring, or NULL
* if not found.
*/
char *minimemmem(char *haystack, long haystacklen, char *needle,
long needlelen)
{
char *found;
if (haystack == NULL)
return NULL;
if (haystacklen < 1)
return NULL;
if (needle == NULL)
return haystack;
if (needlelen < 1)
return haystack;
while (haystacklen > needlelen) {
found = memchr(haystack, needle[0], haystacklen);
if (found == NULL)
return NULL;
haystacklen -= (found - haystack);
if (haystacklen < needlelen)
return NULL;
haystack = found;
if (memcmp(haystack, needle, needlelen) == 0)
return haystack;
haystack++;
haystacklen--;
}
return NULL;
}
#ifndef HAVE_GETOPT
char *minioptarg = NULL;
int minioptind = 0;
int miniopterr = 1;
int minioptopt = 0;
/*
* Minimalist getopt() clone, which handles short options only and doesn't
* permute argv[].
*/
int minigetopt(int argc, char **argv, char *optstring)
{
static int nextchar = 0;
int optchar;
int i;
if ((minioptind == 0) && (argc > 0))
minioptind++;
if ((nextchar > 0) && (argv[minioptind][nextchar] == 0)) {
minioptind++;
nextchar = 0;
}
if (minioptind >= argc)
return -1;
/*
* End of options if arg doesn't start with "-"
*/
if (argv[minioptind][0] != '-')
return -1;
/*
* End of options if arg is just "-"
*/
if (argv[minioptind][1] == 0)
return -1;
/*
* End of options if arg is "--", but don't include the "--" in the
* non-option arguments
*/
if ((argv[minioptind][1] == '-') && (argv[minioptind][2] == 0)) {
minioptind++;
return -1;
}
if (nextchar == 0)
nextchar = 1;
optchar = argv[minioptind][nextchar++];
for (i = 0; optstring[i] != 0 && optstring[i] != optchar; i++) {
}
if (optstring[i] == 0) {
minioptopt = optchar;
if (miniopterr)
fprintf(stderr, "%s: invalid option -- %c\n",
argv[0], optchar);
return '?';
}
if (optstring[i + 1] != ':') {
minioptarg = NULL;
return optchar;
}
/*
* At this point we've got an option that takes an argument.
*/
/*
* Next character isn't 0, so the argument is within this array
* element (i.e. "-dFOO").
*/
if (argv[minioptind][nextchar] != 0) {
minioptarg = &(argv[minioptind][nextchar]);
nextchar = 0;
minioptind++;
return optchar;
}
/*
* Argument is in the next array element (i.e. "-d FOO").
*/
nextchar = 0;
minioptind++;
if (minioptind >= argc) {
fprintf(stderr, "%s: option `-%c' requires an argument\n",
argv[0], optchar);
return ':';
}
minioptarg = argv[minioptind++];
return optchar;
}
#endif /* HAVE_GETOPT */
/* EOF */
qsf-1.2.7/src/md5.c 0000644 0000764 0000764 00000017011 10664772303 011552 0 ustar aw aw /*
* This code implements the MD5 message-digest algorithm.
* The algorithm is due to Ron Rivest. This code was
* written by Colin Plumb in 1993, no copyright is claimed.
* This code is in the public domain; do with it what you wish.
*
* Equivalent code is available from RSA Data Security, Inc.
* This code has been tested against that, and is equivalent,
* except that you don't need to include two pages of legalese
* with every copy.
*
* To compute the message digest of a chunk of bytes, declare an
* MD5Context structure, pass it to MD5Init, call MD5Update as
* needed on buffers full of bytes, and then call MD5Final, which
* will fill a supplied 16-byte array with the digest.
*/
#include /* for memcpy() */
#include "md5.h"
#include "config.h"
#ifndef IS_BIG_ENDIAN
#define byteReverse(buf, len) /* Nothing */
#else
/*
* Note: this code is harmless on little-endian machines.
*/
void byteReverse(unsigned char *buf, unsigned longs)
{
uint32 t;
do {
t = (uint32) ((unsigned) buf[3] << 8 | buf[2]) << 16 |
((unsigned) buf[1] << 8 | buf[0]);
*(uint32 *) buf = t;
buf += 4;
} while (--longs);
}
#endif
/*
* Start MD5 accumulation. Set bit count to 0 and buffer to mysterious
* initialization constants.
*/
void MD5Init(struct MD5Context *ctx)
{
ctx->buf[0] = 0x67452301;
ctx->buf[1] = 0xefcdab89;
ctx->buf[2] = 0x98badcfe;
ctx->buf[3] = 0x10325476;
ctx->bits[0] = 0;
ctx->bits[1] = 0;
}
/*
* Update context to reflect the concatenation of another buffer full
* of bytes.
*/
void MD5Update(struct MD5Context *ctx, unsigned char *buf,
unsigned int len)
{
uint32 t;
/* Update bitcount */
t = ctx->bits[0];
if ((ctx->bits[0] = t + ((uint32) len << 3)) < t)
ctx->bits[1]++; /* Carry from low to high */
ctx->bits[1] += len >> 29;
t = (t >> 3) & 0x3f; /* Bytes already in shsInfo->data */
/* Handle any leading odd-sized chunks */
if (t) {
unsigned char *p = (unsigned char *) ctx->in + t;
t = 64 - t;
if (len < t) {
memcpy(p, buf, len);
return;
}
memcpy(p, buf, t);
byteReverse(ctx->in, 16);
MD5Transform(ctx->buf, (uint32 *) ctx->in);
buf += t;
len -= t;
}
/* Process data in 64-byte chunks */
while (len >= 64) {
memcpy(ctx->in, buf, 64);
byteReverse(ctx->in, 16);
MD5Transform(ctx->buf, (uint32 *) ctx->in);
buf += 64;
len -= 64;
}
/* Handle any remaining bytes of data. */
memcpy(ctx->in, buf, len);
}
/*
* Final wrapup - pad to 64-byte boundary with the bit pattern
* 1 0* (64-bit count of bits processed, MSB-first)
*/
void MD5Final(unsigned char digest[16], struct MD5Context *ctx)
{
unsigned count;
unsigned char *p;
/* Compute number of bytes mod 64 */
count = (ctx->bits[0] >> 3) & 0x3F;
/* Set the first char of padding to 0x80. This is safe since there is
always at least one byte free */
p = ctx->in + count;
*p++ = 0x80;
/* Bytes of padding needed to make 64 bytes */
count = 64 - 1 - count;
/* Pad out to 56 mod 64 */
if (count < 8) {
/* Two lots of padding: Pad the first block to 64 bytes */
memset(p, 0, count);
byteReverse(ctx->in, 16);
MD5Transform(ctx->buf, (uint32 *) ctx->in);
/* Now fill the next block with 56 bytes */
memset(ctx->in, 0, 56);
} else {
/* Pad block to 56 bytes */
memset(p, 0, count - 8);
}
byteReverse(ctx->in, 14);
/* Append length in bits and transform */
((uint32 *) ctx->in)[14] = ctx->bits[0];
((uint32 *) ctx->in)[15] = ctx->bits[1];
MD5Transform(ctx->buf, (uint32 *) ctx->in);
byteReverse((unsigned char *) ctx->buf, 4);
memcpy(digest, ctx->buf, 16);
memset(ctx, 0, sizeof(ctx)); /* In case it's sensitive */
}
/* The four core functions - F1 is optimized somewhat */
/* #define F1(x, y, z) (x & y | ~x & z) */
#define F1(x, y, z) (z ^ (x & (y ^ z)))
#define F2(x, y, z) F1(z, x, y)
#define F3(x, y, z) (x ^ y ^ z)
#define F4(x, y, z) (y ^ (x | ~z))
/* This is the central step in the MD5 algorithm. */
#define MD5STEP(f, w, x, y, z, data, s) \
( w += f(x, y, z) + data, w = w<>(32-s), w += x )
/*
* The core of the MD5 algorithm, this alters an existing MD5 hash to
* reflect the addition of 16 longwords of new data. MD5Update blocks
* the data and converts bytes into longwords for this routine.
*/
void MD5Transform(uint32 buf[4], uint32 in[16])
{
register uint32 a, b, c, d;
a = buf[0];
b = buf[1];
c = buf[2];
d = buf[3];
MD5STEP(F1, a, b, c, d, in[0] + 0xd76aa478, 7);
MD5STEP(F1, d, a, b, c, in[1] + 0xe8c7b756, 12);
MD5STEP(F1, c, d, a, b, in[2] + 0x242070db, 17);
MD5STEP(F1, b, c, d, a, in[3] + 0xc1bdceee, 22);
MD5STEP(F1, a, b, c, d, in[4] + 0xf57c0faf, 7);
MD5STEP(F1, d, a, b, c, in[5] + 0x4787c62a, 12);
MD5STEP(F1, c, d, a, b, in[6] + 0xa8304613, 17);
MD5STEP(F1, b, c, d, a, in[7] + 0xfd469501, 22);
MD5STEP(F1, a, b, c, d, in[8] + 0x698098d8, 7);
MD5STEP(F1, d, a, b, c, in[9] + 0x8b44f7af, 12);
MD5STEP(F1, c, d, a, b, in[10] + 0xffff5bb1, 17);
MD5STEP(F1, b, c, d, a, in[11] + 0x895cd7be, 22);
MD5STEP(F1, a, b, c, d, in[12] + 0x6b901122, 7);
MD5STEP(F1, d, a, b, c, in[13] + 0xfd987193, 12);
MD5STEP(F1, c, d, a, b, in[14] + 0xa679438e, 17);
MD5STEP(F1, b, c, d, a, in[15] + 0x49b40821, 22);
MD5STEP(F2, a, b, c, d, in[1] + 0xf61e2562, 5);
MD5STEP(F2, d, a, b, c, in[6] + 0xc040b340, 9);
MD5STEP(F2, c, d, a, b, in[11] + 0x265e5a51, 14);
MD5STEP(F2, b, c, d, a, in[0] + 0xe9b6c7aa, 20);
MD5STEP(F2, a, b, c, d, in[5] + 0xd62f105d, 5);
MD5STEP(F2, d, a, b, c, in[10] + 0x02441453, 9);
MD5STEP(F2, c, d, a, b, in[15] + 0xd8a1e681, 14);
MD5STEP(F2, b, c, d, a, in[4] + 0xe7d3fbc8, 20);
MD5STEP(F2, a, b, c, d, in[9] + 0x21e1cde6, 5);
MD5STEP(F2, d, a, b, c, in[14] + 0xc33707d6, 9);
MD5STEP(F2, c, d, a, b, in[3] + 0xf4d50d87, 14);
MD5STEP(F2, b, c, d, a, in[8] + 0x455a14ed, 20);
MD5STEP(F2, a, b, c, d, in[13] + 0xa9e3e905, 5);
MD5STEP(F2, d, a, b, c, in[2] + 0xfcefa3f8, 9);
MD5STEP(F2, c, d, a, b, in[7] + 0x676f02d9, 14);
MD5STEP(F2, b, c, d, a, in[12] + 0x8d2a4c8a, 20);
MD5STEP(F3, a, b, c, d, in[5] + 0xfffa3942, 4);
MD5STEP(F3, d, a, b, c, in[8] + 0x8771f681, 11);
MD5STEP(F3, c, d, a, b, in[11] + 0x6d9d6122, 16);
MD5STEP(F3, b, c, d, a, in[14] + 0xfde5380c, 23);
MD5STEP(F3, a, b, c, d, in[1] + 0xa4beea44, 4);
MD5STEP(F3, d, a, b, c, in[4] + 0x4bdecfa9, 11);
MD5STEP(F3, c, d, a, b, in[7] + 0xf6bb4b60, 16);
MD5STEP(F3, b, c, d, a, in[10] + 0xbebfbc70, 23);
MD5STEP(F3, a, b, c, d, in[13] + 0x289b7ec6, 4);
MD5STEP(F3, d, a, b, c, in[0] + 0xeaa127fa, 11);
MD5STEP(F3, c, d, a, b, in[3] + 0xd4ef3085, 16);
MD5STEP(F3, b, c, d, a, in[6] + 0x04881d05, 23);
MD5STEP(F3, a, b, c, d, in[9] + 0xd9d4d039, 4);
MD5STEP(F3, d, a, b, c, in[12] + 0xe6db99e5, 11);
MD5STEP(F3, c, d, a, b, in[15] + 0x1fa27cf8, 16);
MD5STEP(F3, b, c, d, a, in[2] + 0xc4ac5665, 23);
MD5STEP(F4, a, b, c, d, in[0] + 0xf4292244, 6);
MD5STEP(F4, d, a, b, c, in[7] + 0x432aff97, 10);
MD5STEP(F4, c, d, a, b, in[14] + 0xab9423a7, 15);
MD5STEP(F4, b, c, d, a, in[5] + 0xfc93a039, 21);
MD5STEP(F4, a, b, c, d, in[12] + 0x655b59c3, 6);
MD5STEP(F4, d, a, b, c, in[3] + 0x8f0ccc92, 10);
MD5STEP(F4, c, d, a, b, in[10] + 0xffeff47d, 15);
MD5STEP(F4, b, c, d, a, in[1] + 0x85845dd1, 21);
MD5STEP(F4, a, b, c, d, in[8] + 0x6fa87e4f, 6);
MD5STEP(F4, d, a, b, c, in[15] + 0xfe2ce6e0, 10);
MD5STEP(F4, c, d, a, b, in[6] + 0xa3014314, 15);
MD5STEP(F4, b, c, d, a, in[13] + 0x4e0811a1, 21);
MD5STEP(F4, a, b, c, d, in[4] + 0xf7537e82, 6);
MD5STEP(F4, d, a, b, c, in[11] + 0xbd3af235, 10);
MD5STEP(F4, c, d, a, b, in[2] + 0x2ad7d2bb, 15);
MD5STEP(F4, b, c, d, a, in[9] + 0xeb86d391, 21);
buf[0] += a;
buf[1] += b;
buf[2] += c;
buf[3] += d;
}
/* EOF */
qsf-1.2.7/src/db/ 0000755 0000764 0000764 00000000000 10665021416 011277 5 ustar aw aw qsf-1.2.7/src/db/obtree.c 0000644 0000764 0000764 00000042370 10664772303 012740 0 ustar aw aw /*
* Old binary tree database backend.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "database.h"
#include "log.h"
#ifdef USING_OBTREE
#include
#include
#include
#include
#include
#include
#include
#include
#ifdef HAVE_FCNTL
#include
#endif
#define BT_RECORD_SIZE sizeof(struct bt_record)
#define BT_TOKEN_MAX 36
typedef unsigned long bt_ul;
struct bt_record { /* single binary tree record */
bt_ul lower; /* offset of "lower" token */
bt_ul higher; /* offset of "higher" token */
unsigned char token[BT_TOKEN_MAX]; /* RATS: ignore - 0-term. token */
long data[2]; /* token data (i.e. the counts) */
};
typedef struct bt_record *bt_record_t;
struct qdbint_s { /* database state */
int fd; /* file descriptor of database file */
bt_ul size; /* total size of database */
#ifdef HAVE_FCNTL
int lockcount; /* number of times lock asked for */
int locktype; /* type of lock to use (read or write) */
#endif
int gotheadoffs; /* flag, set once head offset read */
bt_ul head_offset; /* offset of head record of tree */
};
static char *bt_lasterror = "";
#ifdef HAVE_FCNTL
/*
* Obtain / release a read or write lock on the database. Returns nonzero on
* error, and blocks until a lock can be obtained.
*/
static int dbbt_lock(qdbint_t db, int lock_type)
{
int ret;
ret = qdb_int__lock(db->fd, lock_type, &(db->lockcount));
if (ret != 0) {
bt_lasterror = strerror(errno);
return 1;
}
return 0;
}
#endif /* HAVE_FCNTL */
/*
* Analogue of fread().
*/
static int dbbt_chunkread(void *ptr, int size, int nmemb, int fd)
{
int numread, togo, got;
for (numread = 0; nmemb > 0; nmemb--, numread++) {
for (togo = size; togo > 0;) {
got = read(fd, ptr, togo); /* RATS: ignore (OK) */
if (got <= 0)
return numread;
togo -= got;
ptr = (void *) (((char *) ptr) + got);
}
}
return numread;
}
/*
* Analogue of fwrite().
*/
static int dbbt_chunkwrite(void *ptr, int size, int nmemb, int fd)
{
int numwritten, togo, written;
for (numwritten = 0; nmemb > 0; nmemb--, numwritten++) {
for (togo = size; togo > 0;) {
written = write(fd, ptr, togo);
if (written <= 0)
return numwritten;
togo -= written;
ptr = (void *) (((char *) ptr) + written);
}
}
return numwritten;
}
/*
* Read a record from the database at the given offset into the given record
* structure, returning nonzero on failure.
*/
static int dbbt_read_record(qdbint_t db, bt_ul offset, bt_record_t record)
{
int got;
if (lseek(db->fd, offset, SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
return 1;
}
got = dbbt_chunkread(record, BT_RECORD_SIZE, 1, db->fd);
if (got < 1) {
bt_lasterror = strerror(errno);
return 1;
}
record->token[BT_TOKEN_MAX - 1] = 0;
return 0;
}
/*
* Write a record to the database at the given offset, returning nonzero on
* failure.
*/
static int dbbt_write_record(qdbint_t db, bt_ul offset, bt_record_t record)
{
if (lseek(db->fd, offset, SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
return 1;
}
if (dbbt_chunkwrite(record, BT_RECORD_SIZE, 1, db->fd) < 1) {
bt_lasterror = strerror(errno);
return 1;
}
return 0;
}
/*
* Find the given token in the database and fill in the given record
* structure if found, also filling in the offset of the record (or 0 if not
* found) and the offset of the parent record (0 if none).
*
* Returns -1 if the token looked for was "lower" than its parent or +1 if
* "higher", or 0 if there was no parent record (i.e. this is the first
* record).
*/
static int dbbt_find_token(qdbint_t db, qdb_datum key, bt_record_t record,
bt_ul * offset, bt_ul * parent)
{
int hilow = 0;
int x;
*offset = 0;
*parent = 0;
if (db == NULL)
return 0;
if (db->size < 2 * sizeof(long))
return 0;
if (!db->gotheadoffs) {
lseek(db->fd, 0, SEEK_SET);
dbbt_chunkread(offset, sizeof(*offset), 1, db->fd);
db->head_offset = *offset;
db->gotheadoffs = 1;
} else {
*offset = db->head_offset;
}
while (*offset > 0) {
int len;
if (dbbt_read_record(db, *offset, record)) {
*offset = 0;
break;
}
x = strncmp((char *) (record->token), (char *) (key.data),
key.size);
len = strlen((char *) (record->token));
if (len < key.size) {
x = -1;
} else if (len > key.size) {
x = 1;
}
if (x == 0) {
return hilow;
} else if (x < 0) {
*parent = *offset;
hilow = -1;
*offset = record->lower;
} else {
*parent = *offset;
hilow = 1;
*offset = record->higher;
}
}
return hilow;
}
/*
* Return nonzero if the given file is of this database type.
*/
int qdb_obtree_identify(const char *file)
{
if (file == NULL)
return 0;
if (strncasecmp(file, "obtree:", 7) == 0)
return 1;
return 0;
}
/*
* Open the given database in the given way (new database, read-only, or
* read-write); return a qdbint_t or NULL on error.
*/
qdbint_t qdb_obtree_open(const char *file, qdb_open_t method)
{
qdbint_t db;
int fd = -1;
#ifdef HAVE_FCNTL
int locktype = F_RDLCK;
#endif
int forced_type = 0;
if (strncasecmp(file, "obtree:", 7) == 0) {
file += 7;
forced_type = 1;
}
switch (method) {
case QDB_NEW:
fd = open(file, /* RATS: ignore (no race) */
O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
if (fd < 0)
bt_lasterror = strerror(errno);
#ifdef HAVE_FCNTL
locktype = F_WRLCK;
#endif
break;
case QDB_READONLY:
fd = open(file, /* RATS: ignore (no race) */
O_RDONLY);
if (fd < 0)
bt_lasterror = strerror(errno);
break;
case QDB_READWRITE:
fd = open(file, /* RATS: ignore (no race) */
O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
if (fd < 0)
bt_lasterror = strerror(errno);
#ifdef HAVE_FCNTL
locktype = F_WRLCK;
#endif
break;
default:
break;
}
if (fd < 0)
return NULL;
db = calloc(1, sizeof(*db));
if (db == NULL) {
bt_lasterror = strerror(errno);
close(fd);
return NULL;
}
db->size = lseek(fd, 0, SEEK_END);
db->fd = fd;
#ifdef HAVE_FCNTL
db->locktype = locktype;
if (dbbt_lock(db, locktype)) {
close(fd);
free(db);
return NULL;
}
#endif
/*
* Complain at the user to stop using this backend if it wasn't
* specifically chosen.
*/
if (!forced_type) {
log_add(0, "%s",
_("WARNING: Using deprecated obtree backend!"));
log_add(0, "%s",
_("WARNING: Dump, delete, and restore your"));
log_add(0, "%s",
_("WARNING: databases to upgrade them to the"));
log_add(0, "%s",
_("WARNING: new format and stop this warning."));
} else {
log_add(1, "%s",
_("warning: obtree backend is deprecated"));
}
/*
* If the database has zero size and we're writing to it, assume
* it's new and don't complain.
*/
if ((db->size == 0) && (method != QDB_READONLY))
return db;
/*
* We now do some simple checks to make sure that the file is a
* database of the format we're expecting.
*/
/*
* If it's shorter than 2 long ints, it's not a valid database.
*/
if (db->size < 2 * sizeof(long)) {
bt_lasterror = _("invalid database (too small)");
close(fd);
free(db);
return NULL;
}
/*
* If its size, discounting the two longs at the start, isn't a
* multiple of our record size, it's not a valid database.
*/
if (((db->size - 2 * sizeof(long)) % BT_RECORD_SIZE) != 0) {
bt_lasterror = _("invalid database (irregular size)");
close(fd);
free(db);
return NULL;
}
/*
* If the first long int (the "head offset") is larger than the size
* of the file, this isn't a valid database.
*/
{
bt_ul offset = db->size + 1;
lseek(fd, 0, SEEK_SET);
dbbt_chunkread(&offset, sizeof(offset), 1, fd);
if (offset > db->size) {
bt_lasterror =
_("invalid database (bad head offset)");
close(fd);
free(db);
return NULL;
}
lseek(fd, 0, SEEK_SET);
}
return db;
}
/*
* Close the given database.
*/
void qdb_obtree_close(qdbint_t db)
{
if (db == NULL)
return;
#ifdef HAVE_FCNTL
while (db->lockcount > 0)
dbbt_lock(db, F_UNLCK);
#endif
close(db->fd);
free(db);
}
/*
* Fetch a value from the database. The datum returned needs its val.data
* free()ing after use. If val.data is NULL, no value was found for the
* given key.
*/
qdb_datum qdb_obtree_fetch(qdbint_t db, qdb_datum key)
{
struct bt_record record;
unsigned long offset, parent;
qdb_datum val;
val.data = NULL;
val.size = 0;
if (db == NULL)
return val;
if (key.size >= BT_TOKEN_MAX)
key.size = BT_TOKEN_MAX - 1;
dbbt_find_token(db, key, &record, &offset, &parent);
if (offset == 0)
return val;
val.size = 2 * sizeof(long);
val.data = calloc(1, val.size);
if (val.data == NULL) {
val.size = 0;
return val;
}
((long *) val.data)[0] = record.data[0];
((long *) val.data)[1] = record.data[1];
return val;
}
/*
* Return a file descriptor for the given database, or -1 on error.
*/
int qdb_obtree_fd(qdbint_t db)
{
if (db == NULL)
return -1;
return db->fd;
}
/*
* Store the given key with the given value into the database, replacing any
* existing value for that key. Returns nonzero on error.
*/
int qdb_obtree_store(qdbint_t db, qdb_datum key, qdb_datum val)
{
struct bt_record record, head;
unsigned long offset, parent, nextfree;
int x;
if (db == NULL)
return 1;
memset(&head, 0, BT_RECORD_SIZE);
memset(&record, 0, BT_RECORD_SIZE);
if (key.size >= BT_TOKEN_MAX)
key.size = BT_TOKEN_MAX - 1;
x = dbbt_find_token(db, key, &record, &offset, &parent);
memcpy(record.token, key.data, key.size);
record.token[key.size] = 0;
record.data[0] = ((long *) val.data)[0];
record.data[1] = ((long *) val.data)[1];
qdb_int__sig_block();
/*
* Record exists - overwrite it.
*/
if (offset > 0) {
if (dbbt_write_record(db, offset, &record)) {
qdb_int__sig_unblock();
return 1;
}
qdb_int__sig_unblock();
return 0;
}
record.lower = 0;
record.higher = 0;
/*
* Database has just been created, so fill in the header and write
* this record as the first one.
*/
if (db->size <= sizeof(offset)) {
if (lseek(db->fd, 0, SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
offset = 2 * sizeof(offset);
if (dbbt_chunkwrite(&offset, sizeof(offset), 1, db->fd) <
1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
db->gotheadoffs = 0;
offset = 0;
if (dbbt_chunkwrite(&offset, sizeof(offset), 1, db->fd) <
1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
head.lower = (2 * sizeof(offset)) + BT_RECORD_SIZE;
if (dbbt_chunkwrite(&head, BT_RECORD_SIZE, 1, db->fd) < 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if (dbbt_chunkwrite(&record, BT_RECORD_SIZE, 1, db->fd) <
1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
db->size = lseek(db->fd, 0, SEEK_CUR);
qdb_int__sig_unblock();
return 0;
}
/*
* Get offset of next free space block.
*/
if (lseek(db->fd, sizeof(offset), SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if (dbbt_chunkread(&offset, sizeof(offset), 1, db->fd) < 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
offset &= 0x7FFFFFFF;
/*
* If offset is 0 or we can't read from that offset, it's a new
* block at the end of the file, otherwise we take the next free
* offset from there and store it in the core free pointer, and then
* use that free offset.
*/
if (lseek(db->fd, offset, SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if ((offset == 0)
|| (dbbt_chunkread(&nextfree, sizeof(nextfree), 1, db->fd) < 1)
) {
offset = lseek(db->fd, 0, SEEK_END);
nextfree = 0x80000000;
}
if (lseek(db->fd, sizeof(offset), SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if (dbbt_chunkwrite(&nextfree, sizeof(nextfree), 1, db->fd) < 0) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if (dbbt_write_record(db, offset, &record)) {
qdb_int__sig_unblock();
return 1;
}
/*
* Now attach the new record to its parent, if applicable.
*/
if (parent > 0) {
if (dbbt_read_record(db, parent, &record)) {
qdb_int__sig_unblock();
return 1;
}
if (x < 0) {
record.lower = offset;
} else {
record.higher = offset;
}
if (dbbt_write_record(db, parent, &record)) {
qdb_int__sig_unblock();
return 1;
}
}
qdb_int__sig_unblock();
return 0;
}
/*
* Return the "first" key in the database, suitable for using with repeated
* calls to qdb_nextkey() to walk through every key in the database.
*/
qdb_datum qdb_obtree_firstkey(qdbint_t db)
{
struct bt_record record;
qdb_datum key;
key.data = NULL;
key.size = 0;
if (lseek(db->fd, 2 * sizeof(unsigned long), SEEK_SET) ==
(off_t) - 1)
return key;
if (dbbt_chunkread(&record, BT_RECORD_SIZE, 1, db->fd) < 1) {
return key;
}
record.token[BT_TOKEN_MAX - 1] = 0;
key.data = (unsigned char *) strdup((char *) (record.token));
key.size = strlen((char *) (record.token));
return key;
}
/*
* Return the "next" key in the database, or key.data=NULL when all keys
* have been returned.
*/
qdb_datum qdb_obtree_nextkey(qdbint_t db, qdb_datum key)
{
struct bt_record record;
unsigned long offset, parent;
qdb_datum newkey;
newkey.data = NULL;
newkey.size = 0;
if (key.data == NULL) {
return newkey;
}
dbbt_find_token(db, key, &record, &offset, &parent);
if (offset < 1) {
return newkey;
}
if (lseek(db->fd, offset + BT_RECORD_SIZE, SEEK_SET) ==
(off_t) - 1) {
return newkey;
}
do {
if (dbbt_chunkread(&record, BT_RECORD_SIZE, 1, db->fd) < 1) {
return newkey;
}
} while (record.lower & 0x80000000);
record.token[BT_TOKEN_MAX - 1] = 0;
newkey.data = (unsigned char *) strdup((char *) (record.token));
newkey.size = strlen((char *) (record.token));
return newkey;
}
/*
* Reposition the given record in the binary tree, by finding an existing
* record to link it to. Returns nonzero on error.
*/
static int qdb_obtree_delete__reposition(qdbint_t db, unsigned long offset,
char *token)
{
struct bt_record record;
unsigned long offs, parent;
qdb_datum key;
int x;
key.data = (unsigned char *) token;
key.size = strlen(token);
x = dbbt_find_token(db, key, &record, &offs, &parent);
if (parent < 1)
return 1;
if (dbbt_read_record(db, parent, &record))
return 1;
if (x < 0) {
record.lower = offset;
} else {
record.higher = offset;
}
if (dbbt_write_record(db, parent, &record))
return 1;
return 0;
}
/*
* Delete the given key from the database. Returns nonzero on error.
*/
int qdb_obtree_delete(qdbint_t db, qdb_datum key)
{
struct bt_record record, recparent, reclower, rechigher;
unsigned long offset, parent, nextfree;
int x;
if (db == NULL)
return 1;
if (key.size < 1)
return 1;
x = dbbt_find_token(db, key, &record, &offset, &parent);
if (offset < 1)
return 1;
/*
* Get a copy of the lower and higher records, if any.
*/
if (record.lower > 0) {
if (dbbt_read_record(db, record.lower, &reclower))
return 1;
}
if (record.higher > 0) {
if (dbbt_read_record(db, record.higher, &rechigher))
return 1;
}
qdb_int__sig_block();
/*
* Remove the link to this record from its parent.
*/
if (parent > 0) {
if (dbbt_read_record(db, parent, &recparent)) {
qdb_int__sig_unblock();
return 1;
}
if (x < 0) {
recparent.lower = 0;
} else {
recparent.higher = 0;
}
if (dbbt_write_record(db, parent, &recparent)) {
qdb_int__sig_unblock();
return 1;
}
}
/*
* Re-attach any children of this record to the database.
*/
if (record.lower > 0)
qdb_obtree_delete__reposition(db, record.lower,
(char *) (reclower.token));
if (record.higher > 0)
qdb_obtree_delete__reposition(db, record.higher,
(char *) (rechigher.token));
/*
* Now we add this record's offset to the head of the "free records"
* linked list.
*/
if (lseek(db->fd, sizeof(offset), SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
nextfree = 0x80000000;
if (dbbt_chunkread(&nextfree, sizeof(nextfree), 1, db->fd) < 0) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if (lseek(db->fd, sizeof(offset), SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
offset |= 0x80000000;
if (dbbt_chunkwrite(&offset, sizeof(offset), 1, db->fd) < 0) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
offset &= 0x7FFFFFFF;
if (lseek(db->fd, offset, SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if (dbbt_chunkwrite(&nextfree, sizeof(nextfree), 1, db->fd) < 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
qdb_int__sig_unblock();
return 0;
}
/*
* Temporarily release the lock on the database.
*/
void qdb_obtree_unlock(qdbint_t db)
{
#ifdef HAVE_FCNTL
dbbt_lock(db, F_UNLCK);
#endif
}
/*
* Reassert the lock on the database.
*/
void qdb_obtree_relock(qdbint_t db)
{
#ifdef HAVE_FCNTL
dbbt_lock(db, db->locktype);
#endif
db->gotheadoffs = 0;
}
/*
* Return a string describing the last database error to occur.
*/
char *qdb_obtree_error(void)
{
return bt_lasterror;
}
#endif /* USING_OBTREE */
/* EOF */
qsf-1.2.7/src/db/list.c 0000644 0000764 0000764 00000036172 10664772303 012436 0 ustar aw aw /*
* In-memory flat list database backend.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "database.h"
#include "log.h"
#ifdef USING_LIST
#include
#include
#include
#include
#include
#include
#include
#include
#include
#ifdef HAVE_FCNTL
#include
#endif
#ifdef HAVE_UTIME
#include
#include
#endif
#include
#define ML_TOKEN_MAX 36 /* maximum token length */
#define ML_INCSIZE 10000 /* increments by which array size grows */
#define ML_ADDITIONBUF 1000 /* size of addition buffer */
struct mh_record_s {
char token[ML_TOKEN_MAX]; /* RATS: ignore - 0-term. token */
uint32_t data[3]; /* token data (i.e. the counts) */
};
struct qdbint_s {
int fd;
long size;
#ifdef HAVE_FCNTL
int lockcount; /* number of times lock asked for */
int locktype; /* type of lock to use (read or write) */
#endif
char *filename; /* filename of database */
int modified; /* flag, set if db modified at all */
int restoring; /* flag, set if no qsort on store */
struct mh_record_s *array; /* array holding all list entries */
long array_len; /* highest used index + 1 */
long array_alloced; /* size of array */
long walk_index; /* current index for ..._nextkey() */
struct mh_record_s *addition; /* array holding all list entries */
long addition_len; /* highest used index + 1 */
};
static char *mh_lasterror = "";
#ifdef HAVE_FCNTL
/*
* Obtain / release a read or write lock on the database. Returns nonzero on
* error, and blocks until a lock can be obtained.
*/
static int dbl_lock(qdbint_t db, int lock_type)
{
int ret;
ret = qdb_int__lock(db->fd, lock_type, &(db->lockcount));
if (ret != 0) {
mh_lasterror = strerror(errno);
return 1;
}
return 0;
}
#endif /* HAVE_FCNTL */
/*
* Analogue of fread().
*/
static int dbl_chunkread(void *ptr, int size, int nmemb, int fd)
{
int numread, togo, got;
for (numread = 0; nmemb > 0; nmemb--, numread++) {
for (togo = size; togo > 0;) {
got = read(fd, ptr, togo); /* RATS: ignore (OK) */
if (got <= 0)
return numread;
togo -= got;
ptr = (void *) (((char *) ptr) + got);
}
}
return numread;
}
/*
* Analogue of fwrite().
*/
static int dbl_chunkwrite(void *ptr, int size, int nmemb, int fd)
{
int numwritten, togo, written;
for (numwritten = 0; nmemb > 0; nmemb--, numwritten++) {
for (togo = size; togo > 0;) {
written = write(fd, ptr, togo);
if (written <= 0)
return numwritten;
togo -= written;
ptr = (void *) (((char *) ptr) + written);
}
}
return numwritten;
}
/*
* Read a single long integer from the database and return it, converting
* from network byte order.
*/
static long dbl_longread(qdbint_t db)
{
uint32_t val;
val = 0;
dbl_chunkread(&val, sizeof(val), 1, db->fd);
return ntohl(val);
}
/*
* Write a single long integer to the database, converting to network byte
* order. Returns nonzero on error.
*/
static int dbl_longwrite(qdbint_t db, long val)
{
uint32_t newval;
newval = htonl(val);
return dbl_chunkwrite(&newval, sizeof(newval), 1,
db->fd) == 1 ? 0 : -1;
}
/*
* Comparison function to compare two database records' tokens.
*/
static int dbl_compare(const void *a, const void *b)
{
if (a == NULL)
return -1;
if (b == NULL)
return 1;
if (((struct mh_record_s *) a)->token[0] == 0)
return 1;
if (((struct mh_record_s *) b)->token[0] == 0)
return -1;
return strncmp(((struct mh_record_s *) a)->token,
((struct mh_record_s *) b)->token, ML_TOKEN_MAX);
}
/*
* Return a file descriptor for the given database, or -1 on error.
*/
int qdb_list_fd(qdbint_t db)
{
if (db == NULL)
return -1;
return db->fd;
}
/*
* Return nonzero if the given file is of this database type.
*/
int qdb_list_identify(const char *file)
{
int fd;
char buf[8]; /* RATS: ignore (checked) */
if (file == NULL)
return 0;
if (strncasecmp(file, "list:", 5) == 0)
return 1;
fd = open(file, O_RDONLY);
if (fd < 0)
return 0;
if (dbl_chunkread(buf, 4, 1, fd) < 1) {
close(fd);
return 0;
}
close(fd);
if (strncmp(buf, "QSF2", 4) == 0)
return 1;
return 0;
}
/*
* Open the given database in the given way (new database, read-only, or
* read-write); return a qdbint_t or NULL on error.
*/
qdbint_t qdb_list_open(const char *file, qdb_open_t method)
{
qdbint_t db;
int fd = -1;
#ifdef HAVE_FCNTL
int locktype = F_RDLCK;
#endif
if (strncasecmp(file, "list:", 5) == 0)
file += 5;
switch (method) {
case QDB_NEW:
fd = open(file, /* RATS: ignore (no race) */
O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
if (fd < 0)
mh_lasterror = strerror(errno);
#ifdef HAVE_FCNTL
locktype = F_WRLCK;
#endif
break;
case QDB_READONLY:
fd = open(file, /* RATS: ignore (no race) */
O_RDONLY);
if (fd < 0)
mh_lasterror = strerror(errno);
break;
case QDB_READWRITE:
fd = open(file, /* RATS: ignore (no race) */
O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
if (fd < 0)
mh_lasterror = strerror(errno);
#ifdef HAVE_FCNTL
locktype = F_WRLCK;
#endif
break;
default:
break;
}
if (fd < 0)
return NULL;
db = calloc(1, sizeof(*db));
if (db == NULL) {
mh_lasterror = strerror(errno);
close(fd);
return NULL;
}
db->filename = strdup(file);
db->fd = fd;
#ifdef HAVE_FCNTL
db->locktype = locktype;
#endif
db->modified = 0;
db->array = NULL;
db->array_len = 0;
db->array_alloced = 0;
db->addition_len = 0;
db->addition = calloc(ML_ADDITIONBUF, sizeof(struct mh_record_s));
if (db->addition == NULL) {
mh_lasterror = strerror(errno);
close(fd);
free(db->filename);
free(db);
return NULL;
}
#ifdef HAVE_FCNTL
if (dbl_lock(db, db->locktype)) {
close(db->fd);
if (db->filename)
free(db->filename);
if (db->addition)
free(db->addition);
free(db);
return NULL;
}
#endif
db->size = lseek(db->fd, 0, SEEK_END);
if (db->size == (off_t) - 1)
db->size = 0;
/*
* If this is a new database, and it's not readonly, write our
* header to it and return.
*/
if ((db->size <= 8) && (method != QDB_READONLY)) {
lseek(db->fd, 0, SEEK_SET);
dbl_chunkwrite("QSF2", 4, 1, db->fd);
dbl_longwrite(db, 0);
db->size = 8;
db->modified = 1;
return db;
}
lseek(db->fd, 4, SEEK_SET);
db->array_len = dbl_longread(db);
if ((db->array_len < 0) || (db->array_len > 10000000)) {
mh_lasterror =
_("invalid database (array size out of range)");
#ifdef HAVE_FCNTL
dbl_lock(db, F_UNLCK);
#endif
close(db->fd);
if (db->filename)
free(db->filename);
if (db->addition)
free(db->addition);
free(db);
return NULL;
}
db->array_alloced = db->array_len + ML_INCSIZE;
db->array = calloc(db->array_alloced, sizeof(struct mh_record_s));
if (db->array == NULL) {
mh_lasterror = strerror(errno);
#ifdef HAVE_FCNTL
dbl_lock(db, F_UNLCK);
#endif
close(db->fd);
if (db->filename)
free(db->filename);
if (db->addition)
free(db->addition);
free(db);
return NULL;
}
if (dbl_chunkread
(db->array, sizeof(struct mh_record_s), db->array_len,
db->fd) < db->array_len) {
mh_lasterror = _("invalid database (file truncated)");
#ifdef HAVE_FCNTL
dbl_lock(db, F_UNLCK);
#endif
close(db->fd);
if (db->filename)
free(db->filename);
free(db->array);
if (db->addition)
free(db->addition);
free(db);
return NULL;
}
qsort(db->array, db->array_len, sizeof(struct mh_record_s),
dbl_compare);
return db;
}
/*
* Extend the array by "amount" items. Returns nonzero on error.
*/
static int qdb_list__extendarray(qdbint_t db, long amount)
{
long origlen, origalloced;
void *ptr;
origlen = db->array_len;
db->array_len += amount;
if (db->array_len < db->array_alloced)
return 0;
origalloced = db->array_alloced;
while (db->array_alloced < db->array_len) {
db->array_alloced += ML_INCSIZE;
}
if (db->array == NULL) {
ptr =
calloc(db->array_alloced, sizeof(struct mh_record_s));
} else {
ptr = realloc(db->array, /* RATS: ignore (OK) */
db->array_alloced *
sizeof(struct mh_record_s));
}
if (ptr == NULL) {
mh_lasterror = strerror(errno);
db->array_len = origlen;
db->array_alloced = origalloced;
return 1;
}
db->array = ptr;
return 0;
}
/*
* Dump the addition list on to the end of the array, and re-sort the array.
* Returns nonzero on error.
*/
static int qdb_list__dumpaddition(qdbint_t db)
{
if (db->addition_len == 0)
return 0;
if (qdb_list__extendarray(db, db->addition_len))
return 1;
memcpy(&(db->array[db->array_len - db->addition_len]),
db->addition,
db->addition_len * sizeof(struct mh_record_s));
db->addition_len = 0;
if (!db->restoring) {
qsort(db->array, db->array_len, sizeof(struct mh_record_s),
dbl_compare);
}
return 0;
}
/*
* Save any changes that were made to the database. This function only
* stores the data, it doesn't free memory or update timestamps. Returns
* nonzero on error.
*/
static int qdb_list__savedb(qdbint_t db)
{
unsigned long newsize;
/*
* Append the addition list to the array.
*/
if (qdb_list__dumpaddition(db))
return 1;
/*
* Write the contents of the array first.
*/
lseek(db->fd, 8, SEEK_SET);
if (dbl_chunkwrite
(db->array, sizeof(struct mh_record_s), db->array_len,
db->fd) < db->array_len) {
return 1;
}
newsize = lseek(db->fd, 0, SEEK_CUR);
/*
* Now store the new size of the array.
*/
lseek(db->fd, 4, SEEK_SET);
if (dbl_longwrite(db, db->array_len))
return 1;
/*
* Finally, truncate the database (if it's got smaller).
*/
if (ftruncate(db->fd, newsize) != 0) {
log_add(0, "%s: failed to truncate database: %s",
db->filename, strerror(errno));
}
return 0;
}
/*
* Close the given database.
*/
void qdb_list_close(qdbint_t db)
{
if (db == NULL)
return;
if (db->modified) {
qdb_int__sig_block();
qdb_list__savedb(db);
qdb_int__sig_unblock();
}
if (db->array)
free(db->array);
if (db->addition)
free(db->addition);
#ifdef HAVE_FCNTL
while (db->lockcount > 0)
dbl_lock(db, F_UNLCK);
#endif
close(db->fd);
if (db->modified && db->filename) {
#ifdef HAVE_UTIME
struct utimbuf utb;
utb.actime = time(NULL);
utb.modtime = time(NULL);
utime(db->filename, &utb);
#endif
}
if (db->filename)
free(db->filename);
free(db);
}
/*
* Search the array for the given token. Returns a pointer to the array
* entry or NULL on failure.
*/
static struct mh_record_s *dbl_search(qdbint_t db, qdb_datum key)
{
struct mh_record_s search;
struct mh_record_s *val;
if (db == NULL)
return NULL;
if (key.data == NULL)
return NULL;
memset(search.token, 0, ML_TOKEN_MAX);
strncpy(search.token, (char *) (key.data),
(key.size > ML_TOKEN_MAX) ? ML_TOKEN_MAX : key.size);
if (db->addition_len > 0) {
val = bsearch((void *) (&search), (void *) db->addition,
db->addition_len, sizeof(struct mh_record_s),
dbl_compare);
if (val != NULL)
return val;
}
if (db->array == NULL)
return NULL;
return bsearch((void *) (&search), (void *) db->array,
db->array_len, sizeof(struct mh_record_s),
dbl_compare);
}
/*
* Fetch a value from the database. The datum returned needs its val.data
* free()ing after use. If val.data is NULL, no value was found for the
* given key.
*/
qdb_datum qdb_list_fetch(qdbint_t db, qdb_datum key)
{
struct mh_record_s *result;
qdb_datum val;
val.data = NULL;
val.size = 0;
result = dbl_search(db, key);
if (result == NULL)
return val;
val.size = 3 * sizeof(long);
val.data = calloc(1, val.size);
if (val.data == NULL) {
val.size = 0;
return val;
}
((long *) val.data)[0] = ntohl(result->data[0]);
((long *) val.data)[1] = ntohl(result->data[1]);
((long *) val.data)[2] = ntohl(result->data[2]);
return val;
}
/*
* Store the given key with the given value into the database, replacing any
* existing value for that key. Returns nonzero on error.
*/
int qdb_list_store(qdbint_t db, qdb_datum key, qdb_datum val)
{
struct mh_record_s *result;
if (db == NULL)
return 1;
if ((key.data == NULL) || (val.data == NULL))
return 1;
db->modified = 1;
if (db->restoring) {
result = NULL;
} else {
result = dbl_search(db, key);
}
if (result != NULL) {
result->data[0] = htonl(((long *) val.data)[0]);
result->data[1] = htonl(((long *) val.data)[1]);
result->data[2] = htonl(((long *) val.data)[2]);
} else {
db->addition_len++;
result = &(db->addition[db->addition_len - 1]);
memset(result, 0, sizeof(struct mh_record_s));
strncpy(result->token, (char *) (key.data),
(key.size >
ML_TOKEN_MAX) ? ML_TOKEN_MAX : key.size);
result->data[0] = htonl(((long *) val.data)[0]);
result->data[1] = htonl(((long *) val.data)[1]);
result->data[2] = htonl(((long *) val.data)[2]);
if (db->addition_len >= ML_ADDITIONBUF) {
if (qdb_list__dumpaddition(db))
return 1;
} else if (!db->restoring) {
qsort(db->addition, db->addition_len,
sizeof(struct mh_record_s), dbl_compare);
}
}
return 0;
}
/*
* Delete the given key from the database. Returns nonzero on error.
*/
int qdb_list_delete(qdbint_t db, qdb_datum key)
{
struct mh_record_s *result;
long src, dst;
if (db == NULL)
return 1;
if (key.data == NULL)
return 1;
if (qdb_list__dumpaddition(db))
return 1;
result = dbl_search(db, key);
if (result == NULL)
return 0;
db->modified = 1;
result->token[0] = 0;
for (src = 0, dst = 0; src < db->array_len; src++) {
if (db->array[src].token[0] == 0)
continue;
if (src != dst)
memcpy(&(db->array[dst]), &(db->array[src]),
sizeof(struct mh_record_s));
dst++;
}
db->array_len = dst;
return 0;
}
/*
* Return the "next" key in the database, or key.data=NULL when all keys
* have been returned.
*/
qdb_datum qdb_list_nextkey(qdbint_t db, qdb_datum key)
{
qdb_datum newkey;
char keystr[ML_TOKEN_MAX + 1]; /* RATS: ignore */
newkey.data = NULL;
newkey.size = 0;
db->walk_index++;
while ((db->walk_index < db->array_len)
&& (db->array[db->walk_index].token[0] == 0)
) {
db->walk_index++;
}
if (db->walk_index >= db->array_len)
return newkey;
if (db->array[db->walk_index].token[0] == 0)
return newkey;
memset(keystr, 0, sizeof(keystr));
strncpy(keystr, db->array[db->walk_index].token, ML_TOKEN_MAX);
newkey.data = (unsigned char *) strdup((char *) keystr);
newkey.size = strlen((char *) keystr);
return newkey;
}
/*
* Return the "first" key in the database, suitable for using with repeated
* calls to qdb_nextkey() to walk through every key in the database.
*/
qdb_datum qdb_list_firstkey(qdbint_t db)
{
qdb_datum key;
qdb_list__dumpaddition(db);
db->walk_index = -1;
key.data = NULL;
key.size = 0;
return qdb_list_nextkey(db, key);
}
/*
* Return a string describing the last database error to occur.
*/
char *qdb_list_error(void)
{
return mh_lasterror;
}
/*
* Tell the database that a restore operation is starting.
*/
void qdb_list_restore_start(qdbint_t db)
{
db->restoring = 1;
}
/*
* Tell the database that a restore operation is ending.
*/
void qdb_list_restore_end(qdbint_t db)
{
qdb_list__dumpaddition(db);
qsort(db->array, db->array_len, sizeof(struct mh_record_s),
dbl_compare);
db->restoring = 0;
}
#endif /* USING_LIST */
/* EOF */
qsf-1.2.7/src/db/main.c 0000644 0000764 0000764 00000032300 10664772303 012374 0 ustar aw aw /*
* Main entry point for database functions.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "database.h"
#include
#include
#include
#include
#ifdef HAVE_FCNTL
#include
#endif
#ifdef USING_OBTREE
int qdb_obtree_identify(const char *);
qdbint_t qdb_obtree_open(const char *, qdb_open_t);
int qdb_obtree_fd(qdbint_t);
void qdb_obtree_close(qdbint_t);
qdb_datum qdb_obtree_fetch(qdbint_t, qdb_datum);
int qdb_obtree_store(qdbint_t, qdb_datum, qdb_datum);
int qdb_obtree_delete(qdbint_t, qdb_datum);
qdb_datum qdb_obtree_firstkey(qdbint_t);
qdb_datum qdb_obtree_nextkey(qdbint_t, qdb_datum);
char *qdb_obtree_error(void);
void qdb_obtree_unlock(qdbint_t);
void qdb_obtree_relock(qdbint_t);
#endif /* USING_OBTREE */
#ifdef USING_BTREE
int qdb_btree_identify(const char *);
qdbint_t qdb_btree_open(const char *, qdb_open_t);
int qdb_btree_fd(qdbint_t);
void qdb_btree_close(qdbint_t);
qdb_datum qdb_btree_fetch(qdbint_t, qdb_datum);
int qdb_btree_store(qdbint_t, qdb_datum, qdb_datum);
int qdb_btree_delete(qdbint_t, qdb_datum);
qdb_datum qdb_btree_firstkey(qdbint_t);
qdb_datum qdb_btree_nextkey(qdbint_t, qdb_datum);
void qdb_btree_optimise(qdbint_t);
char *qdb_btree_error(void);
void qdb_btree_unlock(qdbint_t);
void qdb_btree_relock(qdbint_t);
#endif /* USING_BTREE */
#ifdef USING_LIST
int qdb_list_identify(const char *);
qdbint_t qdb_list_open(const char *, qdb_open_t);
int qdb_list_fd(qdbint_t);
void qdb_list_close(qdbint_t);
qdb_datum qdb_list_fetch(qdbint_t, qdb_datum);
int qdb_list_store(qdbint_t, qdb_datum, qdb_datum);
int qdb_list_delete(qdbint_t, qdb_datum);
qdb_datum qdb_list_firstkey(qdbint_t);
qdb_datum qdb_list_nextkey(qdbint_t, qdb_datum);
char *qdb_list_error(void);
void qdb_list_restore_start(qdbint_t);
void qdb_list_restore_end(qdbint_t);
#endif /* USING_LIST */
#ifdef USING_GDBM
int qdb_gdbm_identify(const char *);
qdbint_t qdb_gdbm_open(const char *, qdb_open_t);
int qdb_gdbm_fd(qdbint_t);
void qdb_gdbm_close(qdbint_t);
qdb_datum qdb_gdbm_fetch(qdbint_t, qdb_datum);
int qdb_gdbm_store(qdbint_t, qdb_datum, qdb_datum);
int qdb_gdbm_delete(qdbint_t, qdb_datum);
qdb_datum qdb_gdbm_firstkey(qdbint_t);
qdb_datum qdb_gdbm_nextkey(qdbint_t, qdb_datum);
void qdb_gdbm_optimise(qdbint_t);
char *qdb_gdbm_error(void);
#endif /* USING_GDBM */
#ifdef USING_MYSQL
int qdb_mysql_identify(const char *);
qdbint_t qdb_mysql_open(const char *, qdb_open_t);
int qdb_mysql_fd(qdbint_t);
void qdb_mysql_close(qdbint_t);
qdb_datum qdb_mysql_fetch(qdbint_t, qdb_datum);
int qdb_mysql_store(qdbint_t, qdb_datum, qdb_datum);
int qdb_mysql_delete(qdbint_t, qdb_datum);
qdb_datum qdb_mysql_firstkey(qdbint_t);
qdb_datum qdb_mysql_nextkey(qdbint_t, qdb_datum);
char *qdb_mysql_error(void);
#endif /* USING_MYSQL */
#ifdef USING_SQLITE
int qdb_sqlite_identify(const char *);
qdbint_t qdb_sqlite_open(const char *, qdb_open_t);
int qdb_sqlite_fd(qdbint_t);
void qdb_sqlite_close(qdbint_t);
qdb_datum qdb_sqlite_fetch(qdbint_t, qdb_datum);
int qdb_sqlite_store(qdbint_t, qdb_datum, qdb_datum);
int qdb_sqlite_delete(qdbint_t, qdb_datum);
qdb_datum qdb_sqlite_firstkey(qdbint_t);
qdb_datum qdb_sqlite_nextkey(qdbint_t, qdb_datum);
char *qdb_sqlite_error(void);
void qdb_sqlite_unlock(qdbint_t);
void qdb_sqlite_relock(qdbint_t);
#endif /* USING_SQLITE */
struct qdbtype_s {
char *name;
int (*_identify) (const char *);
qdbint_t(*_open) (const char *, qdb_open_t);
int (*_fd) (qdbint_t);
void (*_close) (qdbint_t);
qdb_datum(*_fetch) (qdbint_t, qdb_datum);
int (*_store) (qdbint_t, qdb_datum, qdb_datum);
int (*_delete) (qdbint_t, qdb_datum);
qdb_datum(*_firstkey) (qdbint_t);
qdb_datum(*_nextkey) (qdbint_t, qdb_datum);
char *(*_error) (void);
void (*_optimise) (qdbint_t); /* optional */
void (*_unlock) (qdbint_t); /* optional */
void (*_relock) (qdbint_t); /* optional */
void (*_restore_start) (qdbint_t); /* optional */
void (*_restore_end) (qdbint_t); /* optional */
};
struct qdb_s {
struct qdbtype_s *type;
int typeindex;
struct qdbint_s *data;
};
static struct qdbtype_s qdb__backends[] = {
#ifdef USING_LIST
{
"list",
qdb_list_identify,
qdb_list_open,
qdb_list_fd,
qdb_list_close,
qdb_list_fetch,
qdb_list_store,
qdb_list_delete,
qdb_list_firstkey,
qdb_list_nextkey,
qdb_list_error,
NULL, /* optimise */
NULL, /* unlock */
NULL, /* relock */
qdb_list_restore_start,
qdb_list_restore_end},
#endif
#ifdef USING_BTREE
{
"btree",
qdb_btree_identify,
qdb_btree_open,
qdb_btree_fd,
qdb_btree_close,
qdb_btree_fetch,
qdb_btree_store,
qdb_btree_delete,
qdb_btree_firstkey,
qdb_btree_nextkey,
qdb_btree_error,
qdb_btree_optimise,
qdb_btree_unlock,
qdb_btree_relock,
NULL, /* restore_start */
NULL}, /* restore_end */
#endif
#ifdef USING_OBTREE
{
"obtree",
qdb_obtree_identify,
qdb_obtree_open,
qdb_obtree_fd,
qdb_obtree_close,
qdb_obtree_fetch,
qdb_obtree_store,
qdb_obtree_delete,
qdb_obtree_firstkey,
qdb_obtree_nextkey,
qdb_obtree_error,
NULL, /* optimise */
qdb_obtree_unlock,
qdb_obtree_relock,
NULL, /* restore_start */
NULL}, /* restore_end */
#endif
#ifdef USING_GDBM
{
"GDBM",
qdb_gdbm_identify,
qdb_gdbm_open,
qdb_gdbm_fd,
qdb_gdbm_close,
qdb_gdbm_fetch,
qdb_gdbm_store,
qdb_gdbm_delete,
qdb_gdbm_firstkey,
qdb_gdbm_nextkey,
qdb_gdbm_error,
qdb_gdbm_optimise,
NULL, /* unlock */
NULL, /* relock */
NULL, /* restore_start */
NULL}, /* restore_end */
#endif
#ifdef USING_MYSQL
{
"MySQL",
qdb_mysql_identify,
qdb_mysql_open,
qdb_mysql_fd,
qdb_mysql_close,
qdb_mysql_fetch,
qdb_mysql_store,
qdb_mysql_delete,
qdb_mysql_firstkey,
qdb_mysql_nextkey,
qdb_mysql_error,
NULL, /* optimise */
NULL, /* unlock */
NULL, /* relock */
NULL, /* restore_start */
NULL}, /* restore_end */
#endif
#ifdef USING_SQLITE
{
"sqlite",
qdb_sqlite_identify,
qdb_sqlite_open,
qdb_sqlite_fd,
qdb_sqlite_close,
qdb_sqlite_fetch,
qdb_sqlite_store,
qdb_sqlite_delete,
qdb_sqlite_firstkey,
qdb_sqlite_nextkey,
qdb_sqlite_error,
NULL, /* optimise */
qdb_sqlite_unlock,
qdb_sqlite_relock,
NULL, /* restore_start */
NULL}, /* restore_end */
#endif
{NULL, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}
};
static int qdb__lastbackend = 0;
/*
* Return a file descriptor for the given database, or -1 on error.
*/
int qdb_fd(qdb_t db)
{
if (db == NULL)
return -1;
qdb__lastbackend = db->typeindex;
return db->type->_fd(db->data);
}
/*
* Return a string describing the type of the given database, or "(none)" on
* error. The string should NOT be free()d.
*/
char *qdb_type(qdb_t db)
{
if (db == NULL)
return "(none)";
return db->type->name;
}
/*
* Open the given database in the given way (new database, read-only, or
* read-write); return a qdb_t or NULL on error.
*/
qdb_t qdb_open(const char *file, qdb_open_t method)
{
int type, i;
qdb_t db;
type = -1;
for (i = 0; type == -1 && qdb__backends[i].name != NULL; i++) {
if (qdb__backends[i]._identify(file))
type = i;
}
/*
* If no valid type was determined, fall back to obtree if the file
* already exists, list if not (providing they are both compiled
* in). If neither is compiled in, fall back to the first available
* backend in the list.
*/
if (type == -1) {
char *usetype = NULL;
FILE *fptr;
#ifdef USING_LIST
usetype = "list";
#endif
#ifdef USING_OBTREE
fptr = fopen(file, "rb");
if (fptr) {
fclose(fptr);
usetype = "obtree";
}
#endif
if (usetype) {
for (i = 0;
type == -1 && qdb__backends[i].name != NULL;
i++) {
if (strcmp(qdb__backends[i].name, usetype)
== 0)
type = i;
}
}
if (type == -1)
type = 0;
}
qdb__lastbackend = type;
db = calloc(1, sizeof(*db));
if (db == NULL) {
return NULL;
}
db->type = &(qdb__backends[type]);
db->typeindex = type;
db->data = db->type->_open(file, method);
if (db->data == NULL) {
free(db);
return NULL;
}
return db;
}
/*
* Close the given database.
*/
void qdb_close(qdb_t db)
{
if (db == NULL)
return;
qdb__lastbackend = db->typeindex;
db->type->_close(db->data);
free(db);
}
/*
* NOTE: From here on down, we take copies of "key" and "val" qdb_datum
* objects before sending them on to the backend. For some reason OpenBSD
* corrupts the key and val if we just pass them on, but if we pass on
* copies, it's OK.
*/
/*
* Fetch a value from the database. The datum returned needs its val.data
* free()ing after use. If val.data is NULL, no value was found for the
* given key.
*/
qdb_datum qdb_fetch(qdb_t db, qdb_datum key)
{
qdb_datum keycopy;
if (db == NULL) {
qdb_datum val;
val.data = NULL;
val.size = 0;
return val;
}
if (key.data == NULL) {
qdb_datum val;
val.data = NULL;
val.size = 0;
return val;
}
keycopy = key;
qdb__lastbackend = db->typeindex;
return db->type->_fetch(db->data, keycopy);
}
/*
* Store the given key with the given value into the database, replacing any
* existing value for that key. Returns nonzero on error.
*/
int qdb_store(qdb_t db, qdb_datum key, qdb_datum val)
{
qdb_datum keycopy, valcopy;
if (db == NULL)
return 1;
if ((key.data == NULL) || (val.data == NULL))
return 1;
keycopy = key;
valcopy = val;
qdb__lastbackend = db->typeindex;
return db->type->_store(db->data, keycopy, valcopy);
}
/*
* Delete the given key from the database. Returns nonzero on error.
*/
int qdb_delete(qdb_t db, qdb_datum key)
{
qdb_datum keycopy;
if (db == NULL)
return 1;
if (key.data == NULL)
return 1;
keycopy = key;
qdb__lastbackend = db->typeindex;
return db->type->_delete(db->data, keycopy);
}
/*
* Return the "first" key in the database, suitable for using with repeated
* calls to qdb_nextkey() to walk through every key in the database.
*/
qdb_datum qdb_firstkey(qdb_t db)
{
qdb__lastbackend = db->typeindex;
return db->type->_firstkey(db->data);
}
/*
* Return the "next" key in the database, or key.data=NULL when all keys
* have been returned.
*/
qdb_datum qdb_nextkey(qdb_t db, qdb_datum key)
{
qdb_datum keycopy;
keycopy = key;
qdb__lastbackend = db->typeindex;
return db->type->_nextkey(db->data, keycopy);
}
/*
* Reorganise the database for better efficiency.
*/
void qdb_optimise(qdb_t db)
{
qdb__lastbackend = db->typeindex;
if (db->type->_optimise == NULL)
return;
db->type->_optimise(db->data);
}
/*
* Return a string describing the last database error to occur.
*/
char *qdb_error(void)
{
return qdb__backends[qdb__lastbackend]._error();
}
/*
* Temporarily release the lock on the database.
*/
void qdb_unlock(qdb_t db)
{
qdb__lastbackend = db->typeindex;
if (db->type->_unlock == NULL)
return;
db->type->_unlock(db->data);
}
/*
* Reassert the lock on the database.
*/
void qdb_relock(qdb_t db)
{
qdb__lastbackend = db->typeindex;
if (db->type->_relock == NULL)
return;
db->type->_relock(db->data);
}
/*
* Tell the database that a restore operation is starting.
*/
void qdb_restore_start(qdb_t db)
{
qdb__lastbackend = db->typeindex;
if (db->type->_restore_start == NULL)
return;
db->type->_restore_start(db->data);
}
/*
* Tell the database that a restore operation is ending.
*/
void qdb_restore_end(qdb_t db)
{
qdb__lastbackend = db->typeindex;
if (db->type->_restore_end == NULL)
return;
db->type->_restore_end(db->data);
}
/*
* Utility functions that can be used by the back-end follow.
*/
/*
* Block common interrupt signals, so a user doesn't accidentally kill this
* process while it's doing something critical to data integrity.
*/
void qdb_int__sig_block(void)
{
#ifdef SIGHUP
signal(SIGHUP, SIG_IGN); /* RATS: ignore (OK) */
#endif
#ifdef SIGINT
signal(SIGINT, SIG_IGN); /* RATS: ignore (OK) */
#endif
#ifdef SIGQUIT
signal(SIGQUIT, SIG_IGN); /* RATS: ignore (OK) */
#endif
#ifdef SIGTERM
signal(SIGTERM, SIG_IGN); /* RATS: ignore (OK) */
#endif
}
/*
* Unblock previously blocked signals.
*/
void qdb_int__sig_unblock(void)
{
#ifdef SIGHUP
signal(SIGHUP, SIG_DFL); /* RATS: ignore (OK) */
#endif
#ifdef SIGINT
signal(SIGINT, SIG_DFL); /* RATS: ignore (OK) */
#endif
#ifdef SIGQUIT
signal(SIGQUIT, SIG_DFL); /* RATS: ignore (OK) */
#endif
#ifdef SIGTERM
signal(SIGTERM, SIG_DFL); /* RATS: ignore (OK) */
#endif
}
/*
* Obtain / release a read or write lock on the database. Returns zero on
* success, 1 on error. Blocks until a lock can be obtained. Keeps track of
* the number of times locked, so we don't accidentally try to lock a file
* descriptor twice.
*/
int qdb_int__lock(int fd, int lock_type, int *lockcount)
{
#ifdef HAVE_FCNTL
struct flock lock;
int lockdummy = 0;
if (lockcount == NULL)
lockcount = &lockdummy;
if ((lock_type == F_UNLCK) && (*lockcount > 1)) {
*lockcount = (*lockcount) - 1;
return 0;
} else if ((lock_type != F_UNLCK) && (*lockcount > 0)) {
*lockcount = (*lockcount) + 1;
return 0;
}
lock.l_whence = SEEK_SET;
lock.l_start = 0;
lock.l_len = 0;
lock.l_type = lock_type;
if (fcntl(fd, F_SETLKW, &lock)) {
return 1;
}
if (lock_type == F_UNLCK) {
*lockcount = (*lockcount) - 1;
} else {
*lockcount = (*lockcount) + 1;
}
#endif /* HAVE_FCNTL */
return 0;
}
/* EOF */
qsf-1.2.7/src/db/gdbm.c 0000644 0000764 0000764 00000012356 10664772303 012372 0 ustar aw aw /*
* Make our db functions a wrapper for GDBM.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "database.h"
#ifdef USING_GDBM
#include
#include
#include
#include
#include
#ifdef HAVE_FCNTL
#include
#endif
#include
#include
struct qdbint_s {
GDBM_FILE dbf;
#ifndef HAVE_GDBM_FDESC
int fd;
#endif /* !HAVE_GDBM_FDESC */
};
/*
* Return a file descriptor for the given database, or -1 on error.
*/
int qdb_gdbm_fd(qdbint_t db)
{
if (db == NULL)
return -1;
if (db->dbf == NULL)
return -1;
#ifdef HAVE_GDBM_FDESC
return gdbm_fdesc(db->dbf);
#else /* !HAVE_GDBM_FDESC */
return db->fd;
#endif /* HAVE_GDBM_FDESC */
}
/*
* Return nonzero if the given file is of this database type.
*/
int qdb_gdbm_identify(const char *file)
{
FILE *fptr;
unsigned char buf[8]; /* RATS: ignore (checked) */
if (file == NULL)
return 0;
if (strncasecmp(file, "gdbm:", 5) == 0)
return 1;
fptr = fopen(file, "rb");
if (fptr == NULL)
return 0;
if (fread(buf, 4, 1, fptr) < 1) {
fclose(fptr);
return 0;
}
fclose(fptr);
buf[4] = 0;
if ((buf[0] == 0x13)
&& (buf[1] == 0x57)
&& (buf[2] == 0x9a)
&& (buf[3] == 0xce)
)
return 1;
if ((buf[3] == 0x13)
&& (buf[2] == 0x57)
&& (buf[1] == 0x9a)
&& (buf[0] == 0xce)
)
return 1;
if (strncmp((char *) buf, "GDBM", 4) == 0)
return 1;
return 0;
}
/*
* Open the given database in the given way (new database, read-only, or
* read-write); return a qdbint_t or NULL on error.
*/
qdbint_t qdb_gdbm_open(const char *file, qdb_open_t method)
{
GDBM_FILE dbf = NULL;
qdbint_t db;
int tries;
if (strncasecmp(file, "gdbm:", 5) == 0)
file += 5;
for (tries = 0; tries < 60 && dbf == NULL; tries++) {
switch (method) {
case QDB_NEW:
dbf =
gdbm_open((char *) file, 512, GDBM_NEWDB,
S_IRUSR | S_IWUSR, NULL);
break;
case QDB_READONLY:
dbf =
gdbm_open((char *) file, 512, GDBM_READER,
S_IRUSR, NULL);
break;
case QDB_READWRITE:
dbf =
gdbm_open((char *) file, 512, GDBM_WRCREAT,
S_IRUSR | S_IWUSR, NULL);
break;
default:
break;
}
if (dbf != NULL)
break;
switch (gdbm_errno) {
case GDBM_CANT_BE_READER:
case GDBM_CANT_BE_WRITER:
sleep(1);
break;
default:
return NULL;
}
}
if (dbf == NULL)
return NULL;
db = calloc(1, sizeof(*db));
if (db == NULL) {
gdbm_close(dbf);
return NULL;
}
#ifndef HAVE_GDBM_FDESC
db->fd = open(file, O_RDONLY);
#endif /* !HAVE_GDBM_FDESC */
db->dbf = dbf;
return db;
}
/*
* Close the given database.
*/
void qdb_gdbm_close(qdbint_t db)
{
if (db == NULL)
return;
#ifndef HAVE_GDBM_FDESC
if (db->fd >= 0)
close(db->fd);
#endif /* !HAVE_GDBM_FDESC */
gdbm_close(db->dbf);
free(db);
}
/*
* Fetch a value from the database. The datum returned needs its val.data
* free()ing after use. If val.data is NULL, no value was found for the
* given key.
*/
qdb_datum qdb_gdbm_fetch(qdbint_t db, qdb_datum key)
{
datum gkey, gval;
qdb_datum val;
if (db == NULL) {
val.data = NULL;
val.size = 0;
return val;
}
if (key.data == NULL) {
val.data = NULL;
val.size = 0;
return val;
}
gkey.dptr = (char *) (key.data);
gkey.dsize = key.size;
gval = gdbm_fetch(db->dbf, gkey);
val.data = (unsigned char *) (gval.dptr);
val.size = gval.dsize;
return val;
}
/*
* Store the given key with the given value into the database, replacing any
* existing value for that key. Returns nonzero on error.
*/
int qdb_gdbm_store(qdbint_t db, qdb_datum key, qdb_datum val)
{
datum gkey, gval;
if (db == NULL)
return 1;
if ((key.data == NULL) || (val.data == NULL))
return 1;
gkey.dptr = (char *) (key.data);
gkey.dsize = key.size;
gval.dptr = (char *) (val.data);
gval.dsize = val.size;
return gdbm_store(db->dbf, gkey, gval, GDBM_REPLACE);
}
/*
* Delete the given key from the database. Returns nonzero on error.
*/
int qdb_gdbm_delete(qdbint_t db, qdb_datum key)
{
datum gkey;
if (db == NULL)
return 1;
if (key.data == NULL)
return 1;
gkey.dptr = (char *) (key.data);
gkey.dsize = key.size;
return gdbm_delete(db->dbf, gkey);
}
/*
* Return the "first" key in the database, suitable for using with repeated
* calls to qdb_nextkey() to walk through every key in the database.
*/
qdb_datum qdb_gdbm_firstkey(qdbint_t db)
{
datum gkey;
qdb_datum key;
gkey = gdbm_firstkey(db->dbf);
key.data = (unsigned char *) (gkey.dptr);
key.size = gkey.dsize;
return key;
}
/*
* Return the "next" key in the database, or key.data=NULL when all keys
* have been returned.
*/
qdb_datum qdb_gdbm_nextkey(qdbint_t db, qdb_datum key)
{
datum gkey, newgkey;
qdb_datum newkey;
gkey.dptr = (char *) (key.data);
gkey.dsize = key.size;
newgkey = gdbm_nextkey(db->dbf, gkey);
newkey.data = (unsigned char *) (newgkey.dptr);
newkey.size = newgkey.dsize;
return newkey;
}
/*
* Reorganise the database for better efficiency.
*/
void qdb_gdbm_optimise(qdbint_t db)
{
gdbm_reorganize(db->dbf);
}
/*
* Return a string describing the last database error to occur.
*/
char *qdb_gdbm_error(void)
{
return (char *) gdbm_strerror(gdbm_errno);
}
#endif /* USING_GDBM */
/* EOF */
qsf-1.2.7/src/db/btree.c 0000644 0000764 0000764 00000057223 10664772303 012564 0 ustar aw aw /*
* Binary tree database backend.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "database.h"
#include "log.h"
#ifdef USING_BTREE
#include
#include
#include
#include
#include
#include
#include
#include
#ifdef HAVE_FCNTL
#include
#endif
#ifdef HAVE_MMAP
#include
#endif
#ifdef HAVE_UTIME
#include
#include
#endif
#define BT_RECORD_SIZE sizeof(struct bt_record)
#define BT_TOKEN_MAX 36
typedef unsigned long bt_ul;
struct bt_record { /* single binary tree record */
bt_ul lower; /* offset of "lower" token */
bt_ul higher; /* offset of "higher" token */
unsigned char token[BT_TOKEN_MAX]; /* RATS: ignore - 0-term. token */
long data[3]; /* token data (i.e. the counts) */
};
typedef struct bt_record *bt_record_t;
struct qdbint_s { /* database state */
int fd; /* file descriptor of database file */
bt_ul size; /* total size of database */
#ifdef HAVE_FCNTL
int lockcount; /* number of times lock asked for */
int locktype; /* type of lock to use (read or write) */
#endif
int gotheadoffs; /* flag, set once head offset read */
bt_ul head_offset; /* offset of head record of tree */
#ifdef HAVE_MMAP
unsigned char *data; /* mmap()ed database data */
bt_ul filepos; /* current file pointer */
#endif
char *filename; /* filename of database */
int modified; /* flag, set if db modified at all */
};
static char *bt_lasterror = "";
#ifdef HAVE_FCNTL
/*
* Obtain / release a read or write lock on the database. Returns nonzero on
* error, and blocks until a lock can be obtained.
*/
static int dbbt_lock(qdbint_t db, int lock_type)
{
int ret;
ret = qdb_int__lock(db->fd, lock_type, &(db->lockcount));
if (ret != 0) {
bt_lasterror = strerror(errno);
return 1;
}
return 0;
}
#endif /* HAVE_FCNTL */
static off_t dbbt_lseek(qdbint_t db, off_t offset, int whence)
{
#ifdef HAVE_MMAP
if ((db->data == 0) || (db->filepos < 0) || (whence != SEEK_SET)) {
db->filepos = lseek(db->fd, offset, whence);
} else {
db->filepos = offset;
}
return db->filepos;
#else
return lseek(db->fd, offset, whence);
#endif
}
/*
* Analogue of fread().
*/
static int dbbt_chunkread_raw(void *ptr, int size, int nmemb, int fd)
{
int numread, togo, got;
for (numread = 0; nmemb > 0; nmemb--, numread++) {
for (togo = size; togo > 0;) {
got = read(fd, ptr, togo); /* RATS: ignore (OK) */
if (got <= 0)
return numread;
togo -= got;
ptr = (void *) (((char *) ptr) + got);
}
}
return numread;
}
/*
* Analogue of fwrite().
*/
static int dbbt_chunkwrite_raw(void *ptr, int size, int nmemb, int fd)
{
int numwritten, togo, written;
for (numwritten = 0; nmemb > 0; nmemb--, numwritten++) {
for (togo = size; togo > 0;) {
written = write(fd, ptr, togo);
if (written <= 0)
return numwritten;
togo -= written;
ptr = (void *) (((char *) ptr) + written);
}
}
return numwritten;
}
/*
* Analogue of fread(), using memory mapped data if possible.
*/
static int dbbt_chunkread(void *ptr, int size, int nmemb, qdbint_t db)
{
#ifdef HAVE_MMAP
long available, copyable, tocopy;
if ((db->data == 0) || (db->filepos < 0)) {
return dbbt_chunkread_raw(ptr, size, nmemb, db->fd);
}
if (size < 1)
return 0;
available = db->size - db->filepos;
if (available <= 0)
return 0;
copyable = available / size;
if (copyable < nmemb)
nmemb = copyable;
tocopy = nmemb * size;
memcpy(ptr, db->data + db->filepos, tocopy);
db->filepos += tocopy;
return nmemb;
#else /* !HAVE_MMAP */
return dbbt_chunkread_raw(ptr, size, nmemb, db->fd);
#endif /* HAVE_MMAP */
}
/*
* Analogue of fwrite(), using memory mapped data if possible, remapping
* after extending the file if not.
*/
static int dbbt_chunkwrite(void *ptr, int size, int nmemb, qdbint_t db)
{
#ifdef HAVE_MMAP
long available, copyable, tocopy;
db->modified = 1;
if ((db->data == 0) || (db->filepos < 0))
return dbbt_chunkwrite_raw(ptr, size, nmemb, db->fd);
if (size < 1)
return 0;
available = db->size - db->filepos;
copyable = available / size;
if (copyable < nmemb) {
int retval;
munmap(db->data, db->size);
db->data = 0;
lseek(db->fd, db->filepos, SEEK_SET);
retval = dbbt_chunkwrite_raw(ptr, size, nmemb, db->fd);
db->size = lseek(db->fd, 0, SEEK_END);
db->data =
mmap(NULL, db->size, PROT_READ | PROT_WRITE,
MAP_SHARED, db->fd, 0);
if (db->data == MAP_FAILED)
db->data = 0;
db->filepos = db->size;
return retval;
} else {
tocopy = nmemb * size;
memcpy(db->data + db->filepos, ptr, tocopy);
db->filepos += tocopy;
return nmemb;
}
#else /* !HAVE_MMAP */
return dbbt_chunkwrite_raw(ptr, size, nmemb, db->fd);
#endif /* HAVE_MMAP */
}
/*
* Read a record from the database at the given offset into the given record
* structure, returning nonzero on failure.
*/
static int dbbt_read_record(qdbint_t db, bt_ul offset, bt_record_t record)
{
int got;
if (dbbt_lseek(db, offset, SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
return 1;
}
got = dbbt_chunkread(record, BT_RECORD_SIZE, 1, db);
if (got < 1) {
bt_lasterror = strerror(errno);
return 1;
}
record->token[BT_TOKEN_MAX - 1] = 0;
return 0;
}
/*
* Write a record to the database at the given offset, returning nonzero on
* failure.
*/
static int dbbt_write_record(qdbint_t db, bt_ul offset, bt_record_t record)
{
if (dbbt_lseek(db, offset, SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
return 1;
}
if (dbbt_chunkwrite(record, BT_RECORD_SIZE, 1, db) < 1) {
bt_lasterror = strerror(errno);
return 1;
}
return 0;
}
/*
* Find the given token in the database and fill in the given record
* structure if found, also filling in the offset of the record (or 0 if not
* found) and the offset of the parent record (0 if none).
*
* Returns -1 if the token looked for was "lower" than its parent or +1 if
* "higher", or 0 if there was no parent record (i.e. this is the first
* record).
*/
static int dbbt_find_token(qdbint_t db, qdb_datum key, bt_record_t record,
bt_ul * offset, bt_ul * parent)
{
int hilow = 0;
int x;
*offset = 0;
*parent = 0;
if (db == NULL)
return 0;
if (db->size < 2 * sizeof(long))
return 0;
if (!db->gotheadoffs) {
dbbt_lseek(db, 4, SEEK_SET);
dbbt_chunkread(offset, sizeof(*offset), 1, db);
db->head_offset = *offset;
db->gotheadoffs = 1;
} else {
*offset = db->head_offset;
}
while (*offset > 0) {
int len;
if (dbbt_read_record(db, *offset, record)) {
*offset = 0;
break;
}
x = strncmp((char *) (record->token), (char *) (key.data),
key.size);
len = strlen((char *) (record->token));
if (len < key.size) {
x = -1;
} else if (len > key.size) {
x = 1;
}
if (x == 0) {
return hilow;
} else if (x < 0) {
*parent = *offset;
hilow = -1;
*offset = record->lower;
} else {
*parent = *offset;
hilow = 1;
*offset = record->higher;
}
}
return hilow;
}
/*
* Mark the block at the given offset as free space by adding it to the head
* of the free space list. Returns nonzero on error.
*/
static int dbbt_markfree(qdbint_t db, bt_ul offset)
{
bt_ul nextfree, storeoffs;
/*
* Read the "next free" offset from the free space head block
*/
if (dbbt_lseek(db, 4 + sizeof(offset), SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
return 1;
}
nextfree = 0x80000000;
if (dbbt_chunkread(&nextfree, sizeof(nextfree), 1, db) < 0) {
bt_lasterror = strerror(errno);
return 1;
}
/*
* Write the given offset as the new "next free" offset
*/
if (dbbt_lseek(db, 4 + sizeof(offset), SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
return 1;
}
storeoffs = offset;
storeoffs |= 0x80000000;
if (dbbt_chunkwrite(&storeoffs, sizeof(offset), 1, db) < 0) {
bt_lasterror = strerror(errno);
return 1;
}
/*
* Write the old "next free" offset into the block being freed
*/
if (dbbt_lseek(db, offset, SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
return 1;
}
if (dbbt_chunkwrite(&nextfree, sizeof(nextfree), 1, db) < 1) {
bt_lasterror = strerror(errno);
return 1;
}
return 0;
}
/*
* Return nonzero if the given file is of this database type.
*/
int qdb_btree_identify(const char *file)
{
FILE *fptr;
unsigned char buf[8]; /* RATS: ignore (checked) */
if (file == NULL)
return 0;
if (strncasecmp(file, "btree:", 6) == 0)
return 1;
fptr = fopen(file, "rb");
if (fptr == NULL)
return 0;
if (fread(buf, 4, 1, fptr) < 1) {
fclose(fptr);
return 0;
}
fclose(fptr);
if (strncmp((char *) buf, "QSF1", 4) == 0)
return 1;
return 0;
}
/*
* Open the given database in the given way (new database, read-only, or
* read-write); return a qdbint_t or NULL on error.
*/
qdbint_t qdb_btree_open(const char *file, qdb_open_t method)
{
qdbint_t db;
int fd = -1;
#ifdef HAVE_FCNTL
int locktype = F_RDLCK;
#endif
if (strncasecmp(file, "btree:", 6) == 0)
file += 6;
switch (method) {
case QDB_NEW:
fd = open(file, /* RATS: ignore (no race) */
O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
if (fd < 0)
bt_lasterror = strerror(errno);
#ifdef HAVE_FCNTL
locktype = F_WRLCK;
#endif
break;
case QDB_READONLY:
fd = open(file, /* RATS: ignore (no race) */
O_RDONLY);
if (fd < 0)
bt_lasterror = strerror(errno);
break;
case QDB_READWRITE:
fd = open(file, /* RATS: ignore (no race) */
O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
if (fd < 0)
bt_lasterror = strerror(errno);
#ifdef HAVE_FCNTL
locktype = F_WRLCK;
#endif
break;
default:
break;
}
if (fd < 0)
return NULL;
db = calloc(1, sizeof(*db));
if (db == NULL) {
bt_lasterror = strerror(errno);
close(fd);
return NULL;
}
db->filename = strdup(file);
db->fd = fd;
db->modified = 0;
#ifdef HAVE_FCNTL
db->locktype = locktype;
if (dbbt_lock(db, locktype)) {
close(fd);
if (db->filename)
free(db->filename);
free(db);
return NULL;
}
#endif
db->size = dbbt_lseek(db, 0, SEEK_END);
/*
* If this is a new database, and it's not readonly, write our
* header to it and return.
*/
if ((db->size < 5) && (method != QDB_READONLY)) {
dbbt_lseek(db, 0, SEEK_SET);
dbbt_chunkwrite_raw("QSF1", 4, 1, db->fd);
db->size = 4;
db->modified = 1;
#ifdef HAVE_MMAP
db->filepos = 0;
db->data =
mmap(NULL, db->size,
PROT_READ | (method ==
QDB_READONLY ? 0 : PROT_WRITE),
MAP_SHARED, db->fd, 0);
if (db->data == MAP_FAILED)
db->data = 0;
#endif
return db;
}
/*
* We now do some simple checks to make sure that the file is a
* database of the format we're expecting.
*/
/*
* If it's shorter than 4 bytes plus 2 long ints, it's not a valid
* database.
*/
if (db->size < 4 + 2 * sizeof(long)) {
bt_lasterror = _("invalid database (too small)");
close(fd);
if (db->filename)
free(db->filename);
free(db);
return NULL;
}
/*
* If its size, discounting the two longs at the start, isn't a
* multiple of our record size, it's not a valid database.
*/
if (((db->size - (4 + 2 * sizeof(long))) % BT_RECORD_SIZE) != 0) {
bt_lasterror = _("invalid database (irregular size)");
close(fd);
if (db->filename)
free(db->filename);
free(db);
return NULL;
}
/*
* If the first long int (the "head offset") is larger than the size
* of the file, this isn't a valid database.
*/
{
bt_ul offset = db->size + 1;
dbbt_lseek(db, 4, SEEK_SET);
dbbt_chunkread(&offset, sizeof(offset), 1, db);
if (offset > db->size) {
bt_lasterror =
_("invalid database (bad head offset)");
close(fd);
if (db->filename)
free(db->filename);
free(db);
return NULL;
}
dbbt_lseek(db, 4, SEEK_SET);
}
#ifdef HAVE_MMAP
db->data =
mmap(NULL, db->size,
PROT_READ | (method == QDB_READONLY ? 0 : PROT_WRITE),
MAP_SHARED, db->fd, 0);
if (db->data == MAP_FAILED)
db->data = 0;
#endif
return db;
}
/*
* Close the given database.
*/
void qdb_btree_close(qdbint_t db)
{
if (db == NULL)
return;
#ifdef HAVE_MMAP
if (db->data)
munmap(db->data, db->size);
#endif
#ifdef HAVE_FCNTL
while (db->lockcount > 0)
dbbt_lock(db, F_UNLCK);
#endif
close(db->fd);
if (db->modified && db->filename) {
#ifdef HAVE_UTIME
struct utimbuf utb;
utb.actime = time(NULL);
utb.modtime = time(NULL);
utime(db->filename, &utb);
#endif
}
if (db->filename)
free(db->filename);
free(db);
}
/*
* Fetch a value from the database. The datum returned needs its val.data
* free()ing after use. If val.data is NULL, no value was found for the
* given key.
*/
qdb_datum qdb_btree_fetch(qdbint_t db, qdb_datum key)
{
struct bt_record record;
unsigned long offset, parent;
qdb_datum val;
val.data = NULL;
val.size = 0;
if (db == NULL)
return val;
if (key.size >= BT_TOKEN_MAX)
key.size = BT_TOKEN_MAX - 1;
dbbt_find_token(db, key, &record, &offset, &parent);
if (offset == 0)
return val;
val.size = 3 * sizeof(long);
val.data = calloc(1, val.size);
if (val.data == NULL) {
val.size = 0;
return val;
}
((long *) val.data)[0] = record.data[0];
((long *) val.data)[1] = record.data[1];
((long *) val.data)[2] = record.data[2];
return val;
}
/*
* Return a file descriptor for the given database, or -1 on error.
*/
int qdb_btree_fd(qdbint_t db)
{
if (db == NULL)
return -1;
return db->fd;
}
/*
* Store the given key with the given value into the database, replacing any
* existing value for that key. Returns nonzero on error.
*/
int qdb_btree_store(qdbint_t db, qdb_datum key, qdb_datum val)
{
struct bt_record record, head;
unsigned long offset, parent, nextfree;
int x;
if (db == NULL)
return 1;
memset(&head, 0, BT_RECORD_SIZE);
memset(&record, 0, BT_RECORD_SIZE);
if (key.size >= BT_TOKEN_MAX)
key.size = BT_TOKEN_MAX - 1;
x = dbbt_find_token(db, key, &record, &offset, &parent);
memcpy(record.token, key.data, key.size);
record.token[key.size] = 0;
record.data[0] = ((long *) val.data)[0];
record.data[1] = ((long *) val.data)[1];
if (val.size > 2 * sizeof(long))
record.data[2] = ((long *) val.data)[2];
qdb_int__sig_block();
/*
* Record exists - overwrite it.
*/
if (offset > 0) {
if (dbbt_write_record(db, offset, &record)) {
qdb_int__sig_unblock();
return 1;
}
qdb_int__sig_unblock();
return 0;
}
record.lower = 0;
record.higher = 0;
/*
* Database has just been created, so fill in the header and write
* this record as the first one.
*/
if (db->size <= 4 + sizeof(offset)) {
if (dbbt_lseek(db, 4, SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
offset = 4 + 2 * sizeof(offset);
if (dbbt_chunkwrite(&offset, sizeof(offset), 1, db) < 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
db->gotheadoffs = 0;
offset = 0;
if (dbbt_chunkwrite(&offset, sizeof(offset), 1, db) < 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
head.lower = (4 + 2 * sizeof(offset)) + BT_RECORD_SIZE;
if (dbbt_chunkwrite(&head, BT_RECORD_SIZE, 1, db) < 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if (dbbt_chunkwrite(&record, BT_RECORD_SIZE, 1, db) < 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
db->size = dbbt_lseek(db, 0, SEEK_CUR);
qdb_int__sig_unblock();
return 0;
}
/*
* Get offset of next free space block.
*/
if (dbbt_lseek(db, 4 + sizeof(offset), SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if (dbbt_chunkread(&offset, sizeof(offset), 1, db) < 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
offset &= 0x7FFFFFFF;
/*
* If offset is 0 or we can't read from that offset, it's a new
* block at the end of the file, otherwise we take the next free
* offset from there and store it in the core free pointer, and then
* use that free offset.
*/
if (dbbt_lseek(db, offset, SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if ((offset < 5)
|| (dbbt_chunkread(&nextfree, sizeof(nextfree), 1, db) < 1)
) {
offset = dbbt_lseek(db, 0, SEEK_END);
nextfree = 0x80000000;
}
if (dbbt_lseek(db, 4 + sizeof(offset), SEEK_SET) == (off_t) - 1) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if (dbbt_chunkwrite(&nextfree, sizeof(nextfree), 1, db) < 0) {
bt_lasterror = strerror(errno);
qdb_int__sig_unblock();
return 1;
}
if (dbbt_write_record(db, offset, &record)) {
qdb_int__sig_unblock();
return 1;
}
/*
* Now attach the new record to its parent, if applicable.
*/
if (parent > 0) {
if (dbbt_read_record(db, parent, &record)) {
qdb_int__sig_unblock();
return 1;
}
if (x < 0) {
record.lower = offset;
} else {
record.higher = offset;
}
if (dbbt_write_record(db, parent, &record)) {
qdb_int__sig_unblock();
return 1;
}
}
qdb_int__sig_unblock();
return 0;
}
/*
* Return the "first" key in the database, suitable for using with repeated
* calls to qdb_nextkey() to walk through every key in the database.
*/
qdb_datum qdb_btree_firstkey(qdbint_t db)
{
struct bt_record record;
qdb_datum newkey;
newkey.data = NULL;
newkey.size = 0;
if (dbbt_lseek(db, 4 + 2 * sizeof(unsigned long) + BT_RECORD_SIZE,
SEEK_SET) == (off_t) - 1)
return newkey;
do {
if (dbbt_chunkread(&record, BT_RECORD_SIZE, 1, db) < 1) {
return newkey;
}
} while (record.lower & 0x80000000);
record.token[BT_TOKEN_MAX - 1] = 0;
newkey.data = (unsigned char *) strdup((char *) (record.token));
newkey.size = strlen((char *) (record.token));
return newkey;
}
/*
* Return the "next" key in the database, or key.data=NULL when all keys
* have been returned.
*/
qdb_datum qdb_btree_nextkey(qdbint_t db, qdb_datum key)
{
struct bt_record record;
unsigned long offset, parent;
qdb_datum newkey;
newkey.data = NULL;
newkey.size = 0;
if (key.data == NULL) {
return newkey;
}
dbbt_find_token(db, key, &record, &offset, &parent);
if (offset < 1) {
return newkey;
}
if (dbbt_lseek(db, offset + BT_RECORD_SIZE, SEEK_SET) ==
(off_t) - 1) {
return newkey;
}
do {
if (dbbt_chunkread(&record, BT_RECORD_SIZE, 1, db) < 1) {
return newkey;
}
} while (record.lower & 0x80000000);
record.token[BT_TOKEN_MAX - 1] = 0;
newkey.data = (unsigned char *) strdup((char *) (record.token));
newkey.size = strlen((char *) (record.token));
return newkey;
}
/*
* Put the given record back into the binary tree, by storing it over again.
* Returns nonzero on error.
*/
static int qdb_btree_delete__rewrite(qdbint_t db, bt_record_t record)
{
qdb_datum key, val;
struct bt_record newrec;
unsigned long offset, parent;
int x;
key.data = record->token;
key.size = strlen((char *) (record->token));
val.data = (void *) (record->data);
val.size = 3 * sizeof(long);
if (qdb_btree_store(db, key, val))
return 1;
x = dbbt_find_token(db, key, &newrec, &offset, &parent);
if (offset < 4)
return 1;
newrec.lower = record->lower;
newrec.higher = record->higher;
qdb_int__sig_block();
if (dbbt_write_record(db, offset, &newrec)) {
qdb_int__sig_unblock();
return 1;
}
qdb_int__sig_unblock();
return 0;
}
/*
* Delete the given key from the database. Returns nonzero on error.
*/
int qdb_btree_delete(qdbint_t db, qdb_datum key)
{
struct bt_record record, recparent, reclower, rechigher;
unsigned long offset, parent;
int x;
if (db == NULL)
return 1;
if (key.size < 1)
return 1;
x = dbbt_find_token(db, key, &record, &offset, &parent);
if (offset < 5)
return 1;
/*
* Get a copy of the lower and higher records, if any.
*/
if (record.lower > 0) {
if (dbbt_read_record(db, record.lower, &reclower))
return 1;
}
if (record.higher > 0) {
if (dbbt_read_record(db, record.higher, &rechigher))
return 1;
}
qdb_int__sig_block();
/*
* Remove the link to this record from its parent.
*/
if (parent > 0) {
if (dbbt_read_record(db, parent, &recparent)) {
qdb_int__sig_unblock();
return 1;
}
if (x < 0) {
recparent.lower = 0;
} else {
recparent.higher = 0;
}
if (dbbt_write_record(db, parent, &recparent)) {
qdb_int__sig_unblock();
return 1;
}
}
/*
* Now we add this record's offset to the head of the "free records"
* linked list.
*/
if (dbbt_markfree(db, offset)) {
qdb_int__sig_unblock();
return 1;
}
/*
* If there were any direct children, mark them as free as well.
*/
if (record.lower > 0) {
if (dbbt_markfree(db, record.lower)) {
qdb_int__sig_unblock();
return 1;
}
}
if (record.higher > 0) {
if (dbbt_markfree(db, record.higher)) {
qdb_int__sig_unblock();
return 1;
}
}
qdb_int__sig_unblock();
/*
* Re-attach any children of this record to the database.
*/
if (record.lower > 0) {
if (qdb_btree_delete__rewrite(db, &reclower)) {
return 1;
}
}
if (record.higher > 0) {
if (qdb_btree_delete__rewrite(db, &rechigher)) {
return 1;
}
}
return 0;
}
/*
* Temporarily release the lock on the database.
*/
void qdb_btree_unlock(qdbint_t db)
{
#ifdef HAVE_FCNTL
dbbt_lock(db, F_UNLCK);
#endif
}
/*
* Reassert the lock on the database.
*/
void qdb_btree_relock(qdbint_t db)
{
#ifdef HAVE_FCNTL
dbbt_lock(db, db->locktype);
#endif
db->gotheadoffs = 0;
}
/*
* Reorganise the database for better efficiency.
*
* This is done by dumping all the tokens in this database into a temporary
* new one, then copying this temporary database over the top of the current
* database.
*/
void qdb_btree_optimise(qdbint_t db)
{
char filename[1024]; /* RATS: ignore (checked all) */
qdbint_t newdb;
qdb_datum key, val, nextkey;
int newdbfd, got;
#ifdef P_tmpdir
#ifdef HAVE_SNPRINTF
snprintf(filename, sizeof(filename),
#else
sprintf(filename, /* RATS: ignore (checked) */
#endif
"%.*s", (int) (sizeof(filename) - 1),
P_tmpdir "/qsfXXXXXX");
#else
#ifdef HAVE_SNPRINTF
snprintf(filename, sizeof(filename),
#else
sprintf(filename, /* RATS: ignore (checked) */
#endif
"%.*s", (int) (sizeof(filename) - 1), "/tmp/qsfXXXXXX");
#endif
#ifdef HAVE_MKSTEMP
newdbfd = mkstemp(filename);
#else
newdbfd = -1;
if (tmpnam(filename) != NULL) { /* RATS: ignore (OK) */
newdbfd = open(filename, /* RATS: ignore (OK) */
O_RDWR | O_CREAT | O_EXCL,
S_IRUSR | S_IWUSR);
}
#endif
if (newdbfd < 0) {
fprintf(stderr, "%s: %s: %s\n",
filename,
_("failed to create temporary file"),
strerror(errno));
return;
}
newdb = qdb_btree_open(filename, QDB_NEW);
if (newdb == NULL) {
close(newdbfd);
return;
}
key = qdb_btree_firstkey(db);
while (key.data != NULL) {
val = qdb_btree_fetch(db, key);
if (val.data != NULL) {
qdb_btree_store(newdb, key, val);
free(val.data);
}
nextkey = qdb_btree_nextkey(db, key);
free(key.data);
key = nextkey;
}
qdb_btree_close(newdb);
#ifdef HAVE_MMAP
munmap(db->data, db->size);
db->data = 0;
#endif
qdb_int__sig_block();
dbbt_lseek(db, 0, SEEK_SET);
lseek(newdbfd, 0, SEEK_SET);
do {
unsigned char buf[4096]; /* RATS: ignore (checked) */
got = dbbt_chunkread_raw(buf, 1, sizeof(buf), newdbfd);
if (got > 0)
dbbt_chunkwrite_raw(buf, 1, got, db->fd);
} while (got > 0);
if (ftruncate(db->fd, lseek(db->fd, 0, SEEK_CUR)) != 0) {
log_add(0, "%s: failed to truncate database: %s",
db->filename, strerror(errno));
}
close(newdbfd);
remove(filename);
qdb_int__sig_unblock();
db->size = dbbt_lseek(db, 0, SEEK_END);
db->gotheadoffs = 0;
#ifdef HAVE_MMAP
db->data =
mmap(NULL, db->size, PROT_READ | PROT_WRITE, MAP_SHARED,
db->fd, 0);
if (db->data == MAP_FAILED)
db->data = 0;
db->filepos = db->size;
#endif
}
/*
* Return a string describing the last database error to occur.
*/
char *qdb_btree_error(void)
{
return bt_lasterror;
}
#endif /* USING_BTREE */
/* EOF */
qsf-1.2.7/src/db/sqlite.c 0000644 0000764 0000764 00000032710 10664772303 012756 0 ustar aw aw /*
* Make our db functions a wrapper for SQLite.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "database.h"
#include "log.h"
#ifdef USING_SQLITE
#undef DEBUG_SQLITE
#include
#include
#include
#include
#include
#include
#include
#include
#ifdef HAVE_FCNTL
#include
#endif
#include
struct qdbint_s {
int fd;
sqlite *db; /* SQLite database object */
int have_value3; /* flag, set if value3 column exists */
int locked; /* flag, set if database locked */
int compiled; /* flag, set if a zSQL VM has been compiled */
const char *ztail; /* current compiled zSQL tail */
sqlite_vm *vm; /* current VM for zSQL execution */
int readonly; /* flag, set if this database is readonly */
};
static char qdb_sqlite__errstr[4096] = { 0, };
static char *qdb_sqlite__errptr = 0;
#if DEBUG_SQLITE
/*
* Debugging function to output every SQL statement passed to SQLite.
*/
static void qdb_sqlite__debug(void *ptr, const char *str)
{
printf("[%s]\n", str);
}
#endif
/*
* Return a file descriptor for the given database, or -1 on error.
*/
int qdb_sqlite_fd(qdbint_t db)
{
if (db == NULL)
return -1;
return db->fd;
}
/*
* Temporarily release the lock on the database.
*/
void qdb_sqlite_unlock(qdbint_t db)
{
if (db == NULL)
return;
if (!db->locked)
return;
/*
* We "unlock" by ending the current transaction.
*/
sqlite_exec(db->db, "COMMIT", NULL, NULL, &qdb_sqlite__errptr);
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
db->locked = 0;
}
/*
* Reassert the lock on the database.
*/
void qdb_sqlite_relock(qdbint_t db)
{
int tries, ret;
if (db == NULL)
return;
if (db->locked)
return;
/*
* We "lock" by starting a new transaction.
*/
for (tries = 0; tries < 60 && !db->locked; tries++) {
ret =
sqlite_exec(db->db, "BEGIN", NULL, NULL,
&qdb_sqlite__errptr);
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
switch (ret) {
case SQLITE_OK:
db->locked = 1;
break;
case SQLITE_BUSY:
sleep(1);
break;
default:
break;
}
}
}
/*
* Return nonzero if the given file is of this database type.
*/
int qdb_sqlite_identify(const char *file)
{
FILE *fptr;
unsigned char buf[64]; /* RATS: ignore (checked) */
if (file == NULL)
return 0;
if (strncasecmp(file, "sqlite:", 7) == 0)
return 1;
if (strncasecmp(file, "sqlite2:", 8) == 0)
return 1;
fptr = fopen(file, "rb");
if (fptr == NULL)
return 0;
if (fread(buf, 33, 1, fptr) < 1) {
fclose(fptr);
return 0;
}
fclose(fptr);
if (strncmp((char *) buf, "** This file contains an SQLite 2", 33)
== 0)
return 1;
return 0;
}
/*
* Open the given database in the given way (new database, read-only, or
* read-write); return a qdbint_t or NULL on error.
*/
qdbint_t qdb_sqlite_open(const char *file, qdb_open_t method)
{
qdbint_t db;
char **results;
int rows, cols;
if (strncasecmp(file, "sqlite:", 7) == 0)
file += 7;
if (strncasecmp(file, "sqlite2:", 8) == 0)
file += 8;
db = calloc(1, sizeof(*db));
if (db == NULL) {
sprintf(qdb_sqlite__errstr, "%.999s", strerror(errno));
return NULL;
}
db->fd = -1;
db->readonly = (method == QDB_READONLY ? 1 : 0);
db->have_value3 = 1;
/*
* If we need write access, and the file exists, check we can
* physically write to the file first, since sqlite_open() seems not
* to care.
*/
if (!db->readonly) {
db->fd = open(file, O_RDWR);
if (db->fd < 0) {
if (errno != ENOENT) {
sprintf(qdb_sqlite__errstr, "%.999s",
strerror(errno));
free(db);
return NULL;
}
}
}
db->db = sqlite_open(file, 0, &qdb_sqlite__errptr);
if (db->db == NULL) {
sprintf(qdb_sqlite__errstr, "%.999s",
qdb_sqlite__errptr ? qdb_sqlite__errptr : "");
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
free(db);
return NULL;
}
if (method != QDB_READONLY) {
/*
* We deliberately ignore errors at this point, in case the
* database table has already been created.
*/
sqlite_exec(db->db, "CREATE TABLE qsf ("
" token CHAR(64) NOT NULL PRIMARY KEY,"
" value1 INT UNSIGNED NOT NULL,"
" value2 INT UNSIGNED NOT NULL,"
" value3 INT UNSIGNED NOT NULL"
" )", NULL, NULL, &qdb_sqlite__errptr);
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
qdb_sqlite_relock(db);
}
#if DEBUG_SQLITE
sqlite_trace(db->db, qdb_sqlite__debug, NULL);
#endif
if (sqlite_get_table
(db->db, "SELECT value3 FROM qsf LIMIT 0,1", &results, &rows,
&cols, &qdb_sqlite__errptr) != 0) {
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
db->have_value3 = 0;
log_add(1, "%s",
_("warning: old-style 2-value SQLite table"));
} else {
sqlite_free_table(results);
}
db->fd = open(file, O_RDONLY);
return db;
}
/*
* Close the given database.
*/
void qdb_sqlite_close(qdbint_t db)
{
if (db == NULL)
return;
if (db->compiled) {
sqlite_finalize(db->vm, &qdb_sqlite__errptr);
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
}
qdb_sqlite_unlock(db);
if (db->fd >= 0)
close(db->fd);
sqlite_close(db->db);
free(db);
}
/*
* Construct a database query string, replacing any ? characters with an
* escaped quoted (') copy of the next string in the argument list. Each
* string should be given as a (long) length, not including any terminating
* \0, and the char * pointer to the string.
*
* Returns a null-terminated string that needs free()ing after use, or NULL
* on error.
*/
char *qdb_sqlite__query(char *format, ...)
{
char *buf;
long len, offs;
va_list ap;
int i;
long paramlen;
char *param;
va_start(ap, format);
len = 0;
for (i = 0; format[i] != 0; i++) {
if (format[i] == '?') {
paramlen = va_arg(ap, long);
param = va_arg(ap, char *);
len += (paramlen * 3) + 2;
} else {
len++;
}
}
va_end(ap);
len += 2;
buf = malloc(len);
if (buf == NULL) {
return NULL;
}
offs = 0;
va_start(ap, format);
for (i = 0; format[i] != 0; i++) {
if (format[i] == '?') {
int j;
buf[offs++] = '\'';
paramlen = va_arg(ap, long);
param = va_arg(ap, char *);
for (j = 0; j < paramlen; j++) {
int c;
c = param[j];
if (c == '\'') {
buf[offs++] = '\'';
}
buf[offs++] = c;
}
buf[offs++] = '\'';
} else {
buf[offs++] = format[i];
}
}
buf[offs++] = 0;
va_end(ap);
return buf;
}
/*
* Fetch a value from the database. The datum returned needs its val.data
* free()ing after use. If val.data is NULL, no value was found for the
* given key.
*/
qdb_datum qdb_sqlite_fetch(qdbint_t db, qdb_datum key)
{
qdb_datum val;
char *sql;
char **results;
int rows, cols;
long val1, val2, val3;
long *data;
val.data = NULL;
val.size = 0;
if (db == NULL)
return val;
if (key.data == NULL)
return val;
if (db->have_value3) {
sql =
qdb_sqlite__query
("SELECT value1,value2,value3 FROM qsf "
"WHERE token=?", (long) (key.size),
(char *) (key.data));
} else {
sql = qdb_sqlite__query("SELECT value1,value2 FROM qsf "
"WHERE token=?",
(long) (key.size),
(char *) (key.data));
}
if (sql == NULL)
return val;
if (sqlite_get_table
(db->db, sql, &results, &rows, &cols,
&qdb_sqlite__errptr) != 0) {
sprintf(qdb_sqlite__errstr, "%.999s",
qdb_sqlite__errptr ? qdb_sqlite__errptr : "");
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
free(sql);
return val;
}
free(sql);
if ((cols < 2) || (rows != 1)) {
sqlite_free_table(results);
return val;
}
if (db->have_value3) {
val1 = strtol(results[3], NULL, 10);
val2 = strtol(results[4], NULL, 10);
val3 = strtol(results[5], NULL, 10);
} else {
val1 = strtol(results[2], NULL, 10);
val2 = strtol(results[3], NULL, 10);
val3 = 0;
}
sqlite_free_table(results);
data = malloc(3 * sizeof(long));
if (data == NULL) {
sprintf(qdb_sqlite__errstr, "%.999s", strerror(errno));
return val;
}
data[0] = val1;
data[1] = val2;
data[2] = val3;
val.data = (unsigned char *) data;
val.size = 3 * sizeof(long);
return val;
}
/*
* Store the given key with the given value into the database, replacing any
* existing value for that key. Returns nonzero on error.
*/
int qdb_sqlite_store(qdbint_t db, qdb_datum key, qdb_datum val)
{
long *data;
char val1[128]; /* RATS: ignore (checked) */
char val2[128]; /* RATS: ignore (checked) */
char val3[128]; /* RATS: ignore (checked) */
char *sql;
if (db == NULL)
return 1;
if ((key.data == NULL) || (val.data == NULL))
return 1;
if (db->readonly)
return 1;
data = (long *) (val.data);
#ifdef HAVE_SNPRINTF
snprintf(val1, sizeof(val1) - 1, /* RATS: ignore (OK) */
"%ld", data[0]);
snprintf(val2, sizeof(val2) - 1, /* RATS: ignore (OK) */
"%ld", data[1]);
snprintf(val3, sizeof(val3) - 1, /* RATS: ignore (OK) */
"%ld", data[2]);
#else
sprintf(val1, "%ld", data[0]);
sprintf(val2, "%ld", data[1]);
sprintf(val3, "%ld", data[2]);
#endif
val1[sizeof(val1) - 1] = 0;
val2[sizeof(val2) - 1] = 0;
val3[sizeof(val3) - 1] = 0;
if (db->have_value3) {
sql =
qdb_sqlite__query
("INSERT INTO qsf (token,value1,value2,value3) "
"VALUES (?,?,?,?)", (long) (key.size),
(char *) (key.data), (long) (strlen(val1)), val1,
(long) (strlen(val2)), val2, (long) (strlen(val3)),
val3);
} else {
sql =
qdb_sqlite__query
("INSERT INTO qsf (token,value1,value2) "
"VALUES (?,?,?)", (long) (key.size),
(char *) (key.data), (long) (strlen(val1)), val1,
(long) (strlen(val2)), val2);
}
if (sql == NULL)
return 1;
if (sqlite_exec(db->db, sql, NULL, NULL, &qdb_sqlite__errptr) != 0) {
free(sql);
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
if (db->have_value3) {
sql = qdb_sqlite__query("UPDATE qsf "
"SET value1=?,value2=?,value3=? "
"WHERE token=?",
(long) (strlen(val1)),
val1,
(long) (strlen(val2)),
val2,
(long) (strlen(val3)),
val3, (long) (key.size),
(char *) (key.data));
} else {
sql = qdb_sqlite__query("UPDATE qsf "
"SET value1=?,value2=? "
"WHERE token=?",
(long) (strlen(val1)),
val1,
(long) (strlen(val2)),
val2, (long) (key.size),
(char *) (key.data));
}
if (sql == NULL)
return 1;
if (sqlite_exec
(db->db, sql, NULL, NULL, &qdb_sqlite__errptr) != 0) {
free(sql);
sprintf(qdb_sqlite__errstr, "%.999s",
qdb_sqlite__errptr ? qdb_sqlite__errptr :
"");
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
return 1;
}
}
free(sql);
return 0;
}
/*
* Delete the given key from the database. Returns nonzero on error.
*/
int qdb_sqlite_delete(qdbint_t db, qdb_datum key)
{
char *sql;
if (db == NULL)
return 1;
if (key.data == NULL)
return 1;
if (db->readonly)
return 1;
sql = qdb_sqlite__query("DELETE FROM qsf WHERE token=?",
(long) (key.size), (char *) (key.data));
if (sql == NULL)
return 1;
if (sqlite_exec(db->db, sql, NULL, NULL, &qdb_sqlite__errptr) != 0) {
free(sql);
sprintf(qdb_sqlite__errstr, "%.999s",
qdb_sqlite__errptr ? qdb_sqlite__errptr : "");
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
return 1;
}
free(sql);
return 0;
}
/*
* Return the "next" key in the database, or key.data=NULL when all keys
* have been returned.
*/
qdb_datum qdb_sqlite_nextkey(qdbint_t db, qdb_datum key)
{
qdb_datum newkey;
char **values;
char **colnames;
char *token;
int ret, n;
newkey.data = NULL;
newkey.size = 0;
if (!db->compiled)
return newkey;
ret =
sqlite_step(db->vm, &n, (const char ***) (&values),
(const char ***) (&colnames));
if (ret != SQLITE_ROW) {
sqlite_finalize(db->vm, &qdb_sqlite__errptr);
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
db->compiled = 0;
return newkey;
}
token = values[0];
newkey.data = (unsigned char *) calloc(1, 1 + strlen(token));
if (newkey.data == NULL) {
sprintf(qdb_sqlite__errstr, "%.999s", strerror(errno));
return newkey;
}
newkey.size = strlen(token);
strncpy((char *) (newkey.data), token, newkey.size);
return newkey;
}
/*
* Return the "first" key in the database, suitable for using with repeated
* calls to qdb_nextkey() to walk through every key in the database.
*/
qdb_datum qdb_sqlite_firstkey(qdbint_t db)
{
qdb_datum blankkey;
blankkey.data = NULL;
blankkey.size = 0;
if (db->compiled) {
sqlite_finalize(db->vm, &qdb_sqlite__errptr);
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
}
db->compiled = 0;
if (sqlite_compile
(db->db, "SELECT token FROM qsf ORDER BY token", &(db->ztail),
&(db->vm), &qdb_sqlite__errptr) != 0) {
sprintf(qdb_sqlite__errstr, "%.999s",
qdb_sqlite__errptr ? qdb_sqlite__errptr : "");
if (qdb_sqlite__errptr) {
sqlite_freemem(qdb_sqlite__errptr);
qdb_sqlite__errptr = 0;
}
return blankkey;
}
db->compiled = 1;
return qdb_sqlite_nextkey(db, blankkey);
}
/*
* Return a string describing the last database error to occur.
*/
char *qdb_sqlite_error(void)
{
return qdb_sqlite__errstr;
}
#endif /* USING_SQLITE */
/* EOF */
qsf-1.2.7/src/db/mysql.c 0000644 0000764 0000764 00000041061 10664772303 012621 0 ustar aw aw /*
* Make our db functions a wrapper for MySQL.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "database.h"
#include "log.h"
#ifdef USING_MYSQL
#include
#include
#include
#include
#include
#include
#include
#ifdef HAVE_FCNTL
#include
#endif
#include
#include
struct qdbint_s {
int fd;
MYSQL *db;
int readonly;
int have_value3;
char table[64]; /* RATS: ignore (checked all) */
char key1[256]; /* RATS: ignore (checked all) */
char key2[256]; /* RATS: ignore (checked all) */
MYSQL_RES *results;
};
static char qdb_mysql__errstr[4096] = { 0, }; /* RATS: ignore (bounded OK) */
/*
* Return a file descriptor for the given database, or -1 on error.
*/
int qdb_mysql_fd(qdbint_t db)
{
if (db == NULL)
return -1;
return db->fd;
}
/*
* Return nonzero if the given file is of this database type.
*/
int qdb_mysql_identify(const char *file)
{
FILE *fptr;
char string[4096] = { 0, }; /* RATS: ignore (checked all) */
char host[256]; /* RATS: ignore (checked all) */
char user[64]; /* RATS: ignore (checked all) */
char pass[64]; /* RATS: ignore (checked all) */
char database[64]; /* RATS: ignore (checked all) */
char table[64]; /* RATS: ignore (checked all) */
char key1[256]; /* RATS: ignore (checked all) */
char key2[256]; /* RATS: ignore (checked all) */
unsigned int port;
if (file == NULL)
return 0;
if (strncasecmp(file, "mysql:", 6) == 0)
return 1;
fptr = fopen(file, "r");
if (!fptr) {
sprintf(string, "%.*s", (int) (sizeof(string) - 1), file);
} else {
if (fread(string, 1, sizeof(string) - 1, fptr) < 1) {
log_add(0, "%s: read failed: %s", file,
strerror(errno));
}
fclose(fptr);
}
string[sizeof(string) - 1] = 0;
if (sscanf(string, "database=%63[^;];"
"host=%255[^;];"
"port=%u;"
"user=%63[^;];"
"pass=%63[^;];"
"table=%63[^;];"
"key1=%255[^;];"
"key2=%255[^;]",
database,
host, &port, user, pass, table, key1, key2) == 8)
return 1;
if (sscanf(string, "database=%63[^;];"
"host=%255[^;];"
"port=%u;"
"user=%63[^;];"
"pass=;"
"table=%63[^;];"
"key1=%255[^;];"
"key2=%255[^;]",
database, host, &port, user, table, key1, key2) == 7)
return 1;
return 0;
}
/*
* Initiate a database query, replacing any ? characters with an escaped
* quoted (') copy of the next string in the argument list, and replacing
* any ! characters with a similarly escaped string but quoted with
* backquotes (`). Each string should be given as a (long) length, not
* including any terminating \0, and the char * pointer to the string.
*
* Returns nonzero on error.
*/
int qdb_mysql__query(qdbint_t db, char *format, ...)
{
char *buf;
long len, offs;
va_list ap;
int i, ret;
long paramlen;
char *param;
va_start(ap, format);
len = 0;
for (i = 0; format[i] != 0; i++) {
if (format[i] == '?') {
paramlen = va_arg(ap, long);
param = va_arg(ap, char *);
len += (paramlen * 3) + 2;
} else if (format[i] == '!') {
paramlen = va_arg(ap, long);
param = va_arg(ap, char *);
len += (paramlen * 3) + 2;
} else {
len++;
}
}
va_end(ap);
len++;
buf = malloc(len);
if (buf == NULL) {
return 1;
}
offs = 0;
va_start(ap, format);
for (i = 0; format[i] != 0; i++) {
if (format[i] == '?') {
buf[offs++] = '\'';
paramlen = va_arg(ap, long);
param = va_arg(ap, char *);
offs +=
mysql_real_escape_string(db->db,
buf + offs, param,
paramlen);
buf[offs++] = '\'';
} else if (format[i] == '!') {
buf[offs++] = '`';
paramlen = va_arg(ap, long);
param = va_arg(ap, char *);
offs +=
mysql_real_escape_string(db->db,
buf + offs, param,
paramlen);
buf[offs++] = '`';
} else {
buf[offs++] = format[i];
}
}
va_end(ap);
ret = mysql_real_query(db->db, buf, offs);
free(buf);
return ret;
}
/*
* Open the given database in the given way (new database, read-only, or
* read-write); return a qdbint_t or NULL on error.
*/
qdbint_t qdb_mysql_open(const char *file, qdb_open_t method)
{
char host[256]; /* RATS: ignore (checked all) */
char user[64]; /* RATS: ignore (checked all) */
char pass[64]; /* RATS: ignore (checked all) */
char database[64]; /* RATS: ignore (checked all) */
char table[64]; /* RATS: ignore (checked all) */
char key1[256]; /* RATS: ignore (checked all) */
char key2[256]; /* RATS: ignore (checked all) */
char string[4096] = { 0, }; /* RATS: ignore (checked all) */
unsigned int port;
FILE *fptr;
MYSQL *mdb;
qdbint_t db;
if (strncasecmp(file, "mysql:", 6) == 0)
file += 6;
mdb = mysql_init(NULL);
if (mdb == NULL) {
sprintf(qdb_mysql__errstr, "%.*s",
(int) (sizeof(qdb_mysql__errstr) - 1),
_("mysql_init failed"));
return NULL;
}
fptr = fopen(file, "r"); /* RATS: ignore (no race) */
if (!fptr) {
sprintf(string, "%.*s", (int) (sizeof(string) - 1), file);
} else {
if (fread(string, 1, sizeof(string) - 1, fptr) < 1) {
log_add(0, "%s: read failed: %s", file,
strerror(errno));
}
fclose(fptr);
}
string[sizeof(string) - 1] = 0;
pass[0] = 0;
if (sscanf(string, "database=%63[^;];"
"host=%255[^;];"
"port=%u;"
"user=%63[^;];"
"pass=;"
"table=%63[^;];"
"key1=%255[^;];"
"key2=%255[^;]",
database, host, &port, user, table, key1, key2) < 7) {
if (sscanf(string, "database=%63[^;];"
"host=%255[^;];"
"port=%u;"
"user=%63[^;];"
"pass=%63[^;];"
"table=%63[^;];"
"key1=%255[^;];"
"key2=%255[^;]",
database,
host, &port, user, pass, table, key1,
key2) < 8) {
sprintf(qdb_mysql__errstr, "%.*s: %.100s",
(int) (sizeof(qdb_mysql__errstr) - 110),
_("invalid DB spec"), string);
mysql_close(mdb);
return NULL;
}
}
/*
* Make sure all the strings we have just read are zero-terminated.
*/
database[sizeof(database) - 1] = 0;
host[sizeof(host) - 1] = 0;
user[sizeof(user) - 1] = 0;
pass[sizeof(pass) - 1] = 0;
table[sizeof(table) - 1] = 0;
key1[sizeof(key1) - 1] = 0;
key2[sizeof(key2) - 1] = 0;
if (mysql_real_connect
(mdb, host, user, pass, database, port, NULL, 0) == NULL) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("mysql connect failed"), mysql_error(mdb));
mysql_close(mdb);
return NULL;
}
db = calloc(1, sizeof(*db));
if (db == NULL) {
sprintf(qdb_mysql__errstr, "%.999s", strerror(errno));
mysql_close(mdb);
return NULL;
}
db->fd = open(file, O_RDONLY);
db->db = mdb;
strncpy(db->table, table, sizeof(table) - 1);
strncpy(db->key1, key1, sizeof(key1) - 1);
strncpy(db->key2, key2, sizeof(key2) - 1);
/*
* Make sure db->{table,key1,key2} are zero-terminated.
*/
db->table[sizeof(db->table) - 1] = 0;
db->key1[sizeof(db->key1) - 1] = 0;
db->key2[sizeof(db->key2) - 1] = 0;
db->readonly = (method == QDB_READONLY ? 1 : 0);
/*
* See whether the given table exists; if not, and we're not
* supposed to be in readonly mode, try to create it.
*/
if (!db->readonly) {
if (qdb_mysql__query(db, "SELECT value1 FROM ! LIMIT 0,1",
(long) (strlen(db->table)),
db->table)) {
if (qdb_mysql__query(db, "CREATE TABLE ! ("
"key1 BIGINT UNSIGNED NOT NULL, "
"key2 BIGINT UNSIGNED NOT NULL, "
"token VARCHAR(64) DEFAULT '' NOT NULL, "
"value1 INT UNSIGNED NOT NULL, "
"value2 INT UNSIGNED NOT NULL, "
"value3 INT UNSIGNED NOT NULL, "
"PRIMARY KEY (key1,key2,token), "
"KEY (key1), "
"KEY (key2), "
"KEY (token) "
")",
(long) (strlen(db->table)),
db->table) == 0) {
MYSQL_RES *results;
results = mysql_store_result(db->db);
if (results)
mysql_free_result(results);
log_add(1, "%s: %s", db->table,
_("table autocreated"));
} else {
log_add(1, "%s: %s", db->table,
_("invalid table specified"));
}
} else {
MYSQL_RES *results;
results = mysql_store_result(db->db);
if (results)
mysql_free_result(results);
}
}
db->have_value3 = 1;
if (qdb_mysql__query(db, "SELECT value3 FROM ! LIMIT 0,1",
(long) (strlen(db->table)), db->table)) {
db->have_value3 = 0;
log_add(1, "%s",
_("warning: old-style 2-value MySQL table"));
} else {
MYSQL_RES *results;
results = mysql_store_result(db->db);
if (results)
mysql_free_result(results);
}
switch (method) {
case QDB_NEW:
/* fall-through */
case QDB_READWRITE:
break;
case QDB_READONLY:
break;
default:
break;
}
#ifdef HAVE_MYSQL_AUTOCOMMIT
mysql_autocommit(db->db, 0);
#endif
db->results = NULL;
return db;
}
/*
* Close the given database.
*/
void qdb_mysql_close(qdbint_t db)
{
if (db == NULL)
return;
#ifdef HAVE_MYSQL_AUTOCOMMIT
mysql_commit(db->db);
#endif
if (db->fd >= 0)
close(db->fd);
mysql_close(db->db);
free(db);
}
/*
* Fetch a value from the database. The datum returned needs its val.data
* free()ing after use. If val.data is NULL, no value was found for the
* given key.
*/
qdb_datum qdb_mysql_fetch(qdbint_t db, qdb_datum key)
{
qdb_datum val;
char *sql;
MYSQL_RES *results;
MYSQL_ROW row;
unsigned long *lengths;
long *data;
char buf[256]; /* RATS: ignore (checked all) */
val.data = NULL;
val.size = 0;
if (db == NULL)
return val;
if (key.data == NULL)
return val;
if (db->have_value3) {
sql = "SELECT value1,value2,value3 FROM ! "
"WHERE key1 = ? " "AND key2 = ? " "AND token = ?";
} else {
sql = "SELECT value1,value2 FROM ! "
"WHERE key1 = ? " "AND key2 = ? " "AND token = ?";
}
if (qdb_mysql__query(db, sql,
(long) (strlen(db->table)), db->table,
(long) (strlen(db->key1)), db->key1,
(long) (strlen(db->key2)), db->key2,
(long) (key.size), (char *) (key.data))) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("query failed"), mysql_error(db->db));
return val;
}
results = mysql_store_result(db->db);
if (results == NULL) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("query failed on results store"),
mysql_error(db->db));
return val;
}
row = mysql_fetch_row(results);
if (row == NULL) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("query failed on row fetch"),
mysql_error(db->db));
mysql_free_result(results);
return val;
}
lengths = mysql_fetch_lengths(results);
if (lengths == NULL) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("query failed on lengths fetch"),
mysql_error(db->db));
mysql_free_result(results);
return val;
}
data = malloc(3 * sizeof(long));
if (data == NULL) {
sprintf(qdb_mysql__errstr, "%.999s", strerror(errno));
mysql_free_result(results);
return val;
}
sprintf(buf, "%.*s",
(int) ((lengths[0] <
sizeof(buf) - 1) ? lengths[0] : sizeof(buf) - 1),
row[0]);
data[0] = strtol(buf, NULL, 10);
sprintf(buf, "%.*s",
(int) ((lengths[1] <
sizeof(buf) - 1) ? lengths[1] : sizeof(buf) - 1),
row[1]);
data[1] = strtol(buf, NULL, 10);
if (db->have_value3) {
sprintf(buf, "%.*s",
(int) ((lengths[2] <
sizeof(buf) -
1) ? lengths[2] : sizeof(buf) - 1),
row[2]);
data[2] = strtol(buf, NULL, 10);
} else {
data[2] = 0;
}
mysql_free_result(results);
val.data = (unsigned char *) data;
val.size = 3 * sizeof(long);
return val;
}
/*
* Store the given key with the given value into the database, replacing any
* existing value for that key. Returns nonzero on error.
*/
int qdb_mysql_store(qdbint_t db, qdb_datum key, qdb_datum val)
{
long *data;
char val1[128]; /* RATS: ignore (checked) */
char val2[128]; /* RATS: ignore (checked) */
char val3[128]; /* RATS: ignore (checked) */
if (db == NULL)
return 1;
if ((key.data == NULL) || (val.data == NULL))
return 1;
if (db->readonly)
return 1;
data = (long *) (val.data);
#ifdef HAVE_SNPRINTF
snprintf(val1, sizeof(val1) - 1, /* RATS: ignore (OK) */
"%ld", data[0]);
snprintf(val2, sizeof(val2) - 1, /* RATS: ignore (OK) */
"%ld", data[1]);
snprintf(val3, sizeof(val3) - 1, /* RATS: ignore (OK) */
"%ld", data[2]);
#else
sprintf(val1, "%ld", data[0]);
sprintf(val2, "%ld", data[1]);
sprintf(val3, "%ld", data[2]);
#endif
val1[sizeof(val1) - 1] = 0;
val2[sizeof(val2) - 1] = 0;
val3[sizeof(val3) - 1] = 0;
if (db->have_value3) {
if (qdb_mysql__query(db, "REPLACE INTO ! "
"(key1,key2,token,value1,value2,value3) "
"VALUES (?,?,?,?,?,?)",
(long) (strlen(db->table)), db->table,
(long) (strlen(db->key1)), db->key1,
(long) (strlen(db->key2)), db->key2,
(long) (key.size),
(char *) (key.data),
(long) (strlen(val1)), val1,
(long) (strlen(val2)), val2,
(long) (strlen(val3)), val3)) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("store failed"), mysql_error(db->db));
return 1;
}
} else {
if (qdb_mysql__query(db, "REPLACE INTO ! "
"(key1,key2,token,value1,value2) "
"VALUES (?,?,?,?,?)",
(long) (strlen(db->table)), db->table,
(long) (strlen(db->key1)), db->key1,
(long) (strlen(db->key2)), db->key2,
(long) (key.size),
(char *) (key.data),
(long) (strlen(val1)), val1,
(long) (strlen(val2)), val2)) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("store failed"), mysql_error(db->db));
return 1;
}
}
return 0;
}
/*
* Delete the given key from the database. Returns nonzero on error.
*/
int qdb_mysql_delete(qdbint_t db, qdb_datum key)
{
if (db == NULL)
return 1;
if (key.data == NULL)
return 1;
if (db->readonly)
return 1;
if (qdb_mysql__query(db, "DELETE FROM ! "
"WHERE key1=? "
"AND key2=? "
"AND token=?",
(long) (strlen(db->table)), db->table,
(long) (strlen(db->key1)), db->key1,
(long) (strlen(db->key2)), db->key2,
(long) (key.size), (char *) (key.data))) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("delete failed"), mysql_error(db->db));
return 1;
}
return 0;
}
/*
* Return the "next" key in the database, or key.data=NULL when all keys
* have been returned.
*/
qdb_datum qdb_mysql_nextkey(qdbint_t db, qdb_datum key)
{
MYSQL_ROW row;
unsigned long *lengths;
qdb_datum newkey;
newkey.data = NULL;
newkey.size = 0;
row = mysql_fetch_row(db->results);
if (row == NULL) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("row fetch failed"), mysql_error(db->db));
mysql_free_result(db->results);
db->results = NULL;
return newkey;
}
lengths = mysql_fetch_lengths(db->results);
if (lengths == NULL) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("row lengths fetch failed"),
mysql_error(db->db));
mysql_free_result(db->results);
db->results = NULL;
return newkey;
}
newkey.data = (unsigned char *) calloc(1, lengths[0] + 1);
if (newkey.data == NULL) {
sprintf(qdb_mysql__errstr, "%.999s", strerror(errno));
return newkey;
}
newkey.size = lengths[0];
strncpy((char *) (newkey.data), row[0], lengths[0]);
return newkey;
}
/*
* Return the "first" key in the database, suitable for using with repeated
* calls to qdb_nextkey() to walk through every key in the database.
*/
qdb_datum qdb_mysql_firstkey(qdbint_t db)
{
qdb_datum blankkey;
blankkey.data = NULL;
blankkey.size = 0;
if (db->results)
mysql_free_result(db->results);
db->results = NULL;
if (qdb_mysql__query(db, "SELECT token FROM ! "
"WHERE key1 = ? "
"AND key2 = ? "
"ORDER BY token",
(long) (strlen(db->table)), db->table,
(long) (strlen(db->key1)), db->key1,
(long) (strlen(db->key2)), db->key2)) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("select failed"), mysql_error(db->db));
return blankkey;
}
db->results = mysql_store_result(db->db);
if (db->results == NULL) {
sprintf(qdb_mysql__errstr, "%.*s: %.900s",
(int) (sizeof(qdb_mysql__errstr) - 910),
_("result store failed"), mysql_error(db->db));
return blankkey;
}
return qdb_mysql_nextkey(db, blankkey);
}
/*
* Return a string describing the last database error to occur.
*/
char *qdb_mysql_error(void)
{
return qdb_mysql__errstr;
}
#endif /* USING_MYSQL */
/* EOF */
qsf-1.2.7/src/main/ 0000755 0000764 0000764 00000000000 10665021416 011636 5 ustar aw aw qsf-1.2.7/src/main/help.c 0000644 0000764 0000764 00000011366 10664772303 012750 0 ustar aw aw /*
* Output command-line help to stdout.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include
#include
#include
#include
struct optdesc_s {
char *optshort;
char *optlong;
char *param;
char *description;
};
/*
* Display command-line help.
*/
void display_help(void)
{
struct optdesc_s optlist[] = {
{"-d", "--database", _("FILE"),
_("use FILE as a database")},
{"-g", "--global", _("FILE"),
_("use FILE as a global (readonly) database")},
{"-P", "--plain-map", _("FILE"),
_("store plaintext token map in FILE")},
{"-s", "--subject", 0,
_("add \"[SPAM]\" to Subject: of spam")},
{"-S", "--subject-marker", _("MARK"),
_("add \"MARK\" instead of \"[SPAM]\"")},
{"-H", "--header-marker", _("MARK"),
_("set X-Spam header to \"MARK\" instead of \"YES\"")},
{"-n", "--no-header", 0,
_("do not add X-Spam header to spam")},
{"-r", "--add-rating", 0,
_("add an X-Spam-Rating header (0-100, 90+ = spam)")},
{"-A", "--asterisk", 0,
_("add an X-Spam-Level header (0-20 *s)")},
{"-t", "--test", 0,
_("do not filter, just test for spam")},
{"-a", "--allowlist", 0,
_("enable the allow-list")},
{"-y", "--denylist", 0,
_("enable the deny-list")},
{"-L", "--level", _("LEVEL"),
_("set spam threshold level to LEVEL, not 90")},
{"-Q", "--min-tokens", _("NUM"),
_("only give a score if more than NUM tokens found")},
{"-e", "--email", _("EMAIL"),
_
("update/query allow-list for given EMAIL (set to \"MSG\" to use sender address)")},
{"-v", "--verbose", 0,
_("add informational headers to email")},
{"", 0, 0, 0},
{"-T", "--train", _("SPAM NON"),
_("train database using SPAM and NON mbox folders")},
{"-m", "--mark-spam", 0,
_("mark incoming message as spam in the database")},
{"-M", "--mark-nonspam", 0,
_("mark incoming message as non-spam in the database")},
{"-w", "--weight", _("WEIGHT"),
_("mark message with weight of WEIGHT instead of 1")},
/*
* Deprecated options - don't list in the help.
{"-N", "--no-autoprune", 0,
_("do not automatically prune after every 500th entry")},
{"-p", "--prune", 0,
_("remove redundant entries from the database")},
{"-X", "--prune-max", _("NUM"),
_("prune at most NUM tokens, rather than 100000")},
*/
{"", 0, 0, 0},
{"-D", "--dump", _("[FILE]"),
_("dump database as text to FILE or stdout")},
{"-R", "--restore", _("[FILE]"),
_("restore database from text FILE or stdin")},
{"-O", "--tokens", 0,
_("list tokens found in a message")},
{"-E", "--merge", _("OTHERDB"),
_("merge OTHERDB into current database")},
{"", 0, 0, 0},
{"-h", "--help", 0,
_("show this help and exit")},
{"-V", "--version", 0,
_("show version information and exit")},
{0, 0, 0, 0}
};
int i, col1max = 0, tw = 77;
char *optbuf;
int sz;
printf(_("Usage: %s [OPTION]..."), /* RATS: ignore */
PROGRAM_NAME);
printf("\n%s\n\n",
_
("Read the message on standard input and output it with an X-Spam header\n"
"if it is spam."));
for (i = 0; optlist[i].optshort; i++) {
int width = 0;
width = 2 + strlen(optlist[i].optshort); /* RATS: ignore */
#ifdef HAVE_GETOPT_LONG
if (optlist[i].optlong)
width += 2 + strlen(optlist[i].optlong); /* RATS: ignore */
#endif
if (optlist[i].param)
width += 1 + strlen(optlist[i].param); /* RATS: ignore */
if (width > col1max)
col1max = width;
}
col1max++;
sz = col1max + 16;
optbuf = malloc(sz);
if (optbuf == NULL) {
fprintf(stderr, "%s: %s\n", PROGRAM_NAME, strerror(errno));
exit(1);
}
for (i = 0; optlist[i].optshort; i++) {
char *start;
char *end;
if (optlist[i].optshort[0] == 0) {
printf("\n");
continue;
}
#ifdef HAVE_SNPRINTF
snprintf(optbuf, sz, "%s%s%s%s%s", /* RATS: ignore (checked) */
#else
sprintf(optbuf, "%s%s%s%s%s", /* RATS: ignore (checked) */
#endif
optlist[i].optshort,
#ifdef HAVE_GETOPT_LONG
optlist[i].optlong ? ", " : "",
optlist[i].optlong ? optlist[i].optlong : "",
#else
"", "",
#endif
optlist[i].param ? " " : "",
optlist[i].param ? optlist[i].param : "");
printf(" %-*s ", col1max - 2, optbuf);
if (optlist[i].description == NULL) {
printf("\n");
continue;
}
start = optlist[i].description;
while (strlen(start) /* RATS: ignore */ >tw - col1max) {
end = start + tw - col1max;
while ((end > start) && (end[0] != ' '))
end--;
if (end == start) {
end = start + tw - col1max;
} else {
end++;
}
printf("%.*s\n%*s ", (int) (end - start), start,
col1max, "");
if (end == start)
end++;
start = end;
}
printf("%s\n", start);
}
printf("\n");
printf(_("Please report any bugs to %s."), /* RATS: ignore */
BUG_REPORTS_TO);
printf("\n");
}
/* EOF */
qsf-1.2.7/src/main/options.c 0000644 0000764 0000764 00000017420 10664772303 013510 0 ustar aw aw /*
* Parse command-line options.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "options.h"
#include "spam.h"
#include "log.h"
#include
#include
#include
#include
#ifdef HAVE_GETOPT_H
#include
#endif
#include
#ifndef HAVE_GETOPT
int minigetopt(int, char **, char *);
extern char *minioptarg;
extern int minioptind, miniopterr, minioptopt;
#define getopt minigetopt
#define optarg minioptarg
#define optind minioptind
#define opterr miniopterr
#define optopt minioptopt
#endif /* !HAVE_GETOPT */
void display_help(void);
void display_version(void);
/*
* Free an opts_t object.
*/
void opts_free(opts_t opts)
{
if (!opts)
return;
if (opts->argv)
free(opts->argv);
if (opts->plaindata)
spam_plaintext_free(opts);
free(opts);
}
/*
* Parse the given command-line arguments into an opts_t object, handling
* "help" and "version" options internally.
*
* Returns an opts_t, or 0 on error.
*
* Note that the contents of *argv[] (i.e. the command line parameters)
* aren't copied anywhere, just the pointers are copied, so make sure the
* command line data isn't overwritten or argv[1] free()d or whatever.
*/
opts_t opts_parse(int argc, char **argv)
{
#ifdef HAVE_GETOPT_LONG
struct option long_options[] = {
{"help", 0, 0, 'h'},
{"version", 0, 0, 'V'},
{"subject", 0, 0, 's'},
{"subject-marker", 1, 0, 'S'},
{"header-marker", 1, 0, 'H'},
{"no-header", 0, 0, 'n'},
{"rate", 0, 0, 'r'},
{"rating", 0, 0, 'r'},
{"add-rating", 0, 0, 'r'},
{"asterisk", 0, 0, 'A'},
{"asterisks", 0, 0, 'A'},
{"stars", 0, 0, 'A'},
{"add-stars", 0, 0, 'A'},
{"add-asterisk", 0, 0, 'A'},
{"add-asterisks", 0, 0, 'A'},
{"test", 0, 0, 't'},
{"allowlist", 0, 0, 'a'},
{"allow-list", 0, 0, 'a'},
{"denylist", 0, 0, 'y'},
{"deny-list", 0, 0, 'y'},
{"plain", 1, 0, 'P'},
{"plainmap", 1, 0, 'P'},
{"plain-map", 1, 0, 'P'},
{"plaintext", 1, 0, 'P'},
{"plaintextmap", 1, 0, 'P'},
{"plaintext-map", 1, 0, 'P'},
{"level", 1, 0, 'L'},
{"threshold", 1, 0, 'L'},
{"min-tokens", 1, 0, 'Q'},
{"mintokens", 1, 0, 'Q'},
{"email", 1, 0, 'e'},
{"email-only", 1, 0, 'e'},
{"mark-spam", 0, 0, 'm'},
{"mark-nonspam", 0, 0, 'M'},
{"mark-non-spam", 0, 0, 'M'},
{"database", 1, 0, 'd'},
{"global", 1, 0, 'g'},
{"weight", 1, 0, 'w'},
{"train", 0, 0, 'T'},
{"noprune", 0, 0, 'N'},
{"no-prune", 0, 0, 'N'},
{"no-autoprune", 0, 0, 'N'},
{"noautoprune", 0, 0, 'N'},
{"prune-max", 1, 0, 'X'},
{"prunemax", 1, 0, 'X'},
{"prune", 0, 0, 'p'},
{"trim", 0, 0, 'p'},
{"dump", 0, 0, 'D'},
{"restore", 0, 0, 'R'},
{"tokens", 0, 0, 'O'},
{"benchmark", 0, 0, 'B'},
{"merge", 1, 0, 'E'},
{"verbose", 0, 0, 'v'},
{0, 0, 0, 0}
};
int option_index = 0;
#endif
char *short_options = "hVsS:H:nrAtoayL:Q:e:mMd:g:P:w:NTX:pDROBE:v";
int c;
opts_t opts;
opts = calloc(1, sizeof(*opts));
if (!opts) {
fprintf(stderr, /* RATS: ignore (OK) */
_("%s: option structure allocation failed (%s)"),
argv[0], strerror(errno));
fprintf(stderr, "\n");
return 0;
}
opts->program_name = argv[0];
opts->argc = 0;
opts->argv = calloc(argc + 1, sizeof(char *));
if (!opts->argv) {
fprintf(stderr, /* RATS: ignore (OK) */
_
("%s: option structure argv allocation failed (%s)"),
argv[0], strerror(errno));
fprintf(stderr, "\n");
opts_free(opts);
return 0;
}
opts->threshold = 0.9;
opts->min_token_count = 0;
opts->prune_max = 100000;
opts->db1weight = 1;
opts->db2weight = 1;
opts->db3weight = 1;
opts->action = ACTION_TEST;
opts->showprune = 0;
do {
#ifdef HAVE_GETOPT_LONG
c = getopt_long(argc, argv, /* RATS: ignore */
short_options, long_options,
&option_index);
#else
c = getopt(argc, argv, short_options); /* RATS: ignore */
#endif
if (c < 0)
continue;
switch (c) {
case 'h':
display_help();
opts->action = ACTION_NONE;
return opts;
break;
case 'V':
display_version();
opts->action = ACTION_NONE;
return opts;
break;
case 's':
opts->modify_subject = 1;
break;
case 'S':
opts->modify_subject = 1;
opts->subject_marker = optarg;
break;
case 'H':
opts->header_marker = optarg;
break;
case 'n':
opts->no_header = 1;
break;
case 'r':
opts->add_rating = 1;
break;
case 'A':
opts->add_stars = 1;
break;
case 't':
opts->no_filter = 1;
break;
case 'a':
opts->allowlist = 1;
break;
case 'y':
opts->denylist = 1;
break;
case 'L':
opts->threshold = (double) (atoi(optarg)) / 100;
break;
case 'Q':
opts->min_token_count = atoi(optarg);
break;
case 'e':
opts->emailonly = optarg;
opts->allowlist = 1;
break;
case 'm':
opts->modifydenylist = 0;
if (opts->action == ACTION_MARK_SPAM)
opts->modifydenylist = 1;
opts->action = ACTION_MARK_SPAM;
break;
case 'M':
opts->modifydenylist = 0;
if (opts->action == ACTION_MARK_NONSPAM)
opts->modifydenylist = 1;
opts->action = ACTION_MARK_NONSPAM;
break;
case 'd':
opts->database = optarg;
break;
case 'g':
if (opts->globaldb) {
opts->globaldb2 = optarg;
} else {
opts->globaldb = optarg;
}
break;
case 'P':
opts->plainmap = optarg;
break;
case 'w':
opts->weight = atoi(optarg);
break;
case 'T':
opts->action = ACTION_TRAIN;
opts->showprune = 1;
break;
case 'N':
opts->noautoprune = 1;
break;
case 'X':
opts->prune_max = atol(optarg);
break;
case 'p':
opts->action = ACTION_PRUNE;
opts->showprune = 1;
break;
case 'D':
opts->action = ACTION_DUMP;
break;
case 'R':
opts->action = ACTION_RESTORE;
break;
case 'O':
opts->action = ACTION_TOKENS;
break;
case 'B':
opts->action = ACTION_BENCHMARK;
opts->showprune = 1;
break;
case 'E':
opts->action = ACTION_MERGE;
opts->mergefrom = optarg;
break;
case 'v':
opts->loglevel++;
break;
default:
#ifdef HAVE_GETOPT_LONG
fprintf(stderr, /* RATS: ignore (OK) */
_("Try `%s --help' for more information."),
argv[0]);
#else
fprintf(stderr, /* RATS: ignore (OK) */
_("Try `%s -h' for more information."),
argv[0]);
#endif
fprintf(stderr, "\n");
opts_free(opts);
return 0;
break;
}
} while (c != -1);
log_level(opts->loglevel);
if (opts->weight < 1) {
opts->weight = 1;
} else if (opts->weight > 8) {
opts->weight = 8;
}
if (opts->threshold < 0.01) {
opts->threshold = 0.01;
} else if (opts->threshold > 1.00) {
opts->threshold = 1.00;
}
if (opts->prune_max < 10)
opts->prune_max = 10;
while (optind < argc) {
opts->argv[opts->argc++] = argv[optind++];
}
if ((opts->action == ACTION_TRAIN)
&& ((opts->argc < 2) || (opts->argc > 3))) {
fprintf(stderr, "%s: %s\n", argv[0],
_("train syntax: SPAM NONSPAM [MAXROUNDS]"));
opts_free(opts);
return 0;
} else if ((opts->action == ACTION_BENCHMARK)
&& ((opts->argc < 2) || (opts->argc > 3))) {
fprintf(stderr, "%s: %s\n", argv[0],
_("benchmark syntax: SPAM NONSPAM [MAXROUNDS]"));
opts_free(opts);
return 0;
} else if ((opts->action == ACTION_DUMP) && (opts->argc > 1)) {
fprintf(stderr, "%s: %s\n", argv[0],
_
("dump only requires one argument (file to dump to)"));
opts_free(opts);
return 0;
} else if ((opts->action == ACTION_RESTORE) && (opts->argc > 1)) {
fprintf(stderr, "%s: %s\n", argv[0],
_
("restore only requires one argument (file to restore from)"));
opts_free(opts);
return 0;
} else if ((opts->action != ACTION_DUMP)
&& (opts->action != ACTION_RESTORE)
&& (opts->action != ACTION_TRAIN)
&& (opts->action != ACTION_BENCHMARK)
&& (opts->argc > 0)
) {
fprintf(stderr, "%s: %s\n", argv[0],
_
("spurious extra arguments given on command line"));
opts_free(opts);
return 0;
}
return opts;
}
/* EOF */
qsf-1.2.7/src/main/version.c 0000644 0000764 0000764 00000001636 10664772303 013504 0 ustar aw aw /*
* Output version information to stdout.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include
/*
* Display current package version.
*/
void display_version(void)
{
printf( /* RATS: ignore */ _("%s %s - Copyright (C) %s %s"),
PROGRAM_NAME, VERSION, COPYRIGHT_YEAR, COPYRIGHT_HOLDER);
printf("\n%s%s", _("Backends available:"), BACKENDS);
printf("\n\n");
printf( /* RATS: ignore */ _("Web site: %s"), PROJECT_HOMEPAGE);
printf("\n\n");
printf("%s",
_("This program is free software, and is being distributed "
"under the\nterms of the Artistic License 2.0."));
printf("\n\n");
printf("%s", _
("This program is distributed in the hope that it will be useful,\n"
"but WITHOUT ANY WARRANTY; without even the implied warranty of\n"
"MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE."));
printf("\n\n");
}
/* EOF */
qsf-1.2.7/src/main/main.c 0000644 0000764 0000764 00000040765 10664772303 012751 0 ustar aw aw /*
* Main program entry point - read the command line options, then perform
* the appropriate actions.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "options.h"
#include "message.h"
#include "spam.h"
#include "database.h"
#include "log.h"
#include
#include
#include
#include
#include
#include
#include
#ifdef HAVE_FCNTL_H
#include
#endif
#ifdef HAVE_MCHECK_H
#include
#endif
/*
* Process command-line arguments and set option flags, then call functions
* to initialise, and finally enter the main loop.
*/
int main(int argc, char **argv)
{
char buffer[1024]; /* RATS: ignore (checked all) */
char filename[1024]; /* RATS: ignore (checked all) */
opts_t opts;
int retcode = 0;
int needwrite;
int got, sent, totsent;
double score;
qdb_t dbr1, dbr2, dbr3, dbw;
char *home;
msg_t msg = NULL;
int fd = -1;
#ifdef HAVE_MCHECK_H
if (getenv("MALLOC_TRACE")) /* RATS: ignore (value unused) */
mtrace();
#endif
#ifdef ENABLE_NLS
setlocale(LC_ALL, "");
bindtextdomain(PACKAGE, LOCALEDIR);
textdomain(PACKAGE);
#endif
opts = opts_parse(argc, argv);
if (!opts)
return 1;
if (opts->action == ACTION_NONE) {
opts_free(opts);
return 0;
}
atexit(log_free);
log_add(2, _("version %s initialising"), VERSION);
log_add(2, _("backends available:%s"), BACKENDS);
/*
* If we're testing, marking, or tokenising, or allow-list or
* deny-list updating/querying (having not been given an email
* address), we need to read a message from standard input before
* proceeding.
*/
if (((opts->action == ACTION_TEST)
|| (opts->action == ACTION_MARK_SPAM)
|| (opts->action == ACTION_MARK_NONSPAM)
) && ((opts->emailonly == NULL)
|| (strcmp(opts->emailonly, "MSG") == 0)
)
) {
msg = msg_parse(opts);
if (msg == NULL) {
log_add(2, "%s", _("failed to parse message"));
log_errdump(opts->program_name);
opts_free(opts);
return 1;
} else if (msg->content == NULL) {
if (opts->emailonly) {
fprintf(stderr, "%s: %s\n",
opts->program_name,
_
("message is too large or has no content"));
log_add(2, "%s",
_("failed to parse message"));
log_errdump(opts->program_name);
} else if ((opts->action != ACTION_MARK_SPAM)
&& (opts->action != ACTION_MARK_NONSPAM)
&& (!opts->no_filter)
) {
msg_dump(msg);
} else {
log_add(2, "%s",
_("failed to parse message"));
log_errdump(opts->program_name);
}
msg_free(msg);
while (!feof(stdin)) {
got =
fread(buffer, 1, sizeof(buffer),
stdin);
if ((got > 0)
&& ((unsigned int) got >
sizeof(buffer)))
got = sizeof(buffer);
if (opts->emailonly
|| opts->no_filter
|| (opts->action == ACTION_MARK_SPAM)
|| (opts->action ==
ACTION_MARK_NONSPAM)
)
continue;
totsent = 0;
if (got > 0) {
sent = fwrite(buffer + totsent, 1,
got, stdout);
if (sent <= 0)
break;
totsent += sent;
got -= sent;
} else {
break;
}
}
retcode = (opts->emailonly ? 1 : 0);
opts_free(opts);
return retcode;
}
}
/*
* If we're just tokenising, just do that, then exit. If the message
* cannot be parsed, do not dump it on stdout.
*/
if (opts->action == ACTION_TOKENS) {
msg = msg_parse(opts);
if (msg == NULL) {
opts_free(opts);
return 1;
}
retcode = spam_dumptokens(opts, msg);
msg_free(msg);
opts_free(opts);
return retcode;
}
/*
* From this point on, we're going to need databases; if none was
* specified on the command line, we open both /var/lib/qsfdb and
* ~/.qsfdb.
*/
dbr1 = NULL; /* first database to read from */
dbr2 = NULL; /* second database to read from */
dbr3 = NULL; /* third database to read from */
dbw = NULL; /* database to write to (=dbr1 or dbr2 or NULL) */
/*
* Work out whether we need write access.
*/
switch (opts->action) {
case ACTION_DUMP:
case ACTION_RESTORE:
case ACTION_TRAIN:
case ACTION_MARK_SPAM:
case ACTION_MARK_NONSPAM:
case ACTION_PRUNE:
case ACTION_MERGE:
needwrite = 1;
break;
default:
needwrite = 0;
break;
}
log_add(3, "%s: %s", _("need write access to a database"),
needwrite ? _("yes") : _("no"));
/*
* Now determine database locations, and open them.
*/
if ((opts->action == ACTION_BENCHMARK)
&& (opts->database)
&& (strncasecmp(opts->database, "mysql:", 6) == 0)
) {
/*
* Benchmarking mode, with a MySQL database. Open the
* database ready for use.
*/
filename[0] = 0; /* for remove() below */
dbr1 = qdb_open(opts->database, QDB_READWRITE);
dbw = dbr1;
if (dbr1 == NULL) {
log_add(1, "%s: %s: %s", opts->database,
_("failed to open database"), qdb_error());
fprintf(stderr, "%s: %s: %s: %s\n",
opts->program_name,
opts->database,
_("failed to open database"), qdb_error());
retcode = 1;
} else {
log_add(2, "%s: [%s] %s",
_("using database (rw)"),
qdb_type(dbr1), opts->database);
printf("%s: %s\n", _("Backend type"),
qdb_type(dbr1));
}
} else if (opts->action == ACTION_BENCHMARK) {
/*
* Benchmarking mode. Create a temporary file (which is
* deleted after opening) for the database.
*/
#ifdef P_tmpdir
#ifdef HAVE_SNPRINTF
snprintf(filename, sizeof(filename), "%.*s",
#else
sprintf(filename, "%.*s", /* RATS: ignore (OK) */
#endif
(int) (sizeof(filename) - 1),
P_tmpdir "/qsfXXXXXX");
#else
#ifdef HAVE_SNPRINTF
snprintf(filename, sizeof(filename), "%.*s",
#else
sprintf(filename, "%.*s", /* RATS: ignore (OK) */
#endif
(int) (sizeof(filename) - 1), "/tmp/qsfXXXXXX");
#endif
#ifdef HAVE_MKSTEMP
fd = mkstemp(filename);
#else
fd = -1;
if (tmpnam(filename) != NULL) { /* RATS: ignore (OK) */
fd = open(filename, /* RATS: ignore (OK) */
O_RDWR | O_CREAT | O_EXCL,
S_IRUSR | S_IWUSR);
}
#endif
if (fd < 0) {
fprintf(stderr, "%s: %s: %s: %s\n",
opts->program_name,
filename,
_("failed to create temporary file"),
strerror(errno));
retcode = 1;
} else {
char newfilename[4096]; /* RATS: ignore (checked) */
char newtype[64]; /* RATS: ignore (checked) */
newtype[0] = 0;
if (opts->database)
sscanf(opts->database, "%32[0-9A-Za-z]",
newtype);
if (newtype[0] != 0) {
#ifdef HAVE_SNPRINTF
snprintf(newfilename, sizeof(newfilename),
"%.*s:%.*s", 32,
#else
sprintf(newfilename, "%.*s:%.*s", 32, /* RATS: ignore (OK) */
#endif
newtype,
(int) (sizeof(filename) - 1 -
strlen(newtype)), filename);
}
chmod(filename, /* RATS: ignore (not important) */
S_IRUSR | S_IWUSR);
dbr1 =
qdb_open(newtype[0] ==
0 ? filename : newfilename,
QDB_READWRITE);
dbw = dbr1;
if (dbr1 == NULL) {
log_add(1, "%s: %s: %s", filename,
_("failed to open database"),
qdb_error());
fprintf(stderr, "%s: %s: %s: %s\n",
opts->program_name,
filename,
_("failed to open database"),
qdb_error());
retcode = 1;
} else {
log_add(2, "%s: [%s] %s",
_("using database (rw)"),
qdb_type(dbr1), filename);
}
close(fd);
remove(filename); /* RATS: ignore (no race) */
printf("%s: %s\n", _("Backend type"),
qdb_type(dbr1));
}
} else if (opts->database == NULL) {
/*
* Non-benchmarking mode, and no database specified. Work
* out the best databases to use, and open them.
*/
if (opts->globaldb) {
#ifdef HAVE_SNPRINTF
snprintf(buffer, sizeof(buffer), "%.*s",
#else
sprintf(buffer, "%.*s", /* RATS: ignore (OK) */
#endif
(int) (sizeof(buffer) - 1),
opts->globaldb);
} else {
#ifdef HAVE_SNPRINTF
snprintf(buffer, sizeof(buffer),
#else
sprintf(buffer, /* RATS: ignore (OK) */
#endif
"%.*s",
(int) (sizeof(buffer) - 1),
"/var/lib/" PACKAGE "db");
}
if (needwrite) {
if (opts->action == ACTION_RESTORE) {
dbr1 = qdb_open(buffer, QDB_NEW);
} else {
dbr1 = qdb_open(buffer, QDB_READWRITE);
}
}
if (dbr1 == NULL) {
dbr1 = qdb_open(buffer, QDB_READONLY);
if (dbr1)
log_add(2, "%s: [%s] %s",
_("using database (ro)"),
qdb_type(dbr1), buffer);
} else {
dbw = dbr1;
log_add(2, "%s: [%s] %s", _("using database (rw)"),
qdb_type(dbr1), buffer);
}
home = getenv("HOME"); /* RATS: ignore (sanitised) */
if (home == NULL)
home = "/";
if (strlen(home) /* RATS: ignore */ >(sizeof(buffer) - 64))
home = "/";
#ifdef HAVE_SNPRINTF
snprintf(buffer, sizeof(buffer), /* RATS: ignore (OK) */
"%s/.%sdb", home, PACKAGE);
#else
sprintf(buffer, /* RATS: ignore (checked above) */
"%s/.%.*sdb", home,
(int) (sizeof(buffer) - 8 -
strlen(home) /* RATS: ignore */ ),
PACKAGE);
#endif
if (needwrite) {
if (opts->action == ACTION_RESTORE) {
dbr2 = qdb_open(buffer, QDB_NEW);
} else {
dbr2 = qdb_open(buffer, QDB_READWRITE);
}
}
if (dbr2 == NULL) {
dbr2 = qdb_open(buffer, QDB_READONLY);
if (dbr2)
log_add(2, "%s: [%s] %s",
_("using database (ro)"),
qdb_type(dbr2), buffer);
} else if (dbw == NULL) {
dbw = dbr2;
log_add(2, "%s: [%s] %s", _("using database (rw)"),
qdb_type(dbr2), buffer);
}
/*
* Weight the per-user database at 10 times the
* weighting of the global database.
*/
if (dbr2)
opts->db2weight = 10;
if (opts->globaldb2) {
#ifdef HAVE_SNPRINTF
snprintf(buffer, sizeof(buffer),
#else
sprintf(buffer, /* RATS: ignore (OK) */
#endif
"%.*s", (int) (sizeof(buffer) - 1),
opts->globaldb2);
} else {
#ifdef HAVE_SNPRINTF
snprintf(buffer, sizeof(buffer), "%.*s",
#else
sprintf(buffer, "%.*s", /* RATS: ignore (OK) */
#endif
(int) (sizeof(buffer) - 1),
"/var/lib/" PACKAGE "db2");
}
if (dbr3 == NULL) {
dbr3 = qdb_open(buffer, QDB_READONLY);
if (dbr3)
log_add(2, "%s: [%s] %s",
_("using database (ro)"),
qdb_type(dbr3), buffer);
}
if (dbr3) {
/*
* Weight the first global database as twice
* as "heavy" as this one.
*/
opts->db1weight = 2;
opts->db3weight = 1;
}
} else {
/*
* Non-benchmarking mode, and a database has been specified.
* Open all available databases.
*/
dbr1 = qdb_open(opts->database, QDB_READWRITE);
if (dbr1 == NULL) {
dbr1 = qdb_open(opts->database, QDB_READONLY);
if (dbr1)
log_add(2, "%s: [%s] %s",
_("using database (ro)"),
qdb_type(dbr1), opts->database);
} else {
dbw = dbr1;
log_add(2, "%s: [%s] %s", _("using database (rw)"),
qdb_type(dbr1), opts->database);
}
if (dbr1 == NULL) {
log_add(1, "%s: %s: %s", opts->database,
_("failed to open database"), qdb_error());
fprintf(stderr, "%s: %s: %s: %s\n",
opts->program_name, opts->database,
_("failed to open database"), qdb_error());
retcode = 1;
} else {
/*
* Weight the per-user database at 10 times the
* weighting of the global database.
*/
opts->db1weight = 10;
}
if (opts->globaldb) {
dbr2 = qdb_open(opts->globaldb, QDB_READONLY);
if (dbr2 == NULL) {
log_add(1, "%s: %s: %s", opts->globaldb,
_("failed to open database"),
qdb_error());
fprintf(stderr, "%s: %s: %s: %s\n",
opts->program_name, opts->globaldb,
_("failed to open database"),
qdb_error());
retcode = 1;
} else {
log_add(2, "%s: [%s] %s",
_("using database (ro)"),
qdb_type(dbr2), opts->globaldb);
}
}
if (opts->globaldb2) {
dbr3 = qdb_open(opts->globaldb2, QDB_READONLY);
if (dbr3 == NULL) {
log_add(1, "%s: %s: %s",
opts->globaldb2,
_("failed to open database"),
qdb_error());
fprintf(stderr, "%s: %s: %s: %s\n",
opts->program_name,
opts->globaldb2,
_("failed to open database"),
qdb_error());
retcode = 1;
} else {
log_add(2, "%s: [%s] %s",
_("using database (ro)"),
qdb_type(dbr3), opts->globaldb2);
/*
* Weight the first global database as twice
* as "heavy" as this one.
*/
opts->db2weight = 2;
opts->db3weight = 1;
}
}
}
opts->dbr1 = dbr1;
opts->dbr2 = dbr2;
opts->dbr3 = dbr3;
opts->dbw = dbw;
if (opts->dbr1)
log_add(3, "%s %d: %d", _("weight of database"), 1,
opts->db1weight);
if (opts->dbr2)
log_add(3, "%s %d: %d", _("weight of database"), 2,
opts->db2weight);
if (opts->dbr3)
log_add(3, "%s %d: %d", _("weight of database"), 3,
opts->db3weight);
if (retcode) {
/*
* Error condition from earlier - clean up and exit.
*/
if ((opts->action != ACTION_MARK_SPAM)
&& (opts->action != ACTION_MARK_NONSPAM)
&& (!opts->no_filter)
) {
msg_dump(msg);
} else {
log_errdump(opts->program_name);
}
opts_free(opts);
qdb_close(dbr1);
qdb_close(dbr2);
qdb_close(dbr3);
return retcode;
}
if (opts->emailonly) {
/*
* Allow-list and deny-list manipulation / querying mode.
*/
if (strcmp(opts->emailonly, "MSG") == 0) {
if ((msg) && (msg->sender))
opts->emailonly = msg->sender;
if ((msg) && (msg->envsender))
opts->emailonly2 = msg->envsender;
}
if (opts->denylist) {
retcode = spam_denylist_manage(opts);
} else {
retcode = spam_allowlist_manage(opts);
}
log_errdump(opts->program_name);
opts_free(opts);
qdb_close(dbr1);
qdb_close(dbr2);
qdb_close(dbr3);
if (msg)
msg_free(msg);
return retcode;
}
switch (opts->action) {
case ACTION_DUMP:
/*
* Database dump mode.
*/
retcode = spam_db_dump(opts);
log_errdump(opts->program_name);
break;
case ACTION_RESTORE:
/*
* Database restore mode.
*/
retcode = spam_db_restore(opts);
log_errdump(opts->program_name);
break;
case ACTION_PRUNE:
/*
* Database prune mode.
*/
retcode = spam_db_prune(opts);
if (retcode == 0) {
printf("%s", _("Optimising database..."));
qdb_optimise(dbw);
printf(" %s\n", _("done"));
}
log_errdump(opts->program_name);
break;
case ACTION_TRAIN:
/*
* Training mode.
*/
retcode = spam_train(opts);
log_errdump(opts->program_name);
break;
case ACTION_BENCHMARK:
/*
* Benchmarking mode.
*/
retcode = spam_benchmark(opts);
qdb_close(dbr1);
remove(filename); /* RATS: ignore (no race) */
dbr1 = NULL;
/*
* We close the database here, instead of leaving it until
* later, so that we can remove the file as well.
*/
log_errdump(opts->program_name);
break;
case ACTION_MERGE:
/*
* Database merge mode.
*/
retcode = spam_db_merge(opts);
/*
* Note we are closing opts->dbr1 etc instead of our dbr1
* etc because spam_db_merge() shuffles them around.
*/
qdb_close(opts->dbr1);
qdb_close(opts->dbr2);
qdb_close(opts->dbr3);
dbr1 = NULL;
dbr2 = NULL;
dbr3 = NULL;
log_errdump(opts->program_name);
break;
case ACTION_MARK_SPAM:
retcode = spam_update(opts, msg, SPAM);
msg_free(msg);
log_errdump(opts->program_name);
break;
case ACTION_MARK_NONSPAM:
retcode = spam_update(opts, msg, NONSPAM);
msg_free(msg);
log_errdump(opts->program_name);
break;
case ACTION_TEST:
score = spam_check(opts, msg);
if (score < -9999.00) {
if (!opts->no_filter) {
msg_dump(msg);
} else {
log_errdump(opts->program_name);
}
if (msg)
msg_free(msg);
opts_free(opts);
qdb_close(dbr1);
qdb_close(dbr2);
qdb_close(dbr3);
return 0;
}
log_add(2, _("raw score: %g"), score);
log_add(3, _("images found: %d"), msg->num_images);
if (!opts->no_header)
msg_spamheader(msg, opts->header_marker, score);
if (opts->add_rating) {
msg_spamratingheader(msg, score, opts->threshold);
if (opts->no_filter) {
double spamscore, scaledscore;
spamscore = score;
if (spamscore < 0)
spamscore += 0.01;
spamscore += opts->threshold;
scaledscore = spamscore * 100.0;
printf("%d\n", (int) scaledscore);
}
}
if (opts->add_stars)
msg_spamlevelheader(msg, score, opts->threshold);
if (score > 0) {
if (opts->modify_subject)
msg_spamsubject(msg, opts->subject_marker);
if (!opts->no_filter) {
msg_dump(msg);
retcode = 0;
} else {
log_errdump(opts->program_name);
retcode = 1;
}
} else {
if (!opts->no_filter) {
msg_dump(msg);
} else {
log_errdump(opts->program_name);
}
retcode = 0;
}
msg_free(msg);
break;
default:
/*
* This code should never be reached.
*/
log_errdump(opts->program_name);
fprintf(stderr, "%s: %s\n", opts->program_name,
_("unknown action, please report this as a bug!"));
retcode = 1;
break;
}
opts_free(opts);
qdb_close(dbr1);
qdb_close(dbr2);
qdb_close(dbr3);
return retcode;
}
/* EOF */
qsf-1.2.7/src/main/log.c 0000644 0000764 0000764 00000004353 10664772304 012600 0 ustar aw aw /*
* Logging functions.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "log.h"
#include
#include
#include
#include
static int _log_level = 0;
static int _log_lines = 0;
static char **_log_array = 0;
/*
* Set the logging level (0 is off).
*/
void log_level(int level)
{
_log_level = level;
}
/*
* Add a log message to the queue, if logging is enabled with a level
* greater than or equal to "level" (i.e. level 1 is highest priority, then
* level 2, and so on).
*/
void log_add(int level, char *format, ...)
{
char buf[8192]; /* RATS: ignore (checked OK - ish) */
va_list ap;
char **ptr;
int sz;
if (_log_level < level)
return;
va_start(ap, format);
buf[0] = 0;
#ifdef HAVE_VSNPRINTF
vsnprintf(buf, sizeof(buf), format, ap);
#else
vsprintf(buf, format, ap); /* RATS: ignore (unavoidable) */
#endif
va_end(ap);
if (_log_array == 0) {
ptr = (char **) malloc(sizeof(char *) * (_log_lines + 1));
} else {
ptr = (char **) realloc(_log_array, /* RATS: ignore */
sizeof(char *) * (_log_lines + 1));
}
if (ptr == 0)
return;
sz = strlen(buf) + 1;
_log_array = ptr;
_log_array[_log_lines] = malloc(sz);
if (_log_array[_log_lines] == 0)
return;
memcpy(_log_array[_log_lines], buf, sz); /* RATS: ignore (checked) */
_log_lines++;
}
/*
* Dump out all log messages, with the given prefix before each line.
*/
void log_dump(char *prefix)
{
int i;
if (_log_array == 0)
return;
for (i = 0; i < _log_lines; i++) {
if (_log_array[i] == 0)
continue;
printf("%s%s\n", prefix, _log_array[i]);
}
}
/*
* Dump out all log messages, with the given prefix plus ": " before each
* line, to stderr.
*/
void log_errdump(char *prefix)
{
int i;
if (_log_array == 0)
return;
for (i = 0; i < _log_lines; i++) {
if (_log_array[i] == 0)
continue;
fprintf(stderr, "%s: %s\n", prefix, _log_array[i]);
}
}
/*
* Free all memory used by this logging system.
*/
void log_free(void)
{
int i;
if (_log_array == 0)
return;
for (i = 0; i < _log_lines; i++) {
if (_log_array[i] == 0)
continue;
free(_log_array[i]);
_log_array[i] = 0;
}
free(_log_array);
_log_array = 0;
_log_lines = 0;
}
/* EOF */
qsf-1.2.7/src/main/tick.c 0000644 0000764 0000764 00000000745 10664772304 012752 0 ustar aw aw /*
* A ticker to let the user know we've not crashed.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include
#include
/*
* Output a ticker.
*/
void tick(void)
{
static char *ticker = "-\\|/";
static int tickpos = 0;
static time_t last_tick = 0;
if (time(NULL) <= last_tick)
return;
last_tick = time(NULL);
printf("%c%c", ticker[tickpos++], 8);
if (ticker[tickpos] == 0)
tickpos = 0;
}
/* EOF */
qsf-1.2.7/src/spam/ 0000755 0000764 0000764 00000000000 10665021416 011652 5 ustar aw aw qsf-1.2.7/src/spam/update.c 0000644 0000764 0000764 00000004354 10664772304 013316 0 ustar aw aw /*
* Functions for recognising mail as spam and updating the database.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "spami.h"
#include
/*
* Update the database, marking the contents of the given message as either
* spam or non-spam depending on whether "type" is SPAM or NONSPAM. Returns
* nonzero on error.
*/
int spam_update(opts_t opts, msg_t msg, int type)
{
spam_t spam;
token_t token;
long i;
if (opts->dbw == NULL) {
fprintf(stderr, "%s: %s\n", opts->program_name,
_("no database to write to"));
return 1;
}
if (opts->denylist && opts->modifydenylist) {
if (type == SPAM) {
spam_denylist_add(opts, msg->sender);
spam_denylist_add(opts, msg->envsender);
} else {
spam_denylist_remove(opts, msg->sender);
spam_denylist_remove(opts, msg->envsender);
}
} else if (opts->allowlist) {
if (type == SPAM) {
spam_allowlist_remove(opts, msg->sender);
spam_allowlist_remove(opts, msg->envsender);
} else {
spam_allowlist_add(opts, msg->sender);
spam_allowlist_add(opts, msg->envsender);
}
}
spam = spam_tokenise(opts, msg, opts->dbw, NULL, NULL, 1, 1, 1);
if (type == SPAM) {
spam->total_spam += opts->weight;
} else {
spam->total_nonspam += opts->weight;
}
spam->update_count++;
spam_store(opts, " COUNTS", 7,
spam->total_spam, spam->total_nonspam,
spam->update_count);
spam->since_prune++;
spam_store(opts, " SINCEPRUNE", 11, spam->since_prune, 0, 0);
for (i = 0; i < spam->token_count; i++) {
token = spam->tarray[i];
if (type == SPAM) {
token->num_spam += token->count * opts->weight;
} else {
token->num_nonspam += token->count * opts->weight;
}
token->last_updated = spam->update_count;
spam_store(opts, token->token, token->length,
token->num_spam, token->num_nonspam,
token->last_updated);
}
if (spam->update_count > 1) {
int oldshowprune;
oldshowprune = opts->showprune;
opts->showprune = 0;
spam_db_prune(opts);
opts->showprune = oldshowprune;
} else if ((opts->action != ACTION_TRAIN)
&& (opts->action != ACTION_BENCHMARK)
&& (!opts->noautoprune)
&& (spam->since_prune > 500)) {
spam_db_prune(opts);
}
spam_free(spam);
return 0;
}
/* EOF */
qsf-1.2.7/src/spam/merge.c 0000644 0000764 0000764 00000006127 10664772304 013133 0 ustar aw aw /*
* Functions for merging one spam database into another.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "spami.h"
#include "database.h"
#include
#include
#include
#include
/*
* Merge the contents of another spam database into the currently writable
* one. Returns nonzero on error.
*/
int spam_db_merge(opts_t opts)
{
void *dbfrom;
void *dbto;
spam_t spam;
qdb_datum key, nextkey;
char **keylist = NULL;
char **newkeylist;
long keylistsize = 0;
long numkeys = 0;
long i, a, b, c;
dbto = opts->dbw;
if (opts->dbr1 == opts->dbw) {
if (opts->dbr2)
qdb_close(opts->dbr2);
if (opts->dbr3)
qdb_close(opts->dbr3);
opts->dbr2 = qdb_open(opts->mergefrom, QDB_READONLY);
if (opts->dbr2 == NULL) {
fprintf(stderr, "%s: %s: %s: %s\n",
opts->program_name,
opts->mergefrom,
_("failed to open database"), qdb_error());
return 1;
}
dbfrom = opts->dbr2;
} else {
if (opts->dbr1)
qdb_close(opts->dbr1);
opts->dbr1 = qdb_open(opts->mergefrom, QDB_READONLY);
if (opts->dbr1 == NULL) {
fprintf(stderr, "%s: %s: %s: %s\n",
opts->program_name,
opts->mergefrom,
_("failed to open database"), qdb_error());
return 1;
}
dbfrom = opts->dbr1;
}
spam = calloc(1, sizeof(*spam));
if (spam == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
return 1;
}
spam->db1 = dbfrom;
spam->db2 = dbto;
spam->db3 = NULL;
spam->db1weight = 1;
spam->db2weight = 1;
spam->db3weight = 1;
/*
* Get list of all keys in the database we're merging from.
*/
key = qdb_firstkey(dbfrom);
while (key.data != NULL) {
if (keylistsize >= numkeys) {
keylistsize += 10000;
if (numkeys > 0) {
newkeylist = realloc(keylist, /* RATS: ignore */
sizeof(char *) *
keylistsize);
} else {
newkeylist =
malloc(sizeof(char *) * keylistsize);
}
if (newkeylist == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("memory allocation failed"),
strerror(errno));
free(keylist);
free(key.data);
free(spam);
return 1;
}
keylist = newkeylist;
}
keylist[numkeys] = calloc(1, key.size + 1);
if (keylist[numkeys] == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("memory allocation failed"),
strerror(errno));
free(keylist);
free(key.data);
free(spam);
return 1;
}
strncpy(keylist[numkeys], (char *) (key.data), key.size);
numkeys++;
nextkey = qdb_nextkey(dbfrom, key);
free(key.data);
key = nextkey;
}
/*
* For each key in the merge-from database, read the combined value
* for that key in both the merge-from and, if present, the merge-to
* database, then store this combined value into the merge-to
* database.
*/
for (i = 0; i < numkeys; i++) {
a = 0;
b = 0;
spam_fetch(spam, keylist[i], strlen(keylist[i]), &a, &b,
&c);
spam_store(opts, keylist[i], strlen(keylist[i]), a, b, c);
free(keylist[i]);
}
free(keylist);
free(spam);
return 0;
}
/* EOF */
qsf-1.2.7/src/spam/alloc.c 0000644 0000764 0000764 00000000734 10664772304 013124 0 ustar aw aw /*
* Memory allocation/deallocation functions.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "spami.h"
#include
/*
* Free the given spam structure.
*/
void spam_free(spam_t spam)
{
long i;
if (spam == NULL)
return;
if (spam->tarray != NULL) {
for (i = 0; i < spam->token_count; i++) {
free(spam->tarray[i]);
}
free(spam->tarray);
spam->tarray = NULL;
}
free(spam);
}
/* EOF */
qsf-1.2.7/src/spam/train.c 0000644 0000764 0000764 00000013536 10664772304 013153 0 ustar aw aw /*
* Functions for training the database to recognise spam.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "mailbox.h"
#include "spami.h"
#include
#include
#include
#include
#include
void tick(void);
/*
* Train the database by repeatedly classifying the contents of two mail
* folders as spam or non-spam, then updating the database for those
* messages that get incorrectly classified.
*
* If opts->benchmark is set, only the first 75% of messages in each mailbox
* will be considered, and the database will not be optimised after
* training.
*
* Returns nonzero on error.
*/
int spam_train(opts_t opts)
{
FILE *fptr_spam;
FILE *fptr_nonspam;
mbox_t mbox_spam, mbox_nonspam;
long numspam, numnonspam, msgnum;
int round, nobetterspam, nobetternonspam;
long wrongspam, wrongnonspam, prevwrongspam, prevwrongnonspam;
double pctwrongspam, pctwrongnonspam;
int maxrounds;
time_t last_unlock;
msg_t msg;
setvbuf(stdin, NULL, _IONBF, 0);
setvbuf(stdout, NULL, _IONBF, 0);
fptr_spam = fopen(opts->argv[0], "r");
if (fptr_spam == NULL) {
fprintf(stderr, "%s: %s: %s: %s\n", opts->program_name,
opts->argv[0], _("failed to open spam mailbox"),
strerror(errno));
return 1;
}
fptr_nonspam = fopen(opts->argv[1], "r");
if (fptr_nonspam == NULL) {
fprintf(stderr, "%s: %s: %s: %s\n", opts->program_name,
opts->argv[1],
_("failed to open non-spam mailbox"),
strerror(errno));
fclose(fptr_spam);
return 1;
}
maxrounds = 200;
if ((opts->argc > 2) && (opts->argv[2])) {
maxrounds = atoi(opts->argv[2]);
if (maxrounds < 1)
maxrounds = 1;
if (maxrounds > 1000)
maxrounds = 1000;
}
numspam = -1;
numnonspam = -1;
/*
* Release the database lock while we're counting messages.
*/
spam_dbunlock(opts);
printf("%s ", _("Counting messages in folders..."));
mbox_spam = mbox_scan(opts, fptr_spam);
if (mbox_spam == NULL) {
fclose(fptr_nonspam);
fclose(fptr_spam);
return 1;
}
numspam = mbox_count(mbox_spam);
printf("%ld ", numspam);
mbox_nonspam = mbox_scan(opts, fptr_nonspam);
if (mbox_nonspam == NULL) {
fclose(fptr_nonspam);
fclose(fptr_spam);
mbox_free(mbox_spam);
return 1;
}
numnonspam = mbox_count(mbox_nonspam);
printf("%ld\n", numnonspam);
if (opts->action == ACTION_BENCHMARK) {
numspam = (numspam * 3) / 4;
numnonspam = (numnonspam * 3) / 4;
}
nobetterspam = 0;
nobetternonspam = 0;
prevwrongspam = 0;
prevwrongnonspam = 0;
opts->weight = 1;
/*
* Re-lock the database.
*/
spam_dbrelock(opts);
last_unlock = time(NULL);
for (round = 1; round <= maxrounds; round++) {
/*
* Prune the database after every 15 rounds.
*/
if (round > 1 && round % 15 == 1)
spam_db_prune(opts);
printf("%s %d: %s", _("round"), round,
_("checking spam..."));
for (msgnum = 0, wrongspam = 0; msgnum < numspam; msgnum++) {
tick();
/*
* Briefly unlock the database every couple of
* seconds, so that any other processes waiting to
* read from the database can do so.
*/
if (time(NULL) - last_unlock > 1) {
spam_dbunlock(opts);
spam_dbrelock(opts);
last_unlock = time(NULL);
}
mbox_select(opts, mbox_spam, fptr_spam, msgnum);
msg = msg_parse(opts);
if (msg == NULL) {
mbox_select(opts, NULL, NULL, 0);
mbox_free(mbox_spam);
mbox_free(mbox_nonspam);
fclose(fptr_spam);
fclose(fptr_nonspam);
return 1;
}
if (round == 1 && opts->allowlist) {
spam_allowlist_remove(opts, msg->sender);
spam_allowlist_remove(opts,
msg->envsender);
}
if (spam_check(opts, msg) < 0.0) {
wrongspam++;
spam_update(opts, msg, SPAM);
}
msg_free(msg);
}
pctwrongspam = 100.0 * ((double) wrongspam)
/ (numspam > 0 ? (double) numspam : 1.0);
printf( /* RATS: ignore */
_(" reclassified [%3.2f%%] %ld/%ld\n"),
pctwrongspam, wrongspam, numspam);
if (wrongspam >= prevwrongspam) {
nobetterspam++;
} else {
nobetterspam = 0;
}
prevwrongspam = wrongspam;
printf("%s %d: %s", _("round"), round,
_("checking non-spam..."));
for (msgnum = 0, wrongnonspam = 0; msgnum < numnonspam;
msgnum++) {
tick();
if (time(NULL) - last_unlock > 1) {
spam_dbunlock(opts);
spam_dbrelock(opts);
last_unlock = time(NULL);
}
mbox_select(opts, mbox_nonspam, fptr_nonspam,
msgnum);
msg = msg_parse(opts);
if (msg == NULL) {
mbox_select(opts, NULL, NULL, 0);
mbox_free(mbox_spam);
mbox_free(mbox_nonspam);
fclose(fptr_spam);
fclose(fptr_nonspam);
return 1;
}
if (round == 1 && opts->allowlist) {
spam_allowlist_add(opts, msg->sender);
spam_allowlist_add(opts, msg->envsender);
}
if (spam_check(opts, msg) > -0.01) {
wrongnonspam++;
spam_update(opts, msg, NONSPAM);
}
msg_free(msg);
}
pctwrongnonspam = 100.0 * ((double) wrongnonspam)
/ (numnonspam > 0 ? (double) numnonspam : 1.0);
printf( /* RATS: ignore */
_(" reclassified [%3.2f%%] %ld/%ld\n"),
pctwrongnonspam, wrongnonspam, numnonspam);
if (wrongnonspam >= prevwrongnonspam) {
nobetternonspam++;
} else {
nobetternonspam = 0;
}
prevwrongnonspam = wrongnonspam;
if (wrongnonspam == 0 && pctwrongspam < 0.5 && round > 5) {
printf("%s\n",
_("Good results, ending training."));
round = 999999;
}
if (nobetterspam > 10 && nobetternonspam > 10) {
printf("%s\n",
_("Several rounds with no improvement, "
"ending training."));
round = 999999;
}
}
mbox_select(opts, NULL, NULL, 0);
mbox_free(mbox_spam);
mbox_free(mbox_nonspam);
fclose(fptr_spam);
fclose(fptr_nonspam);
if (opts->action == ACTION_BENCHMARK)
return 0;
printf("%s", _("Optimising database..."));
qdb_optimise(opts->dbw);
printf(" %s\n", _("done"));
return 0;
}
/* EOF */
qsf-1.2.7/src/spam/cksum.c 0000644 0000764 0000764 00000001765 10664772304 013161 0 ustar aw aw /*
* Token checksumming functions.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "md5.h"
#include
#include
#include
/*
* Generate a checksum string of the given token and return a malloc()ed
* pointer to it.
*/
unsigned char *spam_checksum(char *key, int len)
{
struct MD5Context md5c;
unsigned char digest[64]; /* RATS: ignore (size OK) */
char buf[16]; /* RATS: ignore (size OK) */
char resultstr[128]; /* RATS: ignore (size OK) */
unsigned char *ptr;
int i;
MD5Init(&md5c);
MD5Update(&md5c, (unsigned char *) key, len);
MD5Final(digest, &md5c);
memcpy(resultstr, "!\000", 2);
for (i = 0; i < 16; i++) {
#ifdef HAVE_SNPRINTF
snprintf(buf, sizeof(buf), "%02x", digest[i]);
#else
sprintf(buf, "%02x", digest[i]);
#endif
buf[2] = 0;
memcpy(resultstr + strlen(resultstr), buf, 3);
}
ptr = (unsigned char *) (strdup(resultstr));
if (ptr == NULL)
abort();
return ptr;
}
/* EOF */
qsf-1.2.7/src/spam/benchmark.c 0000644 0000764 0000764 00000016451 10664772304 013767 0 ustar aw aw /*
* Functions for benchmarking the training process.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "mailbox.h"
#include "spami.h"
#include
#include
#include
#include
#include
#ifdef HAVE_SYS_RESOURCE_H
#include
#endif
#include
void tick(void);
#ifdef HAVE_SYS_RESOURCE_H
/*
* Fill in "t" with the difference in time between "t1" and "t2".
*/
static void diff_timeval(struct timeval *t, struct timeval *t1,
struct timeval *t2)
{
t->tv_sec = t2->tv_sec - t1->tv_sec;
t->tv_usec = t2->tv_usec - t1->tv_usec;
if (t->tv_usec < 0) {
t->tv_sec--;
t->tv_usec += 1000000;
}
}
/*
* Fill in "t" with the sum of "t1" and "t2".
*/
static void add_timeval(struct timeval *t, struct timeval *t1,
struct timeval *t2)
{
t->tv_sec = t1->tv_sec + t2->tv_sec;
t->tv_usec = t1->tv_usec + t2->tv_usec;
if (t->tv_usec >= 1000000) {
t->tv_sec++;
t->tv_usec -= 1000000;
}
}
/*
* Fill in "usage" with the difference between "usage1" and "usage2", to get
* the amount of user and system time elapsed between the two,
*/
static void diff_usage(struct rusage *usage, struct rusage *usage1,
struct rusage *usage2)
{
diff_timeval(&(usage->ru_utime), &(usage1->ru_utime),
&(usage2->ru_utime));
diff_timeval(&(usage->ru_stime), &(usage1->ru_stime),
&(usage2->ru_stime));
}
#endif /* HAVE_SYS_RESOURCE_H */
/*
* Benchmark the training process by training on the first 75% of messages
* in each folder, and then looking at the classification of the remaining
* 25% (looking for false positive/false negative results).
*
* Also outputs some information about how long (in CPU time) the training
* process took.
*
* Returns nonzero on error.
*/
int spam_benchmark(opts_t opts)
{
#ifdef HAVE_SYS_RESOURCE_H
struct rusage usage1, usage2, usage;
struct timeval total;
#endif
FILE *fptr_spam;
FILE *fptr_nonspam;
mbox_t mbox_spam, mbox_nonspam;
long numspam, numnonspam, msgnum;
long false_positive, false_negative;
long spam_checked, nonspam_checked;
double score;
msg_t msg;
#ifdef HAVE_SYS_RESOURCE_H
if (getrusage(RUSAGE_SELF, &usage1)) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("failed to read resource usage"),
strerror(errno));
return 1;
}
#endif
fflush(stdout);
if (spam_train(opts))
return 1;
#ifdef HAVE_SYS_RESOURCE_H
if (getrusage(RUSAGE_SELF, &usage2)) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("failed to read resource usage"),
strerror(errno));
return 1;
}
diff_usage(&usage, &usage1, &usage2);
add_timeval(&total, &(usage.ru_utime), &(usage.ru_stime));
#endif
printf("%s", _("Optimising database..."));
qdb_optimise(opts->dbw);
printf(" %s\n", _("done"));
printf("\n");
#ifdef HAVE_SYS_RESOURCE_H
printf("%s\n %s %5ld.%06ld\n %s %5ld.%06ld\n %s %5ld.%06ld\n",
_("Resource usage during training and optimising:"),
_(" User time (sec):"),
(long) (usage.ru_utime.tv_sec),
(long) (usage.ru_utime.tv_usec),
_("System time (sec):"),
(long) (usage.ru_stime.tv_sec),
(long) (usage.ru_stime.tv_usec),
_(" Total time (sec):"),
(long) (total.tv_sec), (long) (total.tv_usec));
printf("\n");
#endif
fptr_spam = fopen(opts->argv[0], "r");
if (fptr_spam == NULL) {
fprintf(stderr, "%s: %s: %s: %s\n", opts->program_name,
opts->argv[0], _("failed to open spam mailbox"),
strerror(errno));
return 1;
}
fptr_nonspam = fopen(opts->argv[1], "r");
if (fptr_nonspam == NULL) {
fprintf(stderr, "%s: %s: %s: %s\n", opts->program_name,
opts->argv[1],
_("failed to open non-spam mailbox"),
strerror(errno));
fclose(fptr_spam);
return 1;
}
numspam = -1;
numnonspam = -1;
printf("%s ", _("Counting messages in folders..."));
mbox_spam = mbox_scan(opts, fptr_spam);
if (mbox_spam == NULL) {
fclose(fptr_nonspam);
fclose(fptr_spam);
return 1;
}
numspam = mbox_count(mbox_spam);
printf("%ld ", numspam);
mbox_nonspam = mbox_scan(opts, fptr_nonspam);
if (mbox_nonspam == NULL) {
fclose(fptr_nonspam);
fclose(fptr_spam);
mbox_free(mbox_spam);
return 1;
}
numnonspam = mbox_count(mbox_nonspam);
printf("%ld\n", numnonspam);
#ifdef HAVE_SYS_RESOURCE_H
if (getrusage(RUSAGE_SELF, &usage1)) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("failed to read resource usage"),
strerror(errno));
return 1;
}
#endif
printf("Counting incorrect classifications...");
false_positive = 0;
false_negative = 0;
spam_checked = 0;
nonspam_checked = 0;
for (msgnum = 0; msgnum < numspam; msgnum++) {
tick();
mbox_select(opts, mbox_spam, fptr_spam, msgnum);
msg = msg_parse(opts);
if (msg == NULL) {
mbox_select(opts, NULL, NULL, 0);
mbox_free(mbox_spam);
mbox_free(mbox_nonspam);
fclose(fptr_spam);
fclose(fptr_nonspam);
return 1;
}
score = spam_check(opts, msg);
if (score <= 0) {
false_negative++;
}
msg_free(msg);
spam_checked++;
}
for (msgnum = 0; msgnum < numnonspam; msgnum++) {
tick();
mbox_select(opts, mbox_nonspam, fptr_nonspam, msgnum);
msg = msg_parse(opts);
if (msg == NULL) {
mbox_select(opts, NULL, NULL, 0);
mbox_free(mbox_spam);
mbox_free(mbox_nonspam);
fclose(fptr_spam);
fclose(fptr_nonspam);
return 1;
}
score = spam_check(opts, msg);
if (score > 0) {
false_positive++;
}
msg_free(msg);
nonspam_checked++;
}
#ifdef HAVE_SYS_RESOURCE_H
if (getrusage(RUSAGE_SELF, &usage2)) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("failed to read resource usage"),
strerror(errno));
return 1;
}
diff_usage(&usage, &usage1, &usage2);
add_timeval(&total, &(usage.ru_utime), &(usage.ru_stime));
#endif
/*
* Prevent division by zero
*/
if (spam_checked < 1)
spam_checked = 1;
if (nonspam_checked < 1)
nonspam_checked = 1;
printf(" \n\n");
#ifdef HAVE_SYS_RESOURCE_H
printf
("%s\n %s %5ld.%06ld\n %s %5ld.%06ld\n %s %5ld.%06ld\n %s %ld\n",
_("Resource usage during classification:"),
_(" User time (sec):"), (long) (usage.ru_utime.tv_sec),
(long) (usage.ru_utime.tv_usec), _(" System time (sec):"),
(long) (usage.ru_stime.tv_sec),
(long) (usage.ru_stime.tv_usec), _(" Total time (sec):"),
(long) (total.tv_sec), (long) (total.tv_usec),
_("Messages classified:"), spam_checked + nonspam_checked);
printf("\n");
#endif
printf("%s %5.2f%% [%ld/%ld] \t%s\n%s %5.2f%% [%ld/%ld] \t%s\n",
_("False negatives:"),
(100.0 * (double) false_negative) / (double) spam_checked,
false_negative, spam_checked,
_("(failing to mark spam as being spam)"),
_("False positives:"),
(100.0 * (double) false_positive) /
(double) nonspam_checked, false_positive, nonspam_checked,
_("(wrongly marking a real email as spam)"));
printf("\n%s %5.4f%% [%ld/%ld]\n", _("Accuracy:"),
(100.0 *
(double) (spam_checked + nonspam_checked - false_negative -
false_positive) / (double) (spam_checked +
nonspam_checked)),
(long) (spam_checked + nonspam_checked - false_negative -
false_positive),
(long) (spam_checked + nonspam_checked)
);
mbox_select(opts, NULL, NULL, 0);
mbox_free(mbox_spam);
mbox_free(mbox_nonspam);
fclose(fptr_spam);
fclose(fptr_nonspam);
return 0;
}
/* EOF */
qsf-1.2.7/src/spam/check.c 0000644 0000764 0000764 00000013034 10664772304 013104 0 ustar aw aw /*
* Check whether a message is spam.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "spami.h"
#include "log.h"
#include
#include
#include
#include
/*
* Return a probability that the message is spam, where 0.9 and above means
* "definitely spam", using the Robinson method.
*
* Robinson's method:
*
* Set pN = "spam probability" of token N, where pN = f(p(w)), where p(w)
* is (bad / tb) / ((bad / tb) + (good / tg)) - bad is number of times
* token seen in bad messages, good is times token seen in good messages,
* tb and tg are total number of bad and good messages seen; f(w) is (robs
* * robx + gbtot * p(w)) / (robs + gbtot), where gbtot is is good + bad,
* robs is a constant, and robx is a fudge factor calculated from the
* average p(w) of all tokens that have been seen over 10 times.
*
* Then:
*
* P = 1 - ((1-p1)(1-p2)(1-p3)...(1-pN))^(1/n)
* Q = 1 - (p1p2p3...pN)^(1/n)
* S = (P - Q) / (P + Q)
*
* S is then a number from -1 to +1, so we scale it to 0-1 and then divide
* by (0.54/0.9=0.6) and clip to 1, since the spam cutoff point for this
* algorithm is 0.54 and we want it to be 0.9.
*/
double spam_check__robinson(opts_t opts, msg_t msg, spam_t spam)
{
token_t token;
long i, n;
double r, ln2, p_log, q_log, p, q, s, robs, robx;
struct {
double mant;
int exp;
} P, Q;
int e;
P.mant = 1.0;
P.exp = 0;
Q.mant = 1.0;
Q.exp = 0;
robx = spam->robx;
robs = 1.0;
n = 0;
if (spam->token_count < 1)
return 0.5;
for (i = 0; i < spam->token_count; i++) {
double pw, fw, gbtot;
token = spam->tarray[i];
pw = token->prob_spam;
gbtot = token->num_nonspam + token->num_spam;
fw = (robs * robx + gbtot * pw) / (robs + gbtot);
if (fabs(0.5 - pw) < 0.00001)
continue;
P.mant *= 1 - fw;
if (P.mant < 1.0e-200) {
P.mant = frexp(P.mant, &e);
P.exp += e;
}
Q.mant *= fw;
if (Q.mant < 1.0e-200) {
Q.mant = frexp(Q.mant, &e);
Q.exp += e;
}
n++;
}
if (n < 1)
n = 1;
r = 1.0 / (double) n;
ln2 = 0.69314718055994530941;
/*
* Avoid floating point exceptions.
*/
if (P.mant <= 0)
P.mant = 0.00000000001;
if (Q.mant <= 0)
Q.mant = 0.00000000001;
p_log = log(P.mant) + P.exp * ln2;
q_log = log(Q.mant) + Q.exp * ln2;
p = 1.0 - exp(p_log * r);
q = 1.0 - exp(q_log * r);
s = (1.0 + (p - q) / (p + q)) / 2.0;
s = s / 0.6;
if (s > 1.0)
s = 1.0;
return s;
}
/*
* Return a score above zero if the given message is probably spam, zero or
* less if not. If the score is < -9999, then the message should not be
* processed at all (due to the "min-tokens" option).
*/
double spam_check(opts_t opts, msg_t msg)
{
double prob;
spam_t spam;
spam =
spam_tokenise(opts, msg, opts->dbr1, opts->dbr2, opts->dbr3,
opts->db1weight, opts->db2weight,
opts->db3weight);
/*
* If we're not in training or benchmarking mode and we know the
* email address of the message sender or the envelope address,
* check either against the deny-list and if a match is found,
* return a "definitely spam" score. Then check against the
* allow-list, and if a match is found, return a "definitely not
* spam" score. Then check both again for just the "@domain" part.
*/
if ((opts->action != ACTION_TRAIN)
&& (opts->action != ACTION_BENCHMARK)
) {
char *senderdomain;
char *envsenderdomain;
/*
* First check the sender and envelope sender addresses
* against the deny and allow lists.
*/
if ((opts->denylist)
&& ((spam_denylist_match(spam, msg->sender))
|| (spam_denylist_match(spam, msg->envsender))
)
) {
spam_free(spam);
return 0.1;
} else if ((opts->allowlist)
&& ((spam_allowlist_match(spam, msg->sender))
||
(spam_allowlist_match(spam, msg->envsender))
)
) {
spam_free(spam);
return 0.00 - (opts->threshold + 0.01);
}
/*
* Now check just the @domain part of each address.
*/
senderdomain = NULL;
if (msg->sender)
senderdomain = strchr(msg->sender, '@');
envsenderdomain = NULL;
if (msg->envsender)
envsenderdomain = strchr(msg->envsender, '@');
if ((opts->denylist)
&& ((spam_denylist_match(spam, senderdomain))
|| (spam_denylist_match(spam, envsenderdomain))
)
) {
spam_free(spam);
return 0.1;
} else if ((opts->allowlist)
&& ((spam_allowlist_match(spam, senderdomain))
||
(spam_allowlist_match
(spam, envsenderdomain))
)
) {
spam_free(spam);
return 0.00 - (opts->threshold + 0.01);
}
}
/*
* If we've been told to do nothing if there are fewer than a
* particular number of tokens, then return -10000 if there aren't
* enough tokens.
*/
if ((opts->min_token_count > spam->token_count)
&& (opts->action != ACTION_TRAIN)
&& (opts->action != ACTION_BENCHMARK)
) {
spam_free(spam);
return -10000.00;
}
if ((opts->action != ACTION_TRAIN)
&& (opts->action != ACTION_BENCHMARK))
log_add(2, _("token count: %d"), spam->token_count);
/*
* If a spam test caused an override, return the appropriate score
* (definitely spam or non-spam) depending on whether the override
* flag is positive or negative respectively.
*/
if (spam->override != 0) {
prob = 0.1;
if (spam->override < 0)
prob = 0.00 - (opts->threshold + 0.01);
spam_free(spam);
return prob;
}
prob = spam_check__robinson(opts, msg, spam);
spam_free(spam);
if (prob > opts->threshold)
return prob - opts->threshold;
return prob - (opts->threshold + 0.01);
}
/* EOF */
qsf-1.2.7/src/spam/plaintext.c 0000644 0000764 0000764 00000011165 10664772304 014042 0 ustar aw aw /*
* Plaintext database mapping functions.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "spami.h"
#include "log.h"
#include
#include
#include
#include
#ifdef HAVE_FCNTL
#include
#endif
struct spam_p_item_s {
char *hash;
char *token;
};
struct spam_p_s {
FILE *fptr;
#ifdef HAVE_FCNTL
int lockcount;
#endif
struct spam_p_item_s *list;
int list_size;
int list_alloced;
};
#ifdef HAVE_FCNTL
/*
* Obtain / release a read or write lock on the database. Returns nonzero on
* error, and blocks until a lock can be obtained.
*/
static int spam_plaintext__lock(struct spam_p_s *data, int lock_type)
{
struct flock lock;
if ((lock_type == F_UNLCK) && (data->lockcount > 1)) {
data->lockcount--;
return 0;
} else if ((lock_type != F_UNLCK) && (data->lockcount > 0)) {
data->lockcount++;
return 0;
}
lock.l_whence = SEEK_SET;
lock.l_start = 0;
lock.l_len = 0;
lock.l_type = lock_type;
if (fcntl(fileno(data->fptr), F_SETLKW, &lock)) {
return 1;
}
if (lock_type == F_UNLCK) {
data->lockcount--;
} else {
data->lockcount++;
}
return 0;
}
#endif /* HAVE_FCNTL */
/*
* Add the given hash / token pair to the in-memory list.
*/
static void spam_plaintext__append(struct spam_p_s *data, char *hash,
int hashlen, char *token, int tokenlen)
{
struct spam_p_item_s *ptr;
if (data->list_size >= data->list_alloced) {
if (data->list) {
ptr = realloc(data->list, /* RATS: ignore */
(sizeof *ptr) * (data->list_alloced +
1000));
} else {
ptr =
malloc((sizeof *ptr) *
(data->list_alloced + 1000));
}
if (ptr == NULL) {
log_add(1, "%s", strerror(errno));
return;
}
data->list = ptr;
data->list_alloced += 1000;
}
data->list[data->list_size].hash = calloc(1, hashlen + 1);
data->list[data->list_size].token = calloc(1, tokenlen + 1);
if (data->list[data->list_size].hash)
strncpy(data->list[data->list_size].hash, hash, hashlen);
if (data->list[data->list_size].token)
strncpy(data->list[data->list_size].token, token,
tokenlen);
data->list_size++;
}
/*
* Update the plaintext mapping with the given hash / token pair.
*/
void spam_plaintext_update(opts_t opts, char *hash, int hashlen,
char *token, int tokenlen)
{
struct spam_p_s *data;
int i;
if (opts->plainmap == NULL)
return;
if (opts->plaindata == NULL) {
opts->plaindata = calloc(1, sizeof(struct spam_p_s));
if (opts->plaindata == NULL) {
log_add(1, "%s: %s", opts->plainmap,
strerror(errno));
return;
}
}
data = opts->plaindata;
if (data->fptr == NULL) {
char hashbuf[1024]; /* RATS: ignore */
char tokenbuf[1024]; /* RATS: ignore */
data->fptr = fopen(opts->plainmap, "a+");
if (data->fptr == NULL) {
log_add(1, "%s: %s", opts->plainmap,
strerror(errno));
return;
}
#ifdef HAVE_FCNTL
spam_plaintext__lock(data, F_WRLCK);
#endif
fseek(data->fptr, 0, SEEK_SET);
while (!feof(data->fptr)) {
char linebuf[4096]; /* RATS: ignore */
int n;
linebuf[0] = 0;
if (fgets(linebuf, sizeof(linebuf), data->fptr) ==
NULL)
break;
linebuf[sizeof(linebuf) - 1] = 0;
hashbuf[0] = 0;
tokenbuf[0] = 0;
n = sscanf(linebuf, "%1023[^\t\n]\t%1023[^\n]",
hashbuf, tokenbuf);
if (n < 2)
continue;
spam_plaintext__append(data, hashbuf,
strlen(hashbuf), tokenbuf,
strlen(tokenbuf));
}
}
for (i = 0; i < data->list_size; i++) {
if (data->list[i].hash == NULL)
continue;
if (strlen(data->list[i].hash) != hashlen)
continue;
if (strncmp(data->list[i].hash, hash, hashlen) != 0)
continue;
return;
}
spam_plaintext__append(data, hash, hashlen, token, tokenlen);
if (hashlen > 1023)
hashlen = 1023;
if (tokenlen > 1023)
tokenlen = 1023;
for (i = 0; i < tokenlen; i++) {
if (token[i] == '\n') {
tokenlen = i;
break;
}
}
fseek(data->fptr, 0, SEEK_END);
fprintf(data->fptr, "%.*s\t%.*s\n", hashlen, hash, tokenlen,
token);
}
/*
* Free the plaintext handling data area.
*/
void spam_plaintext_free(opts_t opts)
{
struct spam_p_s *data;
int i;
if (opts == NULL)
return;
if (opts->plainmap == NULL)
return;
if (opts->plaindata == NULL)
return;
data = opts->plaindata;
if (data->fptr) {
fflush(data->fptr);
#ifdef HAVE_FCNTL
spam_plaintext__lock(data, F_UNLCK);
#endif
fclose(data->fptr);
}
for (i = 0; i < data->list_size; i++) {
if (data->list[i].hash)
free(data->list[i].hash);
if (data->list[i].token)
free(data->list[i].token);
}
if (data->list)
free(data->list);
free(opts->plaindata);
opts->plaindata = NULL;
}
/* EOF */
qsf-1.2.7/src/spam/dump.c 0000644 0000764 0000764 00000017165 10664772304 013005 0 ustar aw aw /*
* Functions for dumping and restoring the spam database, and for dumping
* tokens from a message.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "spami.h"
#include
#include
#include
#include
#include
#include
#include
#include
/*
* Dump the tokens from the given message on stdout, returning nonzero on
* error.
*/
int spam_dumptokens(opts_t opts, msg_t msg)
{
spam_t spam;
long i;
spam = spam_tokenise(opts, msg, NULL, NULL, NULL, 1, 1, 1);
if (spam == NULL)
return 1;
for (i = 0; i < spam->token_count; i++) {
printf("%ld\t%.*s\n",
spam->tarray[i]->count,
spam->tarray[i]->length, spam->tarray[i]->token);
}
spam_free(spam);
return 0;
}
/*
* Dump the database on stdout in text form, returning nonzero on error.
*/
int spam_db_dump(opts_t opts)
{
FILE *fptr;
qdb_t db;
qdb_datum key, val, nextkey;
long a, b, c;
time_t t;
if ((opts->argc == 1) && (opts->argv[0])
&& (strcmp(opts->argv[0], "-") != 0)) {
struct stat outsb, sb;
/*
* Before trying to write to the file, check it's not the
* same file as any of the databases. We don't care if
* someone moves it in the middle of us trying something.
*/
if (stat(opts->argv[0], &outsb)) { /* RATS: ignore */
if (errno != ENOENT) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name, opts->argv[0],
strerror(errno));
return 1;
}
} else {
/*
* File exists; check against databases.
*/
if ((opts->dbw)
&& (fstat(qdb_fd(opts->dbw), &sb) == 0)
) {
if ((sb.st_dev == outsb.st_dev)
&& (sb.st_ino == outsb.st_ino)
) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
opts->argv[0],
_
("attempted to dump to an existing database"));
return 1;
}
}
if ((opts->dbr1)
&& (fstat(qdb_fd(opts->dbr1), &sb) == 0)
) {
if ((sb.st_dev == outsb.st_dev)
&& (sb.st_ino == outsb.st_ino)
) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
opts->argv[0],
_
("attempted to dump to an existing database"));
return 1;
}
}
if ((opts->dbr2)
&& (fstat(qdb_fd(opts->dbr2), &sb) == 0)
) {
if ((sb.st_dev == outsb.st_dev)
&& (sb.st_ino == outsb.st_ino)
) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
opts->argv[0],
_
("attempted to dump to an existing database"));
return 1;
}
}
if ((opts->dbr3)
&& (fstat(qdb_fd(opts->dbr3), &sb) == 0)
) {
if ((sb.st_dev == outsb.st_dev)
&& (sb.st_ino == outsb.st_ino)
) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
opts->argv[0],
_
("attempted to dump to an existing database"));
return 1;
}
}
}
/*
* Now we've done the checks, so try and open the file for
* writing.
*/
fptr = fopen(opts->argv[0], "w");
if (fptr == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
opts->argv[0], strerror(errno));
return 1;
}
} else {
fptr = stdout;
}
time(&t);
db = opts->dbw;
if (db == NULL)
db = opts->dbr1;
if (db == NULL)
db = opts->dbr2;
if (db == NULL)
db = opts->dbr3;
if (db == NULL) {
fprintf(stderr, "%s: %s\n", opts->program_name,
_("cannot find a database to dump"));
return 1;
}
a = 0;
b = 0;
c = 0;
key.data = (unsigned char *) " COUNTS";
key.size = 7;
val = qdb_fetch(db, key);
if (val.data != NULL) {
a = ((long *) (val.data))[0];
b = ((long *) (val.data))[1];
if (val.size > 2 * sizeof(long))
c = ((long *) (val.data))[2];
free(val.data);
val.data = NULL;
}
fprintf(fptr, "# %s %s %s\n", PROGRAM_NAME, _("database dump"),
VERSION);
fprintf(fptr, "# %s: %s\n", _("Date of dump"), ctime(&t));
fprintf(fptr, "COUNT-SPAM %ld\n", a);
fprintf(fptr, "COUNT-NONSPAM %ld\n", b);
fprintf(fptr, "COUNT-UPDATES %ld\n\n", c);
a = 0;
key.data = (unsigned char *) " SINCEPRUNE";
key.size = 11;
val = qdb_fetch(db, key);
if (val.data != NULL) {
a = ((long *) (val.data))[0];
b = ((long *) (val.data))[1];
free(val.data);
val.data = NULL;
}
fprintf(fptr, "SINCEPRUNE %ld\n\n", a);
fprintf(fptr, "# %s\t%s\t%s\t%s\n\n", _("Token"), _("Spam"),
_("Non-Spam"), _("Last Updated"));
key = qdb_firstkey(db);
while (key.data != NULL) {
val.data = NULL;
if (((key.size == 7)
&& (strncmp((char *) (key.data), " COUNTS", 7) == 0)
) || ((key.size == 11)
&&
(strncmp((char *) (key.data), " SINCEPRUNE", 11)
== 0)
)
) {
val.data = NULL;
} else {
val = qdb_fetch(db, key);
}
if (val.data != NULL) {
a = ((long *) (val.data))[0];
b = ((long *) (val.data))[1];
c = 0;
if (val.size > 2 * sizeof(long))
c = ((long *) (val.data))[2];
fprintf(fptr, "%.*s\t%ld\t%ld\t%ld\n", key.size,
key.data, a, b, c);
free(val.data);
}
nextkey = qdb_nextkey(db, key);
free(key.data);
key = nextkey;
}
if ((opts->argc == 1) && (opts->argv[0])
&& (strcmp(opts->argv[0], "-") != 0))
fclose(fptr);
return 0;
}
/*
* Restore the database from stdin in text form, returning nonzero on error.
*/
int spam_db_restore(opts_t opts)
{
FILE *fptr;
char linebuf[1024]; /* RATS: ignore (checked all) */
char tokenbuf[64]; /* RATS: ignore (checked all) */
qdb_t db = NULL;
qdb_datum key, val;
long a, b, c;
long dat[3];
int got_count = 0;
if ((opts->argc == 1) && (opts->argv[0])
&& (strcmp(opts->argv[0], "-") != 0)) {
fptr = fopen(opts->argv[0], "r");
if (fptr == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
opts->argv[0], strerror(errno));
return 1;
}
} else {
fptr = stdin;
}
a = 0;
b = 0;
c = 0;
db = opts->dbw;
if (db == NULL) {
fprintf(stderr, "%s: %s\n", opts->program_name,
_("cannot write to a database"));
if ((opts->argc == 1) && (opts->argv[0])
&& (strcmp(opts->argv[0], "-") != 0)) {
fclose(fptr);
}
return 1;
}
while (fgets(linebuf, sizeof(linebuf) - 1, fptr) != NULL) {
linebuf[sizeof(linebuf) - 1] = 0;
if (linebuf[0] == '#'
|| linebuf[0] == ' '
|| linebuf[0] == '\t'
|| linebuf[0] == '\r' || linebuf[0] == '\n')
continue;
switch (got_count) {
case 0:
if (sscanf(linebuf, "COUNT-SPAM %ld", &a) == 1)
got_count++;
break;
case 1:
if (sscanf(linebuf, "COUNT-NONSPAM %ld", &b) == 1)
got_count++;
break;
case 2:
if (sscanf(linebuf, "COUNT-UPDATES %ld", &c) == 1)
break;
qdb_restore_start(db);
key.data = (unsigned char *) " COUNTS";
key.size = 7;
val.data = (unsigned char *) dat;
val.size = sizeof(dat);
dat[0] = a;
dat[1] = b;
dat[2] = c;
qdb_store(db, key, val);
got_count++;
default:
c = 0;
if (sscanf(linebuf, "SINCEPRUNE %ld", &a) == 1) {
key.data = (unsigned char *) " SINCEPRUNE";
key.size = 11;
val.data = (unsigned char *) dat;
val.size = sizeof(dat);
dat[0] = a;
dat[1] = 0;
dat[2] = 0;
qdb_store(db, key, val);
} else if (sscanf(linebuf, /* RATS: ignore (const fmt) */
"%40[\\!?" TOKEN_CHARS "] " /*s" */
"%ld %ld %ld",
tokenbuf, &a, &b, &c) >= 3) {
tokenbuf[sizeof(tokenbuf) - 1] = 0;
key.data = (unsigned char *) tokenbuf;
key.size = strlen(tokenbuf);
val.data = (unsigned char *) dat;
val.size = sizeof(dat);
dat[0] = a;
dat[1] = b;
dat[2] = c;
qdb_store(db, key, val);
}
break;
}
}
if (got_count > 2) {
qdb_restore_end(db);
}
if ((opts->argc == 1) && (opts->argv[0])
&& (strcmp(opts->argv[0], "-") != 0)) {
fclose(fptr);
}
return 0;
}
/* EOF */
qsf-1.2.7/src/spam/token.c 0000644 0000764 0000764 00000015574 10664772304 013162 0 ustar aw aw /*
* Functions for breaking a message up into tokens.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "spami.h"
#include
#include
#include
#include
/*
* Add a new token to the token tree.
*/
void spam_token_add(opts_t opts, spam_t spam, char *str, int len)
{
token_t *parentptr;
token_t token;
int c;
/*
* Refuse to add a token that's too small.
*/
if (len < 2)
return;
token = spam->tokens;
parentptr = &(spam->tokens);
while (token != NULL) {
if (len > token->length) {
parentptr = &(token->longer);
} else {
c = strncmp(str, token->token, len);
if (len == token->length && c == 0) {
break;
} else if (c < 0) {
parentptr = &(token->lower);
} else {
parentptr = &(token->higher);
}
}
token = *parentptr;
}
if (token == NULL) {
token = calloc(1, sizeof(*token));
if (token == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
return;
}
*parentptr = token;
spam->token_count++;
token->token = str;
token->length = len;
}
if (token->count < 1)
spam_fetch(spam, str, len, &(token->num_spam),
&(token->num_nonspam), &(token->last_updated));
token->count++;
}
/*
* Add a token and (recursively) all its leaf nodes to the array.
*/
static void spam_token_arrayadd(spam_t spam, token_t token, int depth)
{
if (token == NULL)
return;
if (depth > 10000)
return;
if (spam->_idx >= spam->token_count)
return;
spam->tarray[spam->_idx++] = token;
spam_token_arrayadd(spam, token->lower, depth + 1);
spam_token_arrayadd(spam, token->longer, depth + 1);
spam_token_arrayadd(spam, token->higher, depth + 1);
}
/*
* Return the length of the initial segment of "buf" (length "buflen") that
* consists entirely of characters in "accept".
*/
static int spam_tokenise__span(char *buf, int buflen, char *accept)
{
int len;
for (len = 0; (len < buflen) && strchr(accept, buf[len]); len++) {
}
return len;
}
/*
* Return the length of the initial segment of "buf" (length "buflen") that
* consists entirely of characters not in "reject".
*/
static int spam_tokenise__cspan(char *buf, int buflen, char *reject)
{
int len;
for (len = 0; (len < buflen) && !strchr(reject, buf[len]); len++) {
}
return len;
}
/*
* Tokenise the given message and return a spam structure containing the
* tokens in the message and their spam ratings. Returns NULL on error. The
* following special token names are used:
*
* " COUNTS" - total number of spam and non-spam email messages seen,
* plus the total number of updates we have ever applied
* " SINCEPRUNE" - number of updates since the last prune (first val only)
*
* Note that these special token names all start with a space, so as to not
* clash with any "real" tokens.
*/
spam_t spam_tokenise(opts_t opts, msg_t msg, qdb_t db1, qdb_t db2,
qdb_t db3, int db1weight, int db2weight,
int db3weight)
{
spam_t spam;
long pos, start, len, i, n, dummy;
char *content;
long content_size;
long prevstart;
spam = calloc(1, sizeof(*spam));
if (spam == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
return NULL;
}
spam->db1 = db1;
spam->db2 = db2;
spam->db3 = db3;
spam->dbw = opts->dbw;
spam->db1weight = db1weight;
spam->db2weight = db2weight;
spam->db3weight = db3weight;
spam_fetch(spam, " COUNTS", 7, &(spam->total_spam),
&(spam->total_nonspam), &(spam->update_count));
spam_fetch(spam, " SINCEPRUNE", 11, &(spam->since_prune), &pos,
&dummy);
if (spam->total_spam < 1)
spam->total_spam = 1;
if (spam->total_nonspam < 1)
spam->total_nonspam = 1;
content = msg->textcontent;
content_size = msg->text_size;
if (content == NULL) {
content = msg->content;
content_size = msg->content_size;
}
prevstart = 0;
for (pos = 0; pos < content_size;) {
len =
spam_tokenise__span(content + pos, content_size - pos,
TOKEN_CHARS);
if (len <= 0) {
len = spam_tokenise__cspan(content + pos,
content_size - pos,
TOKEN_CHARS);
pos += len;
if (len < 1)
pos++;
continue;
}
start = pos;
pos += len;
/*
* Don't allow a token to start with 0-9, -, ', !, or .
*/
if ((content[start] >= '0' && content[start] <= '9')
|| content[start] == '-'
|| content[start] == '\''
|| content[start] == '!'
|| content[start] == '?' || content[start] == '.')
continue;
/*
* Skip tokens that are too short or too long
*/
if ((len < 3) || (len > 34))
continue;
/*
* Make all tokens lower case (tests indicate that case
* sensitive tokens lead to a higher false positive rate, as
* well as making the token database bigger)
*/
for (i = 0; i < len; i++) {
if (content[start + i] >= 'A'
&& content[start + i] <= 'Z')
content[start + i] += 32;
}
/*
* Strip -, ', . from end of token
*/
for (i = len - 1; i > 0; i--) {
if (content[start + i] == '-'
|| content[start + i] == '\''
|| content[start + i] == '.') {
len--;
} else {
break;
}
}
/*
* Check length again, eg in case token was "a--"
*/
if (len < 2)
continue;
spam_token_add(opts, spam, content + start, len);
/*
* If this isn't the first token, add a second pseudo-token
* consisting of all the text from the start of the previous
* token to the end of the current one.
*/
if ((prevstart != 0) && (prevstart < start)) {
spam_token_add(opts, spam,
content + prevstart,
(start - prevstart) + len);
}
prevstart = start;
}
spam->override = spam_test(opts, spam, msg);
if (spam->token_count < 1)
return spam;
spam->tarray = calloc(spam->token_count, sizeof(token_t));
if (spam->tarray == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
return spam;
}
spam_token_arrayadd(spam, spam->tokens, 0);
spam->token_count = spam->_idx;
n = 0;
spam->robx = 0.0;
/*
* Calculate the spam probabilities for each token.
*
* Formerly we did (bad / tb) / ((bad / tb) + (good / tg)) where
* tb/tg are total bad/good messages seen. This appears to be
* slightly redundant, so we now just use the token counts directly,
* i.e. we do pw = bad / (bad+good) instead. This approximation
* allows us to not worry about how to modify message total counts
* when the database is pruned.
*/
for (i = 0; i < spam->token_count; i++) {
double good, bad, gbtot, pw;
token_t token;
token = spam->tarray[i];
good = token->num_nonspam;
bad = token->num_spam;
gbtot = good + bad;
pw = 0.0;
if (gbtot > 0) {
pw = bad / (bad + good);
if (gbtot > 10.0) {
spam->robx += pw;
n++;
}
}
token->prob_spam = pw;
}
if (n > 0) {
spam->robx = spam->robx / (double) n;
} else {
spam->robx = 0.5;
}
return spam;
}
/* EOF */
qsf-1.2.7/src/spam/db.c 0000644 0000764 0000764 00000007061 10664772304 012417 0 ustar aw aw /*
* Database fetch/store functions.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "spami.h"
#include
#include
/*
* Fetch the given key from the database, trying all databases if
* applicable; if both databases have a value for that key, they are added
* together - except for the SINCEPRUNE counter, which is only ever read
* from the writable database and is read as 0 if all databases are
* read-only. If the third value is not present, it is 0; if present in more
* than one database, the highest value is returned.
*/
void spam_fetch(spam_t spam, char *key, int len, long *val1, long *val2,
long *val3)
{
qdb_datum dkey, val;
int needcksum = 0;
dkey.data = (unsigned char *) key;
dkey.size = len;
if ((dkey.size == 11) && (strncmp(key, " SINCEPRUNE", 11) == 0)) {
*val1 = 0;
*val2 = 0;
*val3 = 0;
if (spam->dbw) {
val = qdb_fetch(spam->dbw, dkey);
if (val.data != NULL) {
*val1 = ((long *) (val.data))[0];
*val2 = ((long *) (val.data))[1];
if (val.size > 2 * sizeof(long))
*val3 = ((long *) (val.data))[2];
free(val.data);
}
return;
} else {
return;
}
}
if ((key[0] != ' ')
&& (key[0] != '!')
&& (key[0] != '?')
&& (key[0] != '\\')
&& (key[0] != '.')
) {
needcksum = 1;
dkey.data = spam_checksum(key, len);
dkey.size = strlen((char *) (dkey.data));
}
*val3 = 0;
val = qdb_fetch(spam->db1, dkey);
if (val.data != NULL) {
*val1 += spam->db1weight * ((long *) (val.data))[0];
*val2 += spam->db1weight * ((long *) (val.data))[1];
if (val.size > 2 * sizeof(long)) {
long n;
n = ((long *) (val.data))[2];
if (n > *val3)
*val3 = n;
}
free(val.data);
}
val = qdb_fetch(spam->db2, dkey);
if (val.data != NULL) {
*val1 += spam->db2weight * ((long *) (val.data))[0];
*val2 += spam->db2weight * ((long *) (val.data))[1];
if (val.size > 2 * sizeof(long)) {
long n;
n = ((long *) (val.data))[2];
if (n > *val3)
*val3 = n;
}
free(val.data);
}
val = qdb_fetch(spam->db3, dkey);
if (val.data != NULL) {
*val1 += spam->db3weight * ((long *) (val.data))[0];
*val2 += spam->db3weight * ((long *) (val.data))[1];
if (val.size > 2 * sizeof(long)) {
long n;
n = ((long *) (val.data))[2];
if (n > *val3)
*val3 = n;
}
free(val.data);
}
if (needcksum)
free(dkey.data);
}
/*
* Store the given key into the database that is opened for write access.
*/
void spam_store(opts_t opts, char *key, int len, long val1, long val2,
long val3)
{
qdb_datum dkey, dval;
int needcksum = 0;
long dat[3];
dkey.data = (unsigned char *) key;
dkey.size = len;
if ((key[0] != ' ')
&& (key[0] != '!')
&& (key[0] != '?')
&& (key[0] != '\\')
&& (key[0] != '.')
) {
needcksum = 1;
dkey.data = spam_checksum(key, len);
dkey.size = strlen((char *) (dkey.data));
}
dval.data = (unsigned char *) dat;
dval.size = sizeof(dat);
dat[0] = val1;
dat[1] = val2;
dat[2] = val3;
qdb_store(opts->dbw, dkey, dval);
if (needcksum) {
if (opts->plainmap) {
spam_plaintext_update(opts, (char *) (dkey.data),
dkey.size, key, len);
}
free(dkey.data);
}
}
/*
* Temporarily release the lock on the writable database, if any.
*/
void spam_dbunlock(opts_t opts)
{
if (opts == NULL)
return;
if (opts->dbw == NULL)
return;
qdb_unlock(opts->dbw);
}
/*
* Reassert the lock on the writable database, if any.
*/
void spam_dbrelock(opts_t opts)
{
if (opts == NULL)
return;
if (opts->dbw == NULL)
return;
qdb_relock(opts->dbw);
}
/* EOF */
qsf-1.2.7/src/spam/prune.c 0000644 0000764 0000764 00000035723 10664772304 013171 0 ustar aw aw /*
* Functions for pruning a spam database.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "spami.h"
#include
#include
#include
#include
#include
#include
#undef DEBUG_THRESHOLD
#undef DEBUG_DISCARD
void tick(void);
/*
* Scan through the database and return the average spam (or non-spam) token
* count, and fill in the maximum count found.
*/
static double spam_db_prune_old__maxcount(opts_t opts, long *maxcount,
int spam)
{
long num_tokens, num_spam, num_nonspam;
qdb_datum key, val, nextkey;
double avg;
num_tokens = 0;
avg = 0;
*maxcount = 0;
key = qdb_firstkey(opts->dbw);
while (key.data != NULL) {
if (opts->showprune) {
tick();
}
val.data = NULL;
if (((key.size == 7)
&& (strncmp((char *) (key.data), " COUNTS", 7) == 0)
) || ((key.size == 11)
&&
(strncmp((char *) (key.data), " SINCEPRUNE", 11)
== 0)
) || (key.data[0] == '?')
) {
val.data = NULL;
} else {
val = qdb_fetch(opts->dbw, key);
}
if (val.data != NULL) {
num_tokens++;
free(val.data);
}
nextkey = qdb_nextkey(opts->dbw, key);
free(key.data);
key = nextkey;
}
key = qdb_firstkey(opts->dbw);
while (key.data != NULL) {
if (opts->showprune) {
tick();
}
val.data = NULL;
if (((key.size == 7)
&& (strncmp((char *) (key.data), " COUNTS", 7) == 0)
) || ((key.size == 11)
&&
(strncmp((char *) (key.data), " SINCEPRUNE", 11)
== 0)
) || (key.data[0] == '?')
) {
val.data = NULL;
} else {
val = qdb_fetch(opts->dbw, key);
}
if (val.data == NULL) {
nextkey = qdb_nextkey(opts->dbw, key);
free(key.data);
key = nextkey;
continue;
}
num_spam = ((long *) (val.data))[0];
num_nonspam = ((long *) (val.data))[1];
free(val.data);
if (spam == SPAM) {
avg += ((double) num_spam) / ((double) num_tokens);
if (num_spam > *maxcount)
*maxcount = num_spam;
} else {
avg +=
((double) num_nonspam) / ((double) num_tokens);
if (num_nonspam > *maxcount)
*maxcount = num_nonspam;
}
nextkey = qdb_nextkey(opts->dbw, key);
free(key.data);
key = nextkey;
}
return avg;
}
/*
* Scan through the database looking for tokens to strip, and strip them. We
* also cap token counts to stop them getting too high relative to the total
* message counts. Returns nonzero on error.
*/
static int spam_db_prune_old__strip(opts_t opts)
{
qdb_datum key, val, nextkey;
long total_spam, total_nonspam, total_updates;
long num_spam, num_nonspam, last_updated;
double good, bad, prob_spam;
qdb_datum *to_remove = NULL;
qdb_datum *ptr;
long num_to_remove = 0;
long to_remove_alloced = 0;
long total_tokens;
long a, b, c, i;
int clip;
a = 0;
b = 0;
c = 0;
key.data = (unsigned char *) " COUNTS";
key.size = 7;
val = qdb_fetch(opts->dbw, key);
if (val.data != NULL) {
a = ((long *) (val.data))[0];
b = ((long *) (val.data))[1];
if (val.size > 2 * sizeof(long))
c = ((long *) (val.data))[2];
}
total_spam = a;
total_nonspam = b;
total_updates = c;
if (total_spam < 1)
total_spam = 1;
if (total_nonspam < 1)
total_nonspam = 1;
total_tokens = 0;
key = qdb_firstkey(opts->dbw);
while (key.data != NULL) {
if (opts->showprune) {
tick();
}
val.data = NULL;
if (((key.size == 7)
&& (strncmp((char *) (key.data), " COUNTS", 7) == 0)
) || ((key.size == 11)
&&
(strncmp((char *) (key.data), " SINCEPRUNE", 11)
== 0)
) || (key.data[0] == '?')
) {
val.data = NULL;
} else {
val = qdb_fetch(opts->dbw, key);
}
if (val.data == NULL) {
nextkey = qdb_nextkey(opts->dbw, key);
free(key.data);
key = nextkey;
continue;
}
num_spam = ((long *) (val.data))[0];
num_nonspam = ((long *) (val.data))[1];
last_updated = 0;
if (val.size > 2 * sizeof(long))
last_updated = ((long *) (val.data))[2];
free(val.data);
clip = 0;
if (num_spam > 3 * total_spam) {
num_spam = 3 * total_spam;
clip = 1;
}
if (num_nonspam > 3 * total_nonspam) {
num_nonspam = 3 * total_nonspam;
clip = 1;
}
if (clip) {
spam_store(opts, (char *) (key.data), key.size,
num_spam, num_nonspam, last_updated);
}
total_tokens++;
good = 2 * num_nonspam;
bad = num_spam;
good = good / total_nonspam;
if (good > 1.0)
good = 1.0;
bad = bad / total_spam;
if (bad > 1.0)
bad = 1.0;
if (num_nonspam + num_spam < 4) {
prob_spam = 0.5;
} else if ((good < 0.00001) && (bad < 0.00001)) {
prob_spam = 0.5;
} else if (2 * num_nonspam + num_spam > 5) {
prob_spam = bad / (good + bad);
if (prob_spam > 0.9999) {
prob_spam = 0.9999;
} else if (prob_spam < 0.0001) {
prob_spam = 0.0001;
}
} else {
prob_spam = 1.5;
}
if ((prob_spam >= 0.48 && prob_spam <= 0.52)
&& (num_to_remove < opts->prune_max)
) {
/*
* Add this token to the list to delete afterwards
*/
num_to_remove++;
if (num_to_remove > to_remove_alloced) {
to_remove_alloced = num_to_remove + 10000;
ptr = realloc(to_remove, /* RATS: ignore */
to_remove_alloced
* sizeof(qdb_datum));
if (ptr == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_
("memory allocation failed"),
strerror(errno));
return 1;
}
to_remove = ptr;
}
to_remove[num_to_remove - 1].size = key.size;
to_remove[num_to_remove - 1].data =
malloc(key.size);
if (to_remove[num_to_remove - 1].data == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("memory allocation failed"),
strerror(errno));
return 1;
}
memcpy(to_remove[num_to_remove - 1].data, key.data,
key.size);
}
nextkey = qdb_nextkey(opts->dbw, key);
free(key.data);
key = nextkey;
}
if (opts->showprune) {
printf(" %ld/%ld [%3.2f%%]",
num_to_remove, total_tokens,
total_tokens > 0
? 100.0 * (double) num_to_remove
/ (double) total_tokens : 0);
}
/*
* Now we remove any keys in the remove list.
*/
if (num_to_remove > 0) {
for (i = 0; i < num_to_remove; i++) {
if (opts->showprune) {
tick();
}
qdb_delete(opts->dbw, to_remove[i]);
free(to_remove[i].data);
}
if (to_remove != NULL)
free(to_remove);
}
if (opts->showprune) {
printf(" %s\n", _("removed"));
}
return 0;
}
/*
* Prune the currently writable database, removing redundant entries and
* scaling down token and message counts if they get too large. Returns
* nonzero on error.
*
* This is the version for old-style databases that don't use token aging.
*/
static int spam_db_prune_old(opts_t opts)
{
qdb_datum key, val, nextkey;
long total_spam, total_nonspam, total_updates, max_spam,
max_nonspam;
long num_spam, num_nonspam, last_updated;
double avg_spam, avg_nonspam, top_count, scale_by;
if (opts->dbw == NULL) {
if (opts->showprune) {
fprintf(stderr, "%s: %s\n", opts->program_name,
_
("pruning requires write access to the database"));
}
return 1;
}
setvbuf(stdin, NULL, _IONBF, 0);
setvbuf(stdout, NULL, _IONBF, 0);
key.data = (unsigned char *) " SINCEPRUNE";
key.size = 11;
spam_store(opts, (char *) (key.data), key.size, 0, 0, 0);
total_spam = 0;
total_nonspam = 0;
total_updates = 0;
key.data = (unsigned char *) " COUNTS";
key.size = 7;
val = qdb_fetch(opts->dbw, key);
if (val.data != NULL) {
total_spam = ((long *) (val.data))[0];
total_nonspam = ((long *) (val.data))[1];
if (val.size > 2 * sizeof(long))
total_updates = ((long *) (val.data))[2];
free(val.data);
val.data = NULL;
}
if (total_spam < 1)
total_spam = 1;
if (total_nonspam < 1)
total_nonspam = 1;
/*
* Scan through all tokens in the database and get the highest
* counts for spam and non-spam.
*/
if (opts->showprune) {
printf("%s", _("Checking average token counts..."));
}
avg_spam = spam_db_prune_old__maxcount(opts, &max_spam, SPAM);
avg_nonspam =
spam_db_prune_old__maxcount(opts, &max_nonspam, NONSPAM);
if (avg_spam < 1)
avg_spam = 1;
if (avg_nonspam < 1)
avg_nonspam = 1;
if (opts->showprune) {
printf(" %f %f\n", avg_spam, avg_nonspam);
}
/*
* Scan through all tokens in the database looking for those that
* have a spam probability of somewhere around the middle or whose
* counts are very tiny compared to the total spam and nonspam
* counts, and remove them.
*/
if (opts->showprune) {
printf("%s", _("Scanning for removable tokens..."));
}
if (spam_db_prune_old__strip(opts))
return 1;
/*
* Next, we scan through the database and scale down all of the
* counts, and also scale down the total counts by the same amount.
* This helps new additions to the database to actually make an
* impact later on.
*/
top_count = (avg_spam < avg_nonspam ? avg_spam : avg_nonspam);
scale_by = top_count / 1000.0;
if (scale_by < 1.9) {
/* Not worth scaling - just return */
return 0;
}
if (opts->showprune) {
printf( /* RATS: ignore */ _
("Scaling down token counts by a factor of %f (max: %f)..."),
scale_by, top_count);
}
key = qdb_firstkey(opts->dbw);
while (key.data != NULL) {
if (opts->showprune) {
tick();
}
val.data = NULL;
if (key.data[0] != '?') {
val = qdb_fetch(opts->dbw, key);
}
if (val.data == NULL) {
nextkey = qdb_nextkey(opts->dbw, key);
free(key.data);
key = nextkey;
continue;
}
num_spam = ((long *) (val.data))[0];
num_nonspam = ((long *) (val.data))[1];
last_updated = 0;
if (val.size > 2 * sizeof(long))
last_updated = ((long *) (val.data))[2];
free(val.data);
spam_store(opts,
(char *) (key.data),
key.size,
(long) (((double) num_spam) / scale_by),
(long) (((double) num_nonspam) / scale_by),
last_updated);
nextkey = qdb_nextkey(opts->dbw, key);
free(key.data);
key = nextkey;
}
if (opts->showprune) {
printf(" %s\n", _("done"));
}
/*
* Now we go through the database again and strip out removable
* tokens a final time.
*/
if (opts->showprune) {
printf("%s", _("Secondary scan for removable tokens..."));
}
return spam_db_prune_old__strip(opts);
}
/*
* Prune the currently writable database, removing redundant and old
* entries. Returns nonzero on error.
*/
int spam_db_prune(opts_t opts)
{
qdb_datum key, val, nextkey;
long total_updates, num_spam, num_nonspam, last_updated;
qdb_datum *to_remove = NULL;
qdb_datum *ptr;
long num_to_remove = 0;
long to_remove_alloced = 0;
long total_tokens = 0;
if (opts->dbw == NULL) {
if (opts->showprune) {
fprintf(stderr, "%s: %s\n", opts->program_name,
_
("pruning requires write access to the database"));
}
return 1;
}
total_updates = -1;
key.data = (unsigned char *) " COUNTS";
key.size = 7;
val = qdb_fetch(opts->dbw, key);
if (val.data != NULL) {
if (val.size > 2 * sizeof(long))
total_updates = ((long *) (val.data))[2];
free(val.data);
val.data = NULL;
}
setvbuf(stdin, NULL, _IONBF, 0);
setvbuf(stdout, NULL, _IONBF, 0);
if (total_updates < 0)
return spam_db_prune_old(opts);
key.data = (unsigned char *) " SINCEPRUNE";
key.size = 11;
spam_store(opts, (char *) (key.data), key.size, 0, 0, 0);
/*
* Scan through all tokens in the database and make a list of the
* ones we're going to delete. These will be ones which are too old
* or insignificant to be of use; "insignificant" means that the
* contribution to the overall spam score of a message will be too
* small, and the threshold for "too small" moves according to how
* long ago the token was last updated.
*/
if (opts->showprune) {
printf("%s", _("Scanning for removable tokens..."));
}
key = qdb_firstkey(opts->dbw);
while (key.data != NULL) {
double good, bad, prob_spam, significance, threshold, age,
agescale;
if (opts->showprune) {
tick();
}
val.data = NULL;
/*
* Ignore global counters and allow-list entries - they are
* never pruned.
*/
if ((key.data[0] != '?') && (key.data[0] != ' ')) {
val = qdb_fetch(opts->dbw, key);
}
if (val.data == NULL) {
nextkey = qdb_nextkey(opts->dbw, key);
free(key.data);
key = nextkey;
continue;
}
total_tokens++;
num_spam = ((long *) (val.data))[0];
num_nonspam = ((long *) (val.data))[1];
last_updated = 0;
if (val.size > 2 * sizeof(long))
last_updated = ((long *) (val.data))[2];
free(val.data);
good = num_nonspam;
bad = num_spam;
age = total_updates - last_updated;
if (age < 1)
age = 1;
agescale = log(1.0 + (age / 10));
if (((good + bad) / agescale) < 1.0) {
/*
* Throw away tokens with very small counts - the
* older the token, the larger the counts must be
* for the token to be kept.
*/
prob_spam = 0.5;
} else {
prob_spam = bad / (good + bad);
}
significance = fabs(0.5 - prob_spam);
/*
* The threshold of significance: if the probability of
* being spam is at or further than this from even (0.5),
* then the token is worth keeping.
*/
threshold = 0.019;
#ifdef DEBUG_THRESHOLD
fprintf(stderr, "%s %ld %ld %ld %g %g\n",
significance < threshold ? "***" : " ",
num_nonspam, num_spam, (long) age, significance,
(good + bad) / agescale);
#endif
if ((significance < threshold)
&& (num_to_remove < opts->prune_max)) {
#ifdef DEBUG_DISCARD
fprintf(stderr, "%ld %ld %ld %g %g\n", num_nonspam,
num_spam, (long) age, significance,
(good + bad) / agescale);
#endif
/*
* Token is too insignificant to keep - mark it to
* be discarded.
*/
num_to_remove++;
if (num_to_remove > to_remove_alloced) {
to_remove_alloced = num_to_remove + 10000;
ptr = realloc(to_remove, /* RATS: ignore */
to_remove_alloced
* sizeof(qdb_datum));
if (ptr == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_
("memory allocation failed"),
strerror(errno));
return 1;
}
to_remove = ptr;
}
to_remove[num_to_remove - 1].size = key.size;
to_remove[num_to_remove - 1].data =
malloc(key.size);
if (to_remove[num_to_remove - 1].data == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("memory allocation failed"),
strerror(errno));
return 1;
}
memcpy(to_remove[num_to_remove - 1].data, key.data,
key.size);
}
nextkey = qdb_nextkey(opts->dbw, key);
free(key.data);
key = nextkey;
}
if (opts->showprune) {
printf(" %ld/%ld [%3.2f%%]",
num_to_remove, total_tokens,
total_tokens > 0
? 100.0 * (double) num_to_remove
/ (double) total_tokens : 0);
}
/*
* Now we remove any keys in the remove list.
*/
if (num_to_remove > 0) {
long i;
for (i = 0; i < num_to_remove; i++) {
if (opts->showprune) {
tick();
}
if (qdb_delete(opts->dbw, to_remove[i])) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("token deletion failed"),
qdb_error());
}
free(to_remove[i].data);
}
if (to_remove != NULL)
free(to_remove);
}
if (opts->showprune) {
printf(" %s\n", _("removed"));
}
return 0;
}
/* EOF */
qsf-1.2.7/src/spam/spami.h 0000644 0000764 0000764 00000005636 10554714216 013152 0 ustar aw aw /*
* Internal spam handling prototypes, structures, and constants.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#ifndef _SPAMI_H
#define _SPAMI_H 1
#ifndef _OPTIONS_H
#include "options.h"
#endif
#ifndef _MESSAGE_H
#include "message.h"
#endif
#ifndef _DATABASE_H
#include "database.h"
#endif
#ifndef _SPAM_H
#include "spam.h"
#endif
#ifdef __cplusplus
extern "C" {
#endif
struct token_s;
typedef struct token_s *token_t;
struct spam_s;
typedef struct spam_s *spam_t;
struct spam_s { /* structure describing spam state */
qdb_t db1; /* first database to read from */
qdb_t db2; /* second database to read from */
qdb_t db3; /* third database to read from */
qdb_t dbw; /* writable database handle */
int db1weight; /* weighting for first database */
int db2weight; /* weighting for second database */
int db3weight; /* weighting for third database */
int override; /* if non-zero, override spam_check() */
token_t tokens; /* start of token tree */
long token_count; /* number of different tokens */
token_t *tarray; /* token tree arranged as an array */
double robx; /* Robinson "x" value */
long total_spam; /* total spam messages seen */
long total_nonspam; /* total non-spam messages seen */
long since_prune; /* number of updates since last db prune */
long update_count; /* counter, increases every update */
long _idx; /* index used when filling array */
};
struct token_s { /* structure describing an email token */
char *token; /* pointer to token start */
int length; /* length of token */
long count; /* number of times token seen */
long num_spam; /* times token seen in spam */
long num_nonspam; /* times token seen in non-spam */
long last_updated; /* update_count at last update */
double prob_spam; /* probability this token is spammy */
token_t higher; /* pointer to token nearer "Z" */
token_t lower; /* pointer to token nearer "A" */
token_t longer; /* pointer to longer token */
};
void spam_token_add(opts_t, spam_t, char *, int);
spam_t spam_tokenise(opts_t, msg_t, qdb_t, qdb_t, qdb_t, int, int, int);
void spam_free(spam_t);
void spam_fetch(spam_t, char *, int, long *, long *, long *);
void spam_store(opts_t, char *, int, long, long, long);
void spam_dbunlock(opts_t);
void spam_dbrelock(opts_t);
unsigned char *spam_checksum(char *, int);
int spam_allowlist_match(spam_t, char *);
void spam_allowlist_add(opts_t, char *);
void spam_allowlist_remove(opts_t, char *);
int spam_denylist_match(spam_t, char *);
void spam_denylist_add(opts_t, char *);
void spam_denylist_remove(opts_t, char *);
int spam_test(opts_t, spam_t, msg_t);
#ifdef __cplusplus
}
#endif
#endif /* _SPAMI_H */
/* EOF */
qsf-1.2.7/src/spam/allowlist.c 0000644 0000764 0000764 00000015454 10664772304 014051 0 ustar aw aw /*
* Functions dealing with the allow-list and the deny-list.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "spami.h"
#include "log.h"
#include
#include
#include
#include
/*
* Return a malloc()ed pointer to a null-terminated string which is the
* database key for the given allow-list or deny-list email address.
*/
static char *spam_allowlist__token(char *address, int denylist)
{
unsigned char *ptr;
if (address == NULL)
return NULL;
ptr = spam_checksum(address, strlen(address));
if (ptr != NULL) {
if (denylist) {
ptr[0] = '\\';
} else {
ptr[0] = '?';
}
}
return (char *) ptr;
}
/*
* The same as spam_allowlist__token, but converts a copy of the address to
* lower case first.
*/
static char *spam_allowlist__lctoken(char *address, int denylist)
{
char *addrcopy;
char *key;
int i;
addrcopy = strdup(address);
if (addrcopy == NULL)
return NULL;
for (i = 0; addrcopy[i] != 0; i++) {
if (addrcopy[i] >= 'A' && addrcopy[i] <= 'Z')
addrcopy[i] += 32;
}
key = spam_allowlist__token(addrcopy, denylist);
free(addrcopy);
return key;
}
/*
* Look up the given email address in the allow-list or the deny-list,
* returning 1 if it is present, 0 if not. First the address is checked in
* lowercase, then as-is.
*/
static int spam_allowlist__domatch(spam_t spam, int denylist,
char *address)
{
char *key;
long a, b, c;
if (spam == NULL)
return 0;
if (address == NULL)
return 0;
key = spam_allowlist__lctoken(address, denylist);
if (key != NULL) {
a = 0;
b = 0;
c = 0;
spam_fetch(spam, key, strlen(key), &a, &b, &c);
free(key);
if (a > 0) {
log_add(2,
denylist ? _("deny-list match: %s") :
_("allow-list match: %s"), address);
return 1;
}
}
key = spam_allowlist__token(address, denylist);
if (key != NULL) {
a = 0;
b = 0;
c = 0;
spam_fetch(spam, key, strlen(key), &a, &b, &c);
free(key);
if (a > 0) {
log_add(2,
denylist ? _("deny-list match: %s") :
_("allow-list match: %s"), address);
return 1;
}
}
return 0;
}
/*
* Add the given email address to the allow-list or the deny-list. It is
* converted to lower case before being stored.
*/
static void spam_allowlist__doadd(opts_t opts, int denylist, char *address)
{
char *key;
if (opts == NULL)
return;
if (address == NULL)
return;
key = spam_allowlist__lctoken(address, denylist);
if (key == NULL)
return;
spam_store(opts, key, strlen(key), 1, 1, 0);
if (opts->plainmap) {
spam_plaintext_update(opts, key, strlen(key), address,
strlen(address));
}
free(key);
}
/*
* Remove the given email address from the allow-list or the deny-list. Both
* as-is and lower-case versions of the address will be removed.
*/
static void spam_allowlist__doremove(opts_t opts, int denylist,
char *address)
{
char *key;
qdb_datum qkey;
if (opts == NULL)
return;
if (opts->dbw == NULL)
return;
if (address == NULL)
return;
key = spam_allowlist__token(address, denylist);
if (key != NULL) {
qkey.data = (unsigned char *) key;
qkey.size = strlen(key);
qdb_delete(opts->dbw, qkey);
free(key);
}
key = spam_allowlist__lctoken(address, denylist);
if (key != NULL) {
qkey.data = (unsigned char *) key;
qkey.size = strlen(key);
qdb_delete(opts->dbw, qkey);
free(key);
}
}
/*
* Look up the given email address in the allow-list, returning 1 if it is
* present, 0 if not. First the address is checked in lowercase, then as-is.
*/
int spam_allowlist_match(spam_t spam, char *address)
{
return spam_allowlist__domatch(spam, 0, address);
}
/*
* Add the given email address to the allow-list. It is converted to lower
* case before being stored.
*/
void spam_allowlist_add(opts_t opts, char *address)
{
spam_allowlist__doadd(opts, 0, address);
}
/*
* Remove the given email address from the allow-list. Both as-is and
* lower-case versions of the address will be removed.
*/
void spam_allowlist_remove(opts_t opts, char *address)
{
spam_allowlist__doremove(opts, 0, address);
}
/*
* Manage the allow-list manually (-e).
*/
int spam_allowlist_manage(opts_t opts)
{
int inlist;
spam_t spam;
if (opts->action == ACTION_MARK_SPAM) {
spam_allowlist_remove(opts, opts->emailonly);
spam_allowlist_remove(opts, opts->emailonly2);
return 0;
} else if (opts->action == ACTION_MARK_NONSPAM) {
spam_allowlist_add(opts, opts->emailonly);
spam_allowlist_add(opts, opts->emailonly2);
return 0;
}
spam = calloc(1, sizeof(*spam));
if (spam == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
return 1;
}
spam->db1 = opts->dbr1;
spam->db2 = opts->dbr2;
spam->db3 = opts->dbr3;
spam->dbw = opts->dbw;
spam->db1weight = 1;
spam->db2weight = 1;
spam->db3weight = 1;
inlist = spam_allowlist_match(spam, opts->emailonly);
if (!inlist)
inlist = spam_allowlist_match(spam, opts->emailonly2);
free(spam);
if (inlist) {
if (opts->no_filter)
return 0;
printf("YES\n");
} else {
if (opts->no_filter)
return 1;
printf("NO\n");
}
return 0;
}
/*
* Look up the given email address in the deny-list, returning 1 if it is
* present, 0 if not. First the address is checked in lowercase, then as-is.
*/
int spam_denylist_match(spam_t spam, char *address)
{
return spam_allowlist__domatch(spam, 1, address);
}
/*
* Add the given email address to the deny-list. It is converted to lower
* case before being stored.
*/
void spam_denylist_add(opts_t opts, char *address)
{
spam_allowlist__doadd(opts, 1, address);
}
/*
* Remove the given email address from the deny-list. Both as-is and
* lower-case versions of the address will be removed.
*/
void spam_denylist_remove(opts_t opts, char *address)
{
spam_allowlist__doremove(opts, 1, address);
}
/*
* Manage the deny-list manually (-e).
*/
int spam_denylist_manage(opts_t opts)
{
int inlist;
spam_t spam;
if (opts->action == ACTION_MARK_NONSPAM) {
spam_denylist_remove(opts, opts->emailonly);
spam_denylist_remove(opts, opts->emailonly2);
return 0;
} else if (opts->action == ACTION_MARK_SPAM) {
spam_denylist_add(opts, opts->emailonly);
spam_denylist_add(opts, opts->emailonly2);
return 0;
}
spam = calloc(1, sizeof(*spam));
if (spam == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
return 1;
}
spam->db1 = opts->dbr1;
spam->db2 = opts->dbr2;
spam->db3 = opts->dbr3;
spam->dbw = opts->dbw;
spam->db1weight = 1;
spam->db2weight = 1;
spam->db3weight = 1;
inlist = spam_denylist_match(spam, opts->emailonly);
if (!inlist)
inlist = spam_denylist_match(spam, opts->emailonly2);
free(spam);
if (inlist) {
if (opts->no_filter)
return 0;
printf("YES\n");
} else {
if (opts->no_filter)
return 1;
printf("NO\n");
}
return 0;
}
/* EOF */
qsf-1.2.7/src/message/ 0000755 0000764 0000764 00000000000 10665021416 012336 5 ustar aw aw qsf-1.2.7/src/message/parse.c 0000644 0000764 0000764 00000045010 10664772304 013624 0 ustar aw aw /*
* Functions for parsing and handling mail messages.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "message.h"
#include "md5.h"
#include
#include
#include
#include
extern char *minimemmem(char *, long, char *, long);
/*
* Replace the existing msg->sender, if there is one, with a malloc()ed
* string containing the email address part of the given "From:" header
* line.
*
* Note that we don't follow the RFC here, we just look for the last @ and
* work backwards and forwards from there, since that will catch the vast
* majority of cases and is simplest.
*/
static void msg_parse__fromhdr(opts_t opts, msg_t msg, char *line,
long size)
{
long i, a, b;
char *ptr;
for (i = size - 1; i > 0 && line[i] != '@'; i--) {
}
if (line[i] != '@')
return;
for (a = i; a > 0 && line[a] != '<' && line[a] > 32; a--) {
}
if ((line[a] == '<') || (line[a] <= 32))
a++;
for (b = i; b < size && line[b] != '>' && line[b] > 32; b++) {
}
if ((line[b] == '>') || (line[b] <= 32))
b--;
if (b < (a + 2))
return;
ptr = calloc(1, 2 + b - a);
if (ptr == NULL)
return;
strncpy(ptr, line + a, 1 + b - a);
if (msg->sender)
free(msg->sender);
msg->sender = ptr;
}
/*
* Replace the existing msg->envsender, if there is one, with a malloc()ed
* string containing the email address part of the given "Return-Path:"
* header line. This works the same way as msg_parse__fromhdr() above.
*/
static void msg_parse__returnpathhdr(opts_t opts, msg_t msg, char *line,
long size)
{
long i, a, b;
char *ptr;
for (i = size - 1; i > 0 && line[i] != '@'; i--) {
}
if (line[i] != '@')
return;
for (a = i; a > 0 && line[a] != '<' && line[a] > 32; a--) {
}
if ((line[a] == '<') || (line[a] <= 32))
a++;
for (b = i; b < size && line[b] != '>' && line[b] > 32; b++) {
}
if ((line[b] == '>') || (line[b] <= 32))
b--;
if (b < (a + 2))
return;
ptr = calloc(1, 2 + b - a);
if (ptr == NULL)
return;
strncpy(ptr, line + a, 1 + b - a);
if (msg->envsender)
free(msg->envsender);
msg->envsender = ptr;
}
/*
* Parse a single message header, updating internal state as appropriate.
* Returns nonzero on error.
*/
static int msg_parse__header(opts_t opts, msg_t msg)
{
long start, end, boundstart;
char *newptr;
/*
* Find end of this header line (even if split over
* two lines)
*/
start = msg->_pos;
end = msg->_pos;
while ((end < msg->original_size)
&& (msg->original[end] != '\n')) {
end++;
if (msg->original[end] == '\n'
&& (msg->original[end + 1] == ' '
|| msg->original[end + 1] == '\t')) {
end++;
}
}
if ((start == end)
|| (end == (1 + start) && msg->original[start] == '\r')
) {
/*
* Empty line - end of headers
*/
msg->_in_header--;
} else if (strncasecmp(msg->original + msg->_pos,
"Content-Type:", 13) == 0) {
/*
* Content-Type header - look for type, and
* find boundary= and add a known boundary
* if we find one
*/
msg->_pos += 13;
while ((msg->original[msg->_pos] == ' '
|| msg->original[msg->_pos] == '\t'
|| msg->original[msg->_pos] == '\r'
|| msg->original[msg->_pos] == '\n')
&& msg->_pos < end) {
msg->_pos++;
}
msg->_nottext = 1;
if (strncasecmp(msg->original + msg->_pos, "text/", 5) ==
0) {
msg->_nottext = 0;
} else
if (strncasecmp(msg->original + msg->_pos, "image/", 6)
== 0) {
msg->num_images++;
} else
if (strncasecmp
(msg->original + msg->_pos, "message/", 8) == 0) {
/*
* Handle message/ types by allowing a blank line
* and then continuing to parse header lines, so
* that attached messages with inline parts can be
* processed properly.
*/
msg->_in_header = 2;
msg->_nottext = 0;
}
while (((msg->original[msg->_pos] & 0x60) != 'B')
&& (msg->_pos < end)
&& (strncasecmp(msg->original + msg->_pos,
"boundary=", 9) != 0)) {
msg->_pos++;
}
if ((strncasecmp(msg->original + msg->_pos,
"boundary=", 9) == 0)
&& (msg->_bdepth < 8)) {
msg->_pos += 9;
if (msg->original[msg->_pos] == '"') {
msg->_pos++;
boundstart = msg->_pos;
while ((msg->original[msg->_pos] != '"')
&& (msg->_pos < end))
msg->_pos++;
} else {
boundstart = msg->_pos;
while ((msg->original[msg->_pos] != ';')
&& msg->original[msg->_pos] != ' '
&& msg->original[msg->_pos] != '\t'
&& msg->original[msg->_pos] != '\r'
&& (msg->_pos < end))
msg->_pos++;
}
newptr = malloc(4 + msg->_pos - boundstart);
if (newptr == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("malloc failed"),
strerror(errno));
return 1;
}
memcpy(newptr, "\n--", 3);
strncpy(newptr + 3,
msg->original + boundstart,
msg->_pos - boundstart);
newptr[3 + msg->_pos - boundstart] = 0;
if (msg->_bound[msg->_bdepth] != NULL)
free(msg->_bound[msg->_bdepth]);
msg->_bound[msg->_bdepth] = newptr;
msg->_bdepth++;
}
} else if (strncasecmp(msg->original + msg->_pos,
"Content-Transfer-Encoding:", 26) == 0) {
/*
* Content-Transfer-Encoding header - change
* encoding method to expect
*/
msg->_pos += 26;
while (msg->_pos < end
&& (msg->original[msg->_pos] == ' '
|| msg->original[msg->_pos] == '\t'
|| msg->original[msg->_pos] == '\r'
|| msg->original[msg->_pos] == '\n'))
msg->_pos++;
if (strncasecmp(msg->original + msg->_pos,
"Base64", 6) == 0) {
msg->_encoding = 2;
} else if (strncasecmp(msg->original + msg->_pos,
"Quoted-Printable", 16) == 0) {
msg->_encoding = 1;
} else {
msg->_encoding = 0;
}
} else if ((strncasecmp(msg->original + msg->_pos,
"Subject:", 8) == 0)
|| (strncasecmp(msg->original + msg->_pos,
"From:", 5) == 0)
|| (strncasecmp(msg->original + msg->_pos,
"Return-Path:", 12) == 0)
|| (strncasecmp(msg->original + msg->_pos,
"Sender:", 7) == 0)
|| (strncasecmp(msg->original + msg->_pos,
"To:", 3) == 0)
|| (strncasecmp(msg->original + msg->_pos,
"Reply-To:", 9) == 0)) {
char *decoded;
long decodedlen;
/*
* Add various headers to the message
* content
*/
if (strncasecmp(msg->original + msg->_pos,
"Subject: [SPAM]", 15) == 0) {
msg->_pos += 15;
} else
if (strncasecmp(msg->original + msg->_pos, "From:", 5)
== 0) {
/*
* If it's a From: header, look for an email address
* and put it in msg->sender for later possible use
*/
msg_parse__fromhdr(opts, msg,
msg->original + msg->_pos,
end + 1 - msg->_pos);
} else
if (strncasecmp
(msg->original + msg->_pos, "Return-Path:", 12)
== 0) {
/*
* Also store email address of envelope sender
*/
msg_parse__returnpathhdr(opts, msg,
msg->original + msg->_pos,
end + 1 - msg->_pos);
}
/*
* Decode RFC2047-encoded headers.
*/
decodedlen = end + 1 - msg->_pos;
decoded =
msg_decode_rfc2047(msg->original + msg->_pos,
&decodedlen);
if (msg_addcontent(opts, msg, decoded, decodedlen))
return 1;
free(decoded);
}
msg->_pos = end + 1;
return 0;
}
/*
* Decode the message up to the next boundary and, if it is textual, add its
* contents to msg->content. Returns nonzero on error.
*/
static int msg_parse__content(opts_t opts, msg_t msg)
{
struct MD5Context md5c;
unsigned char digest[16]; /* RATS: ignore (checked all) */
char digeststr[64]; /* RATS: ignore (large enough) */
char *newptr;
char *data;
char *content;
long boundstart; /* start of message part being decoded */
long partsize; /* size of message part being decoded */
int i, n;
boundstart = msg->_pos;
partsize = msg->original_size - boundstart;
msg->_pos = msg->original_size;
/*
* Look for the next boundary
*/
for (i = msg->_bdepth - 1; i >= 0; i--) {
if (msg->_bound[i] == NULL)
continue;
newptr = minimemmem(msg->original + boundstart - 1,
partsize,
msg->_bound[i],
strlen(msg->_bound[i]));
if (newptr == NULL)
continue;
msg->_pos = newptr - msg->original;
partsize = msg->_pos - boundstart;
/*
* Move new position to after the boundary marker
*/
msg->_pos += strlen(msg->_bound[i]);
while ((msg->_pos < msg->original_size)
&& (msg->original[msg->_pos] == '\r'
|| msg->original[msg->_pos] == '\n')
)
msg->_pos++;
/*
* Set nesting depth, and we now scan headers
*/
msg->_bdepth = i + 1;
msg->_in_header = 1;
i = -1;
}
if (partsize < 1 || partsize > 409600) {
if (msg->_in_header) {
msg->_nottext = 0;
msg->_encoding = 0;
}
return 0;
}
switch (msg->_encoding) {
case 1:
data = msg_from_qp(msg->original + boundstart, &partsize);
break;
case 2:
data =
msg_from_base64(msg->original + boundstart, &partsize);
break;
default:
data = NULL;
break;
}
content = data;
if (content == NULL)
content = msg->original + boundstart;
if (msg->_nottext) {
MD5Init(&md5c);
MD5Update(&md5c, (unsigned char *) content, partsize);
MD5Final(digest, &md5c);
memcpy(digeststr, " z", 2);
for (n = 0; n < 16; n++) {
#ifdef HAVE_SNPRINTF
snprintf(digeststr + 2 + 2 * n,
sizeof(digeststr) - 2 - 2 * n,
#else
sprintf(digeststr + 2 + 2 * n, /* RATS: ignore */
#endif
"%02X", digest[n]);
}
memcpy(digeststr + strlen(digeststr), "z \000", 3);
if (msg_addcontent
(opts, msg, digeststr, strlen(digeststr)))
return 1;
} else {
if (msg_addcontent(opts, msg, content, partsize))
return 1;
}
if (data)
free(data);
if (msg->_in_header) {
msg->_nottext = 0;
msg->_encoding = 0;
}
return 0;
}
/*
* Make a copy of the message content in msg->textcontent and strip all HTML
* tags from it. Only HTML tags that start with an alphabetic character or /
* are stripped, but only if the part within the <> is under 500 characters,
* and all HTML comments are stripped from the inclusive.
*
* Returns nonzero on error.
*/
static int msg_parse__striphtml(opts_t opts, msg_t msg)
{
long rpos, wpos, i;
int in_tag, in_comment, ch;
char *ptr;
static struct {
char *string;
int ch;
} entities[] = {
{
"&", '&'}, {
">", '>'}, {
"<", '<'}, {
""", '"'}, {
" ", ' '}, {
"¡", '!'}, {
"¢", 'c'}, {
"£", '£'}, {
"¤", '#'}, {
"¥", 'Y'}, {
"¦", '|'}, {
"§", ' '}, {
"¨", ':'}, {
"©", 'C'}, {
"ª", ' '}, {
"«", '"'}, {
"¬", '!'}, {
"", '-'}, {
"®", 'R'}, {
"¯", ' '}, {
"°", ' '}, {
"±", ' '}, {
"²", '2'}, {
"³", '3'}, {
"´", '\''}, {
"µ", 'u'}, {
"¶", 'P'}, {
"·", '.'}, {
"¸", ' '}, {
"¹", '1'}, {
"º", ' '}, {
"»", '"'}, {
"¼", ' '}, {
"½", ' '}, {
"¾", ' '}, {
"¿", '?'}, {
"À", 'A'}, {
"Á", 'A'}, {
"Â", 'A'}, {
"Ã", 'A'}, {
"Ä", 'A'}, {
"Å", 'A'}, {
"Æ", 'A'}, {
"Ç", 'C'}, {
"È", 'E'}, {
"É", 'E'}, {
"Ê", 'E'}, {
"Ë", 'E'}, {
"Ì", 'I'}, {
"Í", 'I'}, {
"Î", 'I'}, {
"Ï", 'I'}, {
"Ð", 'E'}, {
"Ñ", 'N'}, {
"Ò", 'O'}, {
"Ó", 'O'}, {
"Ô", 'O'}, {
"Õ", 'O'}, {
"Ö", 'O'}, {
"×", 'x'}, {
"Ø", 'O'}, {
"Ù", 'U'}, {
"Ú", 'U'}, {
"Û", 'U'}, {
"Ü", 'U'}, {
"Ý", 'Y'}, {
"Þ", 'T'}, {
"ß", 's'}, {
"à", 'a'}, {
"á", 'a'}, {
"â", 'a'}, {
"ã", 'a'}, {
"ä", 'a'}, {
"å", 'a'}, {
"æ", 'a'}, {
"ç", 'c'}, {
"è", 'e'}, {
"é", 'e'}, {
"ê", 'e'}, {
"ë", 'e'}, {
"ì", 'i'}, {
"í", 'i'}, {
"î", 'i'}, {
"ï", 'i'}, {
"ð", 'e'}, {
"ñ", 'n'}, {
"ò", 'o'}, {
"ó", 'o'}, {
"ô", 'o'}, {
"õ", 'o'}, {
"ö", 'o'}, {
"÷", '/'}, {
"ø", 'o'}, {
"ù", 'u'}, {
"ú", 'u'}, {
"û", 'u'}, {
"ü", 'u'}, {
"ý", 'y'}, {
"þ", 't'}, {
"ÿ", 'y'}, {
0, 0}
};
if (msg->content_size < 1)
return 0;
msg->textcontent = malloc(msg->content_size);
if (msg->textcontent == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
return 1;
}
msg->text_size = 0;
for (rpos = 0, wpos = 0, in_tag = 0, in_comment = 0;
rpos < msg->content_size; rpos++) {
if ((in_tag)
&& (msg->content[rpos] == '>')
) {
in_tag = 0;
continue;
}
if ((in_comment)
&& (rpos < (msg->content_size - 3))
&& (strncmp(msg->content + rpos, "-->", 3) == 0)
) {
in_comment = 0;
rpos += 2;
continue;
}
if (in_tag || in_comment)
continue;
if ((rpos < (msg->content_size - 8))
&& (msg->content[rpos] == '&')
&& (msg->content[rpos + 1] != '#')
) {
for (i = 0; entities[i].string; i++) {
if (strncmp
(msg->content + rpos,
entities[i].string,
strlen(entities[i].string)) == 0) {
msg->textcontent[wpos] =
entities[i].ch;
rpos +=
strlen(entities[i].string) - 1;
wpos++;
msg->text_size = wpos;
break;
}
}
if (entities[i].string)
continue;
}
if ((msg->content[rpos] == '&')
&& (rpos < (msg->content_size - 6))
&& (msg->content[rpos + 1] == '#')
) {
ch = 0;
for (i = 2; (rpos + i < msg->content_size)
&& (msg->content[rpos + i] >= '0')
&& (msg->content[rpos + i] <= '9'); i++) {
ch = ch * 10;
ch += (msg->content[rpos + i] - '0');
}
if ((rpos + i < msg->content_size)
&& (msg->content[rpos + i] == ';')
&& (ch > 0)
) {
msg->textcontent[wpos] = ch;
wpos++;
rpos += i + 1;
msg->text_size = wpos;
continue;
}
}
if ((msg->content[rpos] == '<')
&& (rpos < (msg->content_size - 1))
&& (strchr("ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"abcdefghijklmnopqrstuvwxyz"
"/", msg->content[rpos + 1]))
&& (memchr(msg->content + rpos,
'>', msg->content_size - rpos))
&& ((char *) memchr(msg->content + rpos,
'>', msg->content_size - rpos)
- (msg->content + rpos)
< 500)
) {
in_tag = 1;
continue;
}
if ((msg->content[rpos] == '<')
&& (rpos < (msg->content_size - 5))
&& (msg->content[rpos + 1] == '!')
&& (msg->content[rpos + 2] == '-')
&& (msg->content[rpos + 3] == '-')
) {
in_comment = 1;
continue;
}
msg->textcontent[wpos] = msg->content[rpos];
wpos++;
msg->text_size = wpos;
}
ptr = realloc(msg->textcontent, /* RATS: ignore (not sensitive) */
msg->text_size);
if (ptr != NULL)
msg->textcontent = ptr;
return 0;
}
/*
* Trim the whitespace from msg->textcontent, such that long runs of \r,
* space, tab, \n, etc get truncated to a single space.
*
* Also fill in the wordpos[] and wordlength[] arrays, and count the number
* of words.
*/
static void msg_parse__trimwhitespace(opts_t opts, msg_t msg)
{
long rpos, wpos, words_alloced;
int prevws;
char *ptr;
msg->num_words = 0;
words_alloced = 10000;
msg->wordpos = calloc(words_alloced, sizeof(long));
msg->wordlength = calloc(words_alloced, sizeof(int));
for (rpos = 0, wpos = 0, prevws = 0; rpos < msg->text_size; rpos++) {
if ((msg->textcontent[rpos] == ' ')
|| (msg->textcontent[rpos] == '\r')
|| (msg->textcontent[rpos] == '\n')
|| (msg->textcontent[rpos] == '\t')
) {
if (prevws)
continue;
prevws = 1;
msg->textcontent[wpos++] = ' ';
if (msg->num_words > 0) {
msg->wordlength[msg->num_words - 1] =
(wpos - 1) -
msg->wordpos[msg->num_words - 1];
}
continue;
}
/*
* Non-whitespace. If it follows whitespace, or is the first
* character, add it to the word list.
*/
if ((wpos == 0) || (prevws)) {
/*
* First, make sure there's room in the array; if
* not, extend the array.
*/
if (msg->num_words >= words_alloced - 1) {
long *newwordpos;
int *newwordlength;
words_alloced += 10000;
newwordpos = realloc(msg->wordpos, /* RATS: ignore */
words_alloced *
sizeof(long));
if (newwordpos != NULL) {
msg->wordpos = newwordpos;
newwordlength = realloc(msg->wordlength, /* RATS: ignore */
words_alloced
*
sizeof
(int));
if (newwordlength != NULL) {
msg->wordlength =
newwordlength;
} else {
words_alloced -= 10000;
}
} else {
words_alloced -= 10000;
}
}
/*
* Next, assuming the array can hold another entry
* (the above extension could have failed), add the
* word to the list. The word's default length is
* set to the size of the remaining buffer.
*/
if (msg->num_words < words_alloced - 1) {
msg->wordpos[msg->num_words] = wpos;
msg->wordlength[msg->num_words] =
msg->text_size - rpos;
msg->num_words++;
}
}
prevws = 0;
msg->textcontent[wpos++] = msg->textcontent[rpos];
}
msg->text_size = wpos;
ptr = realloc(msg->textcontent, /* RATS: ignore (not sensitive) */
msg->text_size);
if (ptr != NULL)
msg->textcontent = ptr;
return;
}
/*
* Parse a message on standard input, and return an allocated msg_t, or NULL
* on error. If the message cannot be parsed, eg if it is too big, the
* "content" field will remain NULL but "original" will be allocated and
* will contain the entirety of the message read so far.
*/
msg_t msg_parse(opts_t opts)
{
msg_t msg;
msg = msg_alloc(opts);
if (msg == NULL)
return NULL;
/*
* Read message into memory
*/
if (msg_read(opts, msg))
return msg;
/*
* Store original message headers
*/
if (msg_headers_store(opts, msg))
return msg;
msg->_in_header = 1;
msg->_pos = 0;
/*
* Scan through the message, finding Content-Type and
* Content-Transfer-Encoding headers to get message boundaries, and
* split the body in these boundaries (and then scan each part's
* headers, if any, for more boundaries and content types and so
* on); selected headers are added to msg->content, as are all
* textual parts of the message body (after decoding).
*/
while (msg->_pos < msg->original_size) {
if (msg->_in_header > 0) {
if (msg_parse__header(opts, msg))
return msg;
} else {
if (msg_parse__content(opts, msg))
return msg;
}
}
msg_parse__striphtml(opts, msg);
msg_parse__trimwhitespace(opts, msg);
return msg;
}
/* EOF */
qsf-1.2.7/src/message/alloc.c 0000644 0000764 0000764 00000003712 10664772304 013607 0 ustar aw aw /*
* Functions for allocating and freeing memory.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "message.h"
#include
#include
#include
#include
/*
* Add the content of the given length to the "content" field of the message.
*
* Returns nonzero on error.
*/
int msg_addcontent(opts_t opts, msg_t msg, char *data, long size)
{
char *newptr;
if ((msg->content_size + size + 16) > msg->content_alloced) {
newptr = realloc(msg->content, /* RATS: ignore */
msg->content_alloced + size + 8192);
if (newptr == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("realloc failed"), strerror(errno));
return 1;
}
msg->content = newptr;
msg->content_alloced += size + 8192;
}
memcpy(msg->content + msg->content_size, data, size);
msg->content[msg->content_size + size] = 0;
msg->content_size += size;
return 0;
}
/*
* Allocate a new msg_t and return it, or NULL on error.
*/
msg_t msg_alloc(opts_t opts)
{
msg_t msg;
msg = calloc(1, sizeof(*msg));
if (msg == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
return NULL;
}
return msg;
}
/*
* Free the memory a message takes up.
*/
void msg_free(msg_t msg)
{
int i;
if (msg == NULL)
return;
if (msg->original != NULL)
free(msg->original);
if (msg->content != NULL)
free(msg->content);
if (msg->textcontent != NULL)
free(msg->textcontent);
if (msg->sender != NULL)
free(msg->sender);
if (msg->envsender != NULL)
free(msg->envsender);
if (msg->wordpos != NULL)
free(msg->wordpos);
if (msg->wordlength != NULL)
free(msg->wordlength);
for (i = 0; i < msg->num_headers; i++) {
if (msg->header[i] != NULL)
free(msg->header[i]);
}
for (i = 0; i < 8; i++) {
if (msg->_bound[i] != NULL)
free(msg->_bound[i]);
}
if (msg->header != NULL)
free(msg->header);
free(msg);
}
/* EOF */
qsf-1.2.7/src/message/read.c 0000644 0000764 0000764 00000003443 10664772304 013431 0 ustar aw aw /*
* Functions for parsing and handling mail messages.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "message.h"
#include
#include
#include
#include
/*
* Read a message into memory (msg->original) from either stdin or an
* existing buffer (opts->inbuf), aborting if it gets too big. Returns
* nonzero on abort, zero on success.
*/
int msg_read(opts_t opts, msg_t msg)
{
char buffer[1024]; /* RATS: ignore (checked all) */
char *newptr;
long got;
/*
* Just copy inbuf if it's set
*/
if (opts->inbuf != NULL) {
msg->original = malloc(opts->inbufsize);
if (msg->original == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("malloc failed"), strerror(errno));
return 1;
}
memcpy(msg->original, opts->inbuf, opts->inbufsize);
msg->original_size = opts->inbufsize;
return 0;
}
/*
* Read the entire message into memory from stdin, aborting if it
* gets too big
*/
while (!feof(stdin)) {
got = fread(buffer, 1, sizeof(buffer), stdin);
if (got < 0) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("error reading message"),
strerror(errno));
return 1;
}
if (got == 0)
break;
if (got > sizeof(buffer))
got = sizeof(buffer);
newptr = realloc( /* RATS: ignore (not sensitive) */
msg->original,
msg->original_size + got + 64);
if (newptr == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("realloc failed"), strerror(errno));
return 1;
}
memcpy(newptr + msg->original_size, buffer, got);
newptr[msg->original_size + got] = 0;
msg->original = newptr;
msg->original_size += got;
if (msg->original_size >= MAX_MESSAGE_SIZE)
return 1;
}
return 0;
}
/* EOF */
qsf-1.2.7/src/message/rfc2047.c 0000644 0000764 0000764 00000016127 10664772304 013610 0 ustar aw aw /*
* Decode RFC2047-encoded strings.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "message.h"
#include
#include
/*
* Table for decoding hex digits
*/
static char index_hex[256] = {
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, -1, -1, -1, -1, -1, -1,
-1, 10, 11, 12, 13, 14, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, 10, 11, 12, 13, 14, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
};
#define CHARHEX(c) (index_hex[(unsigned char)(c)])
/*
* Take the given single RFC2047 encoded word and store it, decoded, into
* the given buffer with the given maximum length.
*
* It it assumed that "str" contains a valid RFC2047 encoded word, as found
* by the findencoded function below.
*/
static void decode_rfc2047_word(char *str, char *buf, long bufsize)
{
int seenq, encoding;
char *charset;
char *rpos;
char *nextq;
seenq = 0;
charset = NULL;
encoding = 0;
buf[0] = 0;
for (rpos = str; (nextq = strchr(rpos, '?')); rpos = nextq + 1) {
char *ptr;
int n;
seenq++;
switch (seenq) {
case 2: /* CHARSET part */
n = nextq - rpos;
/*
* Since RFC2231 says CHARSET can be of the form
* CHARSET*LANGUAGE, we need to throw away the
* LANGUAGE part if an asterisk is present.
*/
ptr = memchr(rpos, '*', n);
if (ptr)
n = ptr - rpos;
charset = malloc(n + 1);
if (charset != NULL) {
memcpy(charset, rpos, n);
charset[n] = 0;
}
break;
case 3: /* ENC part */
switch (rpos[0]) {
case 'Q':
case 'q':
encoding = 'Q';
break;
case 'B':
case 'b':
encoding = 'B';
break;
default:
if (charset)
free(charset);
return;
}
break;
case 4: /* DATA part */
if (encoding == 'Q') {
/* Quoted-Printable decoding */
while ((rpos < nextq) && (bufsize > 0)) {
if (rpos[0] == '_') {
buf[0] = ' ';
buf++;
bufsize--;
} else if (rpos[0] == '=') {
if (rpos[1] == 0)
break;
if (rpos[2] == 0)
break;
buf[0] =
(CHARHEX(rpos[1]) << 4)
| CHARHEX(rpos[2]);
buf++;
bufsize--;
rpos += 2;
} else {
buf[0] = rpos[0];
buf++;
bufsize--;
}
rpos++;
}
buf[0] = 0;
} else if (encoding == 'B') {
/* Base64 decoding */
char *decbuf;
long size;
size = nextq - rpos;
decbuf = msg_from_base64(rpos, &size);
if (decbuf == NULL) {
if (charset)
free(charset);
return;
}
if (size > bufsize)
size = bufsize;
memcpy(buf, decbuf, size);
free(decbuf);
buf += size;
bufsize -= size;
buf[0] = 0;
}
break;
}
}
if (charset) {
/*
* We don't do anything with the charset information, as
* converting between character sets would make this project
* a lot more complicated than it really needs to be.
*/
free(charset);
}
}
/*
* Find the next RFC2047 encoded word in the given string, assuming that the
* encoding must be B or Q (case insensitive, as per the RFC).
*
* An RFC2047 encoded word looks like this: =?CHARSET?ENC?DATA?=
*
* CHARSET is a character set specifier, ENC is the encoding (Q for
* quoted-printable, B for base64), and DATA is the encoded data.
*
* Returns a pointer to the start of the encoded word and fills in *endptr
* with a pointer to the end of the encoded word, or returns NULL if nothing
* was found.
*/
static char *decode_rfc2047_findencoded(char *str, char **endptr)
{
char *start;
char *end;
for (end = str; (start = strstr(end, "=?"));) {
/*
* Look for the next ? at the end of the CHARSET specifier;
* CHARSET cannot contain "forbidden" characters.
*/
for (end = start + 2; (end[0] > 32)
&& (end[0] < 127)
&& (strchr("()<>@,;:\"/[].=?", end[0]) == NULL);
end++) {
}
/*
* Check we've found the ?ENC? part, where ENC is B or Q
* (not case sensitive).
*/
if (end[0] != '?')
continue;
if (strchr("BQbq", end[1]) == NULL)
continue;
if (end[2] != '?')
continue;
/*
* Skip the DATA part.
*/
for (end = end + 3; (end[0] > 32)
&& (end[0] < 127)
&& (end[0] != '?'); end++) {
}
/*
* Check that the encoded word ends with ?= as it should.
*/
if ((end[0] != '?') || (end[1] != '=')) {
end--;
continue;
}
end += 2;
*endptr = end;
return start;
}
return NULL;
}
/*
* Return a malloc()ed string containing the input string with any RFC2047
* encoded content decoded. The content of "len" is updated to contain the
* size of the output string, and on entry should contain the size of the
* input string.
*
* Returns NULL on error.
*/
char *msg_decode_rfc2047(char *str, long *len)
{
char *in;
char *out;
char *inptr;
char *outptr;
long bytesleft;
int enccount;
if (str == NULL)
return NULL;
if (len == NULL)
return NULL;
if (*len < 1)
return NULL;
if (str[0] == 0)
return NULL;
bytesleft = *len;
in = malloc(bytesleft + 1);
if (in == NULL)
return NULL;
out = malloc(bytesleft + 1);
if (out == NULL) {
free(in);
return NULL;
}
memcpy(in, str, bytesleft);
in[bytesleft] = 0;
inptr = in;
outptr = out;
enccount = 0;
while ((inptr[0] != 0) && (bytesleft > 0)) {
char *start;
char *end;
int n;
/*
* Find the next RFC2047 encoded word.
*/
start = decode_rfc2047_findencoded(inptr, &end);
/*
* No encoded word found - copy the remainder of the string
* to the output and exit the loop.
*/
if (start == NULL) {
strncpy(outptr, inptr, bytesleft);
outptr += bytesleft;
break;
}
/*
* Copy across parts of the string before the encoded word
* to the output. However, we ignore whitespace between
* encoded words if they are all that's there (i.e. we treat
* "ENCWORD ENCWORD" as "ENCWORDENCWORD", but treat "ENCWORD
* foo ENCWORD" as "ENCWORD foo ENCWORD".
*/
if (start != inptr) {
n = start - inptr;
if ((enccount == 0)
|| (strspn(inptr, " \t\r\n") != n)
) {
if (n > bytesleft)
n = bytesleft;
memcpy(outptr, inptr, n);
outptr += n;
bytesleft -= n;
}
}
decode_rfc2047_word(start, outptr, bytesleft);
enccount++;
bytesleft -= (1 + end - start);
inptr = end;
n = strlen(outptr);
outptr += n;
}
outptr[0] = 0;
*len = strlen(out);
free(in);
return out;
}
/* EOF */
qsf-1.2.7/src/message/header.c 0000644 0000764 0000764 00000013564 10664772304 013753 0 ustar aw aw /*
* Functions for modifying mail message headers.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "message.h"
#include
#include
#include
#include
/*
* Store a copy of each header in the message, and store a pointer to the
* start of the message body for later use by msg_dump(). Returns nonzero on
* error.
*/
int msg_headers_store(opts_t opts, msg_t msg)
{
long start, end;
char *newptr;
start = 0;
end = 1;
do {
end = start;
while ((end < msg->original_size)
&& (msg->original[end] != '\n')) {
end++;
}
if (start == end)
break;
newptr = realloc(msg->header, /* RATS: ignore (OK) */
sizeof(char *) * (msg->num_headers + 1));
if (newptr == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("realloc failed"), strerror(errno));
return 1;
}
msg->header = (char **) newptr;
newptr = malloc(1 + end - start);
if (newptr == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("malloc failed"), strerror(errno));
return 1;
}
msg->header[msg->num_headers] = newptr;
memcpy(newptr, msg->original + start, end - start);
newptr[end - start] = 0;
msg->num_headers++;
start = end + 1;
} while ((start != end) && (start < msg->original_size));
msg->body = msg->original + end + 1;
msg->body_size = msg->original_size - (end + 1);
return 0;
}
/*
* Modify the message's Subject line to mark it as spam, using the given
* string as a marker or "[SPAM]" if NULL is supplied.
*/
void msg_spamsubject(msg_t msg, char *marker)
{
char *ptr;
int gotsubject = 0;
int i, sz;
if (marker == NULL)
marker = "[SPAM]";
for (i = 0; i < msg->num_headers; i++) {
if (msg->header[i] == NULL)
continue;
if (strncasecmp(msg->header[i], "Subject:", 8) == 0) {
gotsubject = 1;
if ((msg->header[i][8] == ' ')
&&
(strncasecmp
(msg->header[i] + 9, marker,
strlen(marker)) == 0))
continue;
sz = strlen(msg->header[i]) + strlen(marker) + 4;
ptr = malloc(sz);
if (ptr == NULL)
continue;
#ifdef HAVE_SNPRINTF
snprintf(ptr, sz,
#else
sprintf(ptr, /* RATS: ignore (checked) */
#endif
"%.8s %s%s", msg->header[i], marker,
msg->header[i] + 8);
free(msg->header[i]);
msg->header[i] = ptr;
}
}
if (gotsubject == 1)
return;
/*
* Add a Subject header if there wasn't one
*/
ptr = realloc(msg->header, /* RATS: ignore (OK) */
sizeof(char *) * (msg->num_headers + 1));
if (ptr == NULL)
return;
msg->header = (char **) ptr;
sz = strlen(marker) + 12;
ptr = malloc(sz);
if (ptr == NULL)
return;
#ifdef HAVE_SNPRINTF
snprintf(ptr, sz,
#else
sprintf(ptr, /* RATS: ignore (checked) */
#endif
"Subject: %.*s", (int) (strlen(marker) + 1), marker);
msg->header[msg->num_headers] = ptr;
msg->num_headers++;
}
/*
* Add an X-Spam: header to the message, YES (or header marker) if
* "spamscore" >0, NO if 0.
*/
void msg_spamheader(msg_t msg, char *marker, double spamscore)
{
char *ptr;
int i, sz;
if (marker == NULL)
marker = "YES";
for (i = 0; i < msg->num_headers; i++) {
if (msg->header[i] == NULL)
continue;
if (strncasecmp(msg->header[i], "X-Spam:", 7) == 0) {
free(msg->header[i]);
msg->header[i] = NULL;
}
}
ptr = realloc(msg->header, /* RATS: ignore (OK) */
sizeof(char *) * (msg->num_headers + 1));
if (ptr == NULL)
return;
msg->header = (char **) ptr;
if (spamscore > 0) {
sz = strlen(marker) + 9;
ptr = malloc(sz);
if (ptr == NULL)
return;
#ifdef HAVE_SNPRINTF
snprintf(ptr, sz,
#else
sprintf(ptr, /* RATS: ignore (checked) */
#endif
"X-Spam: %.*s", (int) (strlen(marker) + 1),
marker);
} else {
ptr = strdup("X-Spam: NO");
if (ptr == NULL)
return;
}
msg->header[msg->num_headers] = ptr;
msg->num_headers++;
}
/*
* Add an X-Spam-Rating: header to the message, giving the spam score as a
* decimal percentage from 0 to 100.
*/
void msg_spamratingheader(msg_t msg, double spamscore, double threshold)
{
char buf[256]; /* RATS: ignore (OK) */
double scaledscore;
char *ptr;
int i;
for (i = 0; i < msg->num_headers; i++) {
if (msg->header[i] == NULL)
continue;
if (strncasecmp(msg->header[i], "X-Spam-Rating:", 14) == 0) {
free(msg->header[i]);
msg->header[i] = NULL;
}
}
ptr = realloc(msg->header, /* RATS: ignore (OK) */
sizeof(char *) * (msg->num_headers + 1));
if (ptr == NULL)
return;
msg->header = (char **) ptr;
if (spamscore < 0)
spamscore += 0.01;
spamscore += threshold;
scaledscore = spamscore * 100.0;
#ifdef HAVE_SNPRINTF
snprintf(buf, sizeof(buf) - 1,
"X-Spam-Rating: %d", (int) scaledscore);
#else
sprintf(buf, /* RATS: ignore (OK) */ "X-Spam-Rating: %d",
(int) scaledscore);
#endif
ptr = strdup(buf);
if (ptr == NULL)
return;
msg->header[msg->num_headers] = ptr;
msg->num_headers++;
}
/*
* Add an X-Spam-Level: header to the message, giving the spam score as a
* number of stars from 0 to 20.
*/
void msg_spamlevelheader(msg_t msg, double spamscore, double threshold)
{
char buf[256]; /* RATS: ignore (OK) */
double scaledscore;
char *ptr;
int i, x;
for (i = 0; i < msg->num_headers; i++) {
if (msg->header[i] == NULL)
continue;
if (strncasecmp(msg->header[i], "X-Spam-Level:", 13) == 0) {
free(msg->header[i]);
msg->header[i] = NULL;
}
}
ptr = realloc(msg->header, /* RATS: ignore (OK) */
sizeof(char *) * (msg->num_headers + 1));
if (ptr == NULL)
return;
msg->header = (char **) ptr;
if (spamscore < 0)
spamscore += 0.01;
spamscore += threshold;
scaledscore = spamscore * 100.0;
memcpy(buf, "X-Spam-Level: ", 14);
x = strlen(buf);
for (i = 0; (i < (scaledscore * 0.2)) && (i < 20); i++) {
buf[x] = '*';
x++;
buf[x] = 0;
}
ptr = strdup(buf);
if (ptr == NULL)
return;
msg->header[msg->num_headers] = ptr;
msg->num_headers++;
}
/* EOF */
qsf-1.2.7/src/message/qp.c 0000644 0000764 0000764 00000004470 10664772304 013137 0 ustar aw aw /*
* Decode a block of Quoted-Printable encoded data.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include
#define XX 127
/*
* Table for decoding hexadecimal in quoted-printable
*/
static char index_hex[256] = { /* RATS: ignore (OK) */
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, XX, XX, XX, XX, XX, XX,
XX, 10, 11, 12, 13, 14, 15, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, 10, 11, 12, 13, 14, 15, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX, XX,
};
#define HEXCHAR(c) (index_hex[(unsigned char)(c)])
/*
* Decode a block of Quoted-Printable-encoded data and return a pointer to
* the decoded data, which will have been malloc()ed, or NULL on error. The
* content of "size" is updated to contain the size of the output buffer,
* and on entry should contain the size of the input block.
*/
char *msg_from_qp(char *data, long *size)
{
char *out;
long inpos, outpos;
int byte, c, c1 = 0, c2 = 0;
out = malloc(64 + *size);
if (out == NULL)
return NULL;
for (inpos = 0, outpos = 0, byte = 0; inpos < *size; inpos++) {
c = data[inpos];
switch (byte) {
case 0:
if (c == '=') {
byte++;
} else {
out[outpos++] = c;
}
break;
case 1:
c1 = HEXCHAR(c);
if (c1 == XX) {
byte = 0;
} else {
byte++;
}
break;
case 2:
c2 = HEXCHAR(c);
if (c2 != XX) {
out[outpos++] = c1 << 4 | c2;
}
byte = 0;
break;
default:
break;
}
}
out[outpos] = 0;
*size = outpos;
return out;
}
/* EOF */
qsf-1.2.7/src/message/dump.c 0000644 0000764 0000764 00000002345 10664772304 013463 0 ustar aw aw /*
* Dump a mail message to stdout.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "message.h"
#include "log.h"
#include
#include
/*
* Dump the given message on standard output. If the "content" field is
* null, the "original" field is just dumped; otherwise, the message is
* reconstructed from the "header" array and the "body" field.
*/
void msg_dump(msg_t msg)
{
long pos, sent;
int i;
if (msg == NULL)
return;
if (msg->content == NULL) {
pos = 0;
while (pos < msg->original_size) {
sent = fwrite(msg->original + pos,
1, msg->original_size - pos, stdout);
if (sent <= 0)
break;
pos += sent;
}
return;
}
for (i = 0; i < msg->num_headers; i++) {
if (msg->header[i] == NULL)
continue;
sent =
fwrite(msg->header[i], strlen(msg->header[i]), 1,
stdout);
if (sent <= 0)
break;
sent = fwrite("\n", 1, 1, stdout);
if (sent <= 0)
break;
}
log_dump("X-QSF-Info: ");
if (fwrite("\n", 1, 1, stdout) <= 0)
return;
pos = 0;
while (pos < msg->body_size) {
sent =
fwrite(msg->body + pos, 1, msg->body_size - pos,
stdout);
if (sent <= 0)
break;
pos += sent;
}
}
/* EOF */
qsf-1.2.7/src/message/base64.c 0000644 0000764 0000764 00000005447 10664772304 013610 0 ustar aw aw /*
* Decode Base64-encoded data.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include
/*
* Table for decoding base64
*/
static char index_64[256] = { /* RATS: ignore (OK) */
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 62, -1, -1, -1, 63,
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, -1, -1, -1, -1, -1, -1,
-1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, -1, -1, -1, -1, -1,
-1, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
};
#define CHAR64(c) (index_64[(unsigned char)(c)])
/*
* Decode a block of Base64-encoded data and return a pointer to the decoded
* data, which will have been malloc()ed, or NULL on error. The content of
* "size" is updated to contain the size of the output buffer, and on entry
* should contain the size of the input block.
*/
char *msg_from_base64(char *data, long *size)
{
char *out;
long inpos, outpos;
int byte, c, c1 = 0, c2 = 0, c3 = 0, c4 = 0;
char buf[3]; /* RATS: ignore (checked all) */
out = malloc(64 + *size);
if (out == NULL)
return NULL;
for (inpos = 0, outpos = 0, byte = 0; inpos < *size; inpos++) {
c = data[inpos];
switch (byte) {
case 0:
if (c != '=' && CHAR64(c) == -1)
continue;
c1 = c;
byte++;
break;
case 1:
if (c != '=' && CHAR64(c) == -1)
continue;
c2 = c;
byte++;
break;
case 2:
if (c != '=' && CHAR64(c) == -1)
continue;
c3 = c;
byte++;
break;
case 3:
if (c != '=' && CHAR64(c) == -1)
continue;
c4 = c;
byte++;
break;
default:
break;
}
if (byte < 4)
continue;
byte = 0;
if (c1 == '=' || c2 == '=')
break;
c1 = CHAR64(c1);
c2 = CHAR64(c2);
buf[0] = ((c1 << 2) | ((c2 & 0x30) >> 4));
out[outpos++] = buf[0];
if (c3 == '=')
break;
c3 = CHAR64(c3);
buf[1] = (((c2 & 0x0F) << 4) | ((c3 & 0x3C) >> 2));
out[outpos++] = buf[1];
if (c4 == '=')
break;
c4 = CHAR64(c4);
buf[2] = (((c3 & 0x03) << 6) | c4);
out[outpos++] = buf[2];
}
out[outpos] = 0;
*size = outpos;
return out;
}
/* EOF */
qsf-1.2.7/src/tests/ 0000755 0000764 0000764 00000000000 10665021416 012054 5 ustar aw aw qsf-1.2.7/src/tests/gtube.c 0000644 0000764 0000764 00000001267 10664772304 013344 0 ustar aw aw /*
* Implement GTUBE, the Generic Test for Unsolicited Bulk Email.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "testi.h"
#include
#define GTUBE "XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X"
extern char *minimemmem(char *, long, char *, long);
/*
* Override the spam filter and always mark a message as spam if it contains
* the GTUBE string. This can be used to check that the spam filter is
* working.
*/
int spam_test_gtube(opts_t opts, msg_t msg, spam_t spam)
{
if (minimemmem
(msg->content, msg->content_size, GTUBE, strlen(GTUBE)) == 0)
return 0;
return 1;
}
/* EOF */
qsf-1.2.7/src/tests/main.c 0000644 0000764 0000764 00000011064 10664772304 013156 0 ustar aw aw /*
* Main entry point for calling all special tests.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "testi.h"
#include
#include
/*
* Prototype all tests here, and list them in the spam_test() function
* below.
*/
int spam_test_gtube(opts_t, msg_t, spam_t);
int spam_test_attachment_scr(opts_t, msg_t, spam_t);
int spam_test_attachment_pif(opts_t, msg_t, spam_t);
int spam_test_attachment_exe(opts_t, msg_t, spam_t);
int spam_test_attachment_vbs(opts_t, msg_t, spam_t);
int spam_test_attachment_vba(opts_t, msg_t, spam_t);
int spam_test_attachment_lnk(opts_t, msg_t, spam_t);
int spam_test_attachment_com(opts_t, msg_t, spam_t);
int spam_test_attachment_bat(opts_t, msg_t, spam_t);
int spam_test_attachment_pdf(opts_t, msg_t, spam_t);
int spam_test_attachment_doc(opts_t, msg_t, spam_t);
int spam_test_attachment_xls(opts_t, msg_t, spam_t);
int spam_test_attachment_jpg(opts_t, msg_t, spam_t);
int spam_test_attachment_gif(opts_t, msg_t, spam_t);
int spam_test_attachment_png(opts_t, msg_t, spam_t);
int spam_test_gibberish_consonants(opts_t, msg_t, spam_t);
int spam_test_gibberish_vowels(opts_t, msg_t, spam_t);
int spam_test_gibberish_from_consonants(opts_t, msg_t, spam_t);
int spam_test_gibberish_from_vowels(opts_t, msg_t, spam_t);
int spam_test_gibberish_badstart(opts_t, msg_t, spam_t);
int spam_test_gibberish_hyphens(opts_t, msg_t, spam_t);
int spam_test_gibberish_longwords(opts_t, msg_t, spam_t);
int spam_test_html_comments_in_words(opts_t, msg_t, spam_t);
int spam_test_html_external_img(opts_t, msg_t, spam_t);
int spam_test_html_font(opts_t, msg_t, spam_t);
int spam_test_html_urls(opts_t, msg_t, spam_t);
int spam_test_image_single(opts_t, msg_t, spam_t);
int spam_test_image_multiple(opts_t, msg_t, spam_t);
/*
* Run each test in turn, such that if a test triggers, its special token is
* added to the token list or, if the test says to override the message's
* spam/nonspam value, do that.
*
* Returns 0 normally, nonzero if testing is to be overridden: -1 means to
* always mark the message as non-spam, 1 means to always mark it as spam.
*
* Test functions should return 0 for no action, -1 to override all other
* tests and mark as non-spam, 1 to override all tests and mark as spam, and
* 1+n to continue as usual but add that test's token to the token tree "n"
* times (where "n" is 1 or more).
*
* Tokens should start with "." to distinguish themselves from real tokens,
* which can never start with ".", and should end with "." for cosmetic
* reasons. Tokens should never contain any characters not in TOKEN_CHARS
* (see include/spam.h), and must NEVER be longer than 32 characters.
*/
int spam_test(opts_t opts, spam_t spam, msg_t msg)
{
struct {
char *token;
spamtestfunc_t func;
} test[] = {
{
".GTUBE.", spam_test_gtube}, {
".ATTACH-SCR.", spam_test_attachment_scr}, {
".ATTACH-PIF.", spam_test_attachment_pif}, {
".ATTACH-EXE.", spam_test_attachment_exe}, {
".ATTACH-VBS.", spam_test_attachment_vbs}, {
".ATTACH-VBA.", spam_test_attachment_vba}, {
".ATTACH-LNK.", spam_test_attachment_lnk}, {
".ATTACH-COM.", spam_test_attachment_com}, {
".ATTACH-BAT.", spam_test_attachment_bat}, {
".ATTACH-PDF.", spam_test_attachment_pdf}, {
".ATTACH-DOC.", spam_test_attachment_doc}, {
".ATTACH-XLS.", spam_test_attachment_xls}, {
".ATTACH-JPG.", spam_test_attachment_jpg}, {
".ATTACH-GIF.", spam_test_attachment_gif}, {
".ATTACH-PNG.", spam_test_attachment_png}, {
".GIBBERISH-CONSONANTS.", spam_test_gibberish_consonants},
{
".GIBBERISH-VOWELS.", spam_test_gibberish_vowels}, {
".GIBBERISH-FROMCONS.",
spam_test_gibberish_from_consonants}, {
".GIBBERISH-FROMVOWL.", spam_test_gibberish_from_vowels},
{
".GIBBERISH-BADSTART.", spam_test_gibberish_badstart},
{
".GIBBERISH-HYPHENS.", spam_test_gibberish_hyphens}, {
".GIBBERISH-LONGWORDS.", spam_test_gibberish_longwords},
{
".HTML-COMMENTS-IN-WORDS.",
spam_test_html_comments_in_words}, {
".HTML-EXTERNAL-IMG.", spam_test_html_external_img}, {
".HTML-FONT.", spam_test_html_font}, {
".HTML-IP-IN-URLS.", spam_test_html_urls}, {
".SINGLE-IMAGE.", spam_test_image_single}, {
".MULTIPLE-IMAGES.", spam_test_image_multiple}, {
NULL, NULL}
};
int i, ret, n;
for (i = 0; test[i].func != NULL; i++) {
ret = (test[i].func) (opts, msg, spam);
switch (ret) {
case -1:
return -1;
case 0:
break;
case 1:
return 1;
default:
for (n = 1; n < ret; n++) {
spam_token_add(opts, spam, test[i].token,
strlen(test[i].token));
}
break;
}
}
return 0;
}
/* EOF */
qsf-1.2.7/src/tests/imgcount.c 0000644 0000764 0000764 00000001120 10664772304 014047 0 ustar aw aw /*
* Rules which add tokens depending on how many images are attached.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "testi.h"
/*
* Add a token if the message contains exactly one attached image.
*/
int spam_test_image_single(opts_t opts, msg_t msg, spam_t spam)
{
if (msg->num_images == 1)
return 2;
return 0;
}
/*
* Add a token if the message contains more than one attached image.
*/
int spam_test_image_multiple(opts_t opts, msg_t msg, spam_t spam)
{
if (msg->num_images > 1)
return 2;
return 0;
}
/* EOF */
qsf-1.2.7/src/tests/urls.c 0000644 0000764 0000764 00000006711 10664772304 013222 0 ustar aw aw /*
* Rules to look for URLs in messages.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "testi.h"
#include "spam.h"
#include
extern char *minimemmem(char *, long, char *, long);
/*
* Add a token for every URL found containing an IP address, and also add
* all URLs found and their hostnames as tokens in their own right.
*/
int spam_test_html_urls(opts_t opts, msg_t msg, spam_t spam)
{
int nfound, ldr, ldrl, isip, dots, isint, isurlenc;
long pos, len, i, hoststart, hostend;
char *leaders[] = {
"http:",
"Http:",
"HTTP:",
"ftp:",
"Ftp:",
"FTP:",
"mailto:",
"Mailto:",
"MAILTO:",
NULL
};
char *ptr;
for (nfound = 0, ldr = 0; leaders[ldr]; ldr++) {
ldrl = strlen(leaders[ldr]); /* RATS: ignore (OK) */
for (pos = 0; pos < msg->content_size;) {
ptr =
minimemmem(msg->content + pos,
msg->content_size - pos,
leaders[ldr], ldrl);
if (!ptr)
break;
pos = ptr - msg->content;
if (pos < 0)
break;
/*
* Find the length of the URL (i.e. where it ends),
* counting ? as an end-of-URL character
*/
for (len = 0; (len < msg->content_size - pos)
&& !strchr("?>\"' \n\t\r",
msg->content[pos + len]); len++) {
}
/*
* Find the start of the hostname (after the // part)
*/
hoststart = 5;
while ((hoststart < len)
&& (msg->content[pos + hoststart] == '/')) {
hoststart++;
}
/*
* If the URL has a @ in it, the hostname comes
* after that
*/
for (i = 0; i < len; i++) {
if (msg->content[pos + i] == '@')
hoststart = i + 1;
}
/*
* Don't go past the end of the URL
*/
if (hoststart >= len)
hoststart = 0;
/*
* Find the first / after the hostname, if any
*/
hostend = 0;
for (i = hoststart; i < len; i++) {
if (msg->content[pos + i] == '/') {
hostend = i;
break;
}
}
/*
* Add the URL as a token
*/
spam_token_add(opts, spam, msg->content + pos,
len);
/*
* Add the part of the URL that starts with the
* hostname (i.e. after http://, and any @) as a
* token
*/
if (hoststart > 0) {
spam_token_add(opts, spam, msg->content +
pos + hoststart,
len - hoststart);
/*
* Add the hostname on its own as a token
*/
if (hostend > hoststart) {
spam_token_add(opts, spam,
msg->content + pos +
hoststart,
hostend -
hoststart);
}
}
/*
* See whether the URL's hostname is an IP address,
* just an integer, or might be URL-encoded
*/
isint = 1;
isip = 1;
isurlenc = 0;
dots = 0;
for (i = hoststart; i < len; i++) {
if ((msg->content[pos + i] >= '0')
&& (msg->content[pos + i] <= '9'))
continue;
if (msg->content[pos + i] == '.') {
dots++;
} else if (msg->content[pos + i] == '/') {
if (dots < 3)
isip = 0;
break;
} else {
isip = 0;
isint = 0;
if (msg->content[pos + i] == '%')
isurlenc = 1;
}
}
if (dots < 3)
isip = 0;
if (dots > 0)
isint = 0;
if (isip) {
nfound++;
} else if (isint) {
spam_token_add(opts, spam,
".HTML-INT-IN-URL.", 18);
} else if (isurlenc) {
spam_token_add(opts, spam,
".HTML-URLENCODED-URL.",
21);
}
pos++;
}
}
if (nfound > 0)
return nfound + 1;
return 0;
}
/* EOF */
qsf-1.2.7/src/tests/gibberish.c 0000644 0000764 0000764 00000011526 10664772304 014173 0 ustar aw aw /*
* Rules which look for gibberish - words containing too many consonants or
* vowels in a row.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "testi.h"
#include
#define CONSONANTS "BCDFGHJKLMNPQRSTVWXZbcdfghjklmnpqrstvwxz" \
"ÇÐÑÞßçðñþ"
#define VOWELS "AEIOUYaeiouy" \
"ÀÁÂÃÄÅÆÈÉÊËÌÍÎÏÒÓÔÕÖØÙÚÛÜÝ" \
"àáâãäåæèéêëìíîïòóôõöøùúûüý\0377"
/*
* Return nonzero if a sequence of "count" or more characters from the
* string "accept" are found in the given word, so long as the word is
* shorter than 60 characters (so we don't trip up on base64 encoded stuff).
*/
static int spam_test_gibberish__runon(char *word, int len, char *accept,
int count)
{
int pos = 0;
int n;
if (len >= 60)
return 0;
while (pos < len) {
/*
* Skip any characters not in the accept string.
*/
while (pos < len) {
if (strchr(accept, word[pos]) != NULL)
break;
pos++;
}
if (pos >= len)
return 0;
/*
* Count the number of allowable characters in a row.
*/
n = 0;
while (pos < len) {
if (strchr(accept, word[pos]) == NULL)
break;
pos++;
n++;
}
if (n >= count)
return 1;
}
return 0;
}
/*
* Add a token for every word found that contains more than 5 consonants
* (not including "y") in a row.
*/
int spam_test_gibberish_consonants(opts_t opts, msg_t msg, spam_t spam)
{
int nfound = 0;
long n;
char *word;
int len;
for (n = 0; n < msg->num_words; n++) {
word = msg->textcontent + msg->wordpos[n];
len = msg->wordlength[n];
if (spam_test_gibberish__runon(word, len, CONSONANTS, 5))
nfound++;
}
if (nfound > 0)
return nfound + 1;
return 0;
}
/*
* Add a token for every word found that contains more than 4 vowels
* (including "y") in a row.
*/
int spam_test_gibberish_vowels(opts_t opts, msg_t msg, spam_t spam)
{
int nfound = 0;
long n;
char *word;
int len;
for (n = 0; n < msg->num_words; n++) {
word = msg->textcontent + msg->wordpos[n];
len = msg->wordlength[n];
if (spam_test_gibberish__runon(word, len, VOWELS, 4))
nfound++;
}
if (nfound > 0)
return nfound + 1;
return 0;
}
/*
* Add a token for if the "From:" or "Return-Path:" addresses contain more
* than 5 consonants (not including "y") in a row.
*/
int spam_test_gibberish_from_consonants(opts_t opts, msg_t msg,
spam_t spam)
{
if ((msg->sender)
&&
(spam_test_gibberish__runon
(msg->sender, strlen(msg->sender), CONSONANTS, 5)))
return 2;
if ((msg->envsender)
&&
(spam_test_gibberish__runon
(msg->envsender, strlen(msg->envsender), CONSONANTS, 5)))
return 2;
return 0;
}
/*
* Add a token if the "From:" address contains more than 4 vowels (including
* "y") in a row.
*/
int spam_test_gibberish_from_vowels(opts_t opts, msg_t msg, spam_t spam)
{
if ((msg->sender)
&&
(spam_test_gibberish__runon
(msg->sender, strlen(msg->sender), VOWELS, 4)))
return 2;
if ((msg->envsender)
&&
(spam_test_gibberish__runon
(msg->envsender, strlen(msg->envsender), VOWELS, 4)))
return 2;
return 0;
}
/*
* Add a token for every word found that starts with non-alphanumeric
* characters other than <>"'*_/.
*/
int spam_test_gibberish_badstart(opts_t opts, msg_t msg, spam_t spam)
{
int nfound = 0;
long n;
char *word;
int len;
for (n = 0; n < msg->num_words; n++) {
word = msg->textcontent + msg->wordpos[n];
len = msg->wordlength[n];
if (strchr(CONSONANTS, word[0]))
continue;
if (strchr(VOWELS, word[0]))
continue;
if (strchr("<>\"'*_/", word[0]))
continue;
nfound++;
}
if (nfound > 0)
return nfound + 1;
return 0;
}
/*
* Add a token for every word found that contains more than 3 hyphens or
* underscores.
*/
int spam_test_gibberish_hyphens(opts_t opts, msg_t msg, spam_t spam)
{
int nfound = 0;
long n;
char *word;
int len;
for (n = 0; n < msg->num_words; n++) {
int hyphens = 0;
word = msg->textcontent + msg->wordpos[n];
len = msg->wordlength[n];
while (len > 0) {
int c = word[0];
word++;
len--;
if ((c == '-') || (c == '_')) {
hyphens++;
if (hyphens > 3)
break;
}
}
if (hyphens > 3)
nfound++;
}
if (nfound > 0)
return nfound + 1;
return 0;
}
/*
* Add a token for every word found that is ridiculously long (over 30
* characters), not counting 60+ character words because they may be part of
* a Base64-encoded chunk.
*/
int spam_test_gibberish_longwords(opts_t opts, msg_t msg, spam_t spam)
{
int nfound = 0;
long n;
int len;
char *word;
for (n = 0; n < msg->num_words; n++) {
len = msg->wordlength[n];
word = msg->textcontent + msg->wordpos[n];
if (len <= 30)
continue;
if (len >= 60)
continue;
if (strchr(CONSONANTS, word[0]))
continue;
if (strchr(VOWELS, word[0]))
continue;
nfound++;
}
if (nfound > 0)
return nfound + 1;
return 0;
}
/* EOF */
qsf-1.2.7/src/tests/attached_files.c 0000644 0000764 0000764 00000013516 10664772304 015175 0 ustar aw aw /*
* Rules which add tokens if a message attachment's filename matches various
* patterns.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "testi.h"
#include
#define DISPOSITION_HEADER "\nContent-Disposition:"
extern char *minimemmem(char *, long, char *, long);
/*
* Return the number of times an attachment with the given filename
* extension (eg "scr", "pif", "exe") was found.
*/
static int spam_test_attachment__scan(msg_t msg, char *extension)
{
long pos, lastdot;
int nfound = 0;
pos = 0;
while (pos < msg->body_size) {
int quotes;
char *ptr;
ptr =
minimemmem(msg->body + pos, msg->body_size - pos,
DISPOSITION_HEADER,
strlen(DISPOSITION_HEADER));
if (ptr == 0)
return nfound;
pos = ptr - msg->body;
pos += strlen(DISPOSITION_HEADER);
while ((pos < msg->body_size)
&& ((msg->body[pos] == ' ')
|| (msg->body[pos] == '\t')
)
) {
pos++;
}
if (pos >= (msg->body_size - 12))
return nfound;
if (strncasecmp(msg->body + pos, "attachment;", 11) != 0)
continue;
pos += 11;
while ((pos < msg->body_size)
&& ((msg->body[pos] == ' ')
|| (msg->body[pos] == '\t')
|| (msg->body[pos] == '\r')
|| (msg->body[pos] == '\n')
)
) {
pos++;
}
if (pos >= (msg->body_size - 15))
return nfound;
if (strncasecmp(msg->body + pos, "filename=", 9) != 0)
continue;
pos += 9;
quotes = 0;
if (msg->body[pos] == '"') {
quotes = 1;
pos++;
}
lastdot = 0;
if (quotes) {
while ((pos < msg->body_size)
&& (msg->body[pos] != '"')
&& (msg->body[pos] != '\r')
&& (msg->body[pos] != '\n')
) {
if (msg->body[pos] == '.')
lastdot = pos;
pos++;
}
} else {
while ((pos < msg->body_size)
&& (msg->body[pos] != ';')
&& (msg->body[pos] != ' ')
&& (msg->body[pos] != '"')
&& (msg->body[pos] != '\t')
&& (msg->body[pos] != '\r')
&& (msg->body[pos] != '\n')
) {
if (msg->body[pos] == '.')
lastdot = pos;
pos++;
}
}
if (lastdot == 0)
continue;
if (pos >= (msg->body_size - 2))
return nfound;
lastdot++;
if (strlen(extension) != (pos - lastdot))
continue;
if (strncasecmp
(msg->body + lastdot, extension, pos - lastdot) == 0)
nfound++;
}
return nfound;
}
/*
* Add a token for every attachment with the filename "something.scr".
*/
int spam_test_attachment_scr(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "scr");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.pif".
*/
int spam_test_attachment_pif(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "pif");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.exe".
*/
int spam_test_attachment_exe(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "exe");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.vbs".
*/
int spam_test_attachment_vbs(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "vbs");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.vba".
*/
int spam_test_attachment_vba(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "vba");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.lnk".
*/
int spam_test_attachment_lnk(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "lnk");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.com".
*/
int spam_test_attachment_com(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "com");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.bat".
*/
int spam_test_attachment_bat(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "bat");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.pdf".
*/
int spam_test_attachment_pdf(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "pdf");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.doc".
*/
int spam_test_attachment_doc(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "doc");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.xls".
*/
int spam_test_attachment_xls(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "xls");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.gif".
*/
int spam_test_attachment_gif(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "gif");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.jpg" or
* "something.jpeg".
*/
int spam_test_attachment_jpg(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg,
"jpg") +
spam_test_attachment__scan(msg, "jpeg");
if (n > 0)
return n + 1;
return 0;
}
/*
* Add a token for every attachment with the filename "something.png".
*/
int spam_test_attachment_png(opts_t opts, msg_t msg, spam_t spam)
{
int n;
n = spam_test_attachment__scan(msg, "png");
if (n > 0)
return n + 1;
return 0;
}
/* EOF */
qsf-1.2.7/src/tests/html.c 0000644 0000764 0000764 00000005116 10664772304 013177 0 ustar aw aw /*
* Rules to look for oddities in the HTML of messages.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "testi.h"
#include "spam.h"
#include
extern char *minimemmem(char *, long, char *, long);
/*
* Add a token for every HTML comment found in the middle of a word (i.e.
* with a valid token character either side of it).
*/
int spam_test_html_comments_in_words(opts_t opts, msg_t msg, spam_t spam)
{
int nfound = 0;
long pos = 0;
char *ptr;
while (pos < msg->content_size) {
ptr =
minimemmem(msg->content + pos, msg->content_size - pos,
"content;
if ((pos < 1)
|| (strchr(TOKEN_CHARS, msg->content[pos - 1]) == 0)
) {
pos++;
continue;
}
pos++;
ptr =
minimemmem(msg->content + pos, msg->content_size - pos,
">", 1);
if (!ptr)
break;
pos = ptr - msg->content;
pos++;
if (pos >= msg->content_size)
break;
if (strchr(TOKEN_CHARS, msg->content[pos]) != 0)
nfound++;
}
if (nfound > 0)
return nfound + 1;
return 0;
}
/*
* Add a token for every IMG tag found referring to an external URL
* (containing ://).
*/
int spam_test_html_external_img(opts_t opts, msg_t msg, spam_t spam)
{
int nfound = 0;
long pos = 0;
char *ptr;
while (pos < msg->content_size) {
ptr =
memchr(msg->content + pos, '<',
msg->content_size - pos);
if (!ptr)
break;
pos = ptr - msg->content;
if (pos > msg->content_size - 4)
break;
pos++;
if (strncasecmp(msg->content + pos, "img", 3) == 0) {
ptr =
memchr(msg->content + pos, '>',
msg->content_size - pos);
if (!ptr)
break;
if (minimemmem
(msg->content + pos,
(ptr - msg->content) - pos, "://", 3))
nfound++;
pos = ptr - msg->content;
pos++;
if (pos >= msg->content_size)
break;
}
}
if (nfound > 0)
return nfound + 1;
return 0;
}
/*
* Add a token for every FONT tag found.
*/
int spam_test_html_font(opts_t opts, msg_t msg, spam_t spam)
{
int nfound = 0;
long pos = 0;
char *ptr;
while (pos < msg->content_size) {
ptr =
memchr(msg->content + pos, '<',
msg->content_size - pos);
if (!ptr)
break;
pos = ptr - msg->content;
if (pos > msg->content_size - 5)
break;
pos++;
if (strncasecmp(msg->content + pos, "font", 4) != 0)
continue;
nfound++;
ptr =
memchr(msg->content + pos, '>',
msg->content_size - pos);
if (!ptr)
break;
pos = ptr - msg->content;
}
if (nfound > 0)
return nfound + 1;
return 0;
}
/* EOF */
qsf-1.2.7/src/tests/testi.h 0000644 0000764 0000764 00000001022 10554714241 013352 0 ustar aw aw /*
* Internal spam test prototypes, structures, and constants.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#ifndef _TESTI_H
#define _TESTI_H 1
#ifndef _OPTIONS_H
#include "options.h"
#endif
#ifndef _MESSAGE_H
#include "message.h"
#endif
#ifdef __cplusplus
extern "C" {
#endif
struct spam_s;
typedef struct spam_s *spam_t;
typedef int (*spamtestfunc_t)(opts_t, msg_t, spam_t);
void spam_token_add(opts_t, spam_t, char *, int);
#ifdef __cplusplus
}
#endif
#endif /* _TESTI_H */
/* EOF */
qsf-1.2.7/src/mailbox/ 0000755 0000764 0000764 00000000000 10665021416 012345 5 ustar aw aw qsf-1.2.7/src/mailbox/mailboxi.h 0000644 0000764 0000764 00000001050 10554714242 014321 0 ustar aw aw /*
* Internal definitions for the mailbox functions.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#ifndef _MAILBOXI_H
#define _MAILBOXI_H 1
#ifdef __cplusplus
extern "C" {
#endif
struct mbox_s { /* structure describing a mailbox */
size_t count; /* number of messages */
size_t alloced; /* size of array */
size_t *start; /* array of message start offsets */
size_t *length; /* array of message sizes */
};
#ifdef __cplusplus
}
#endif
#endif /* _MAILBOXI_H */
/* EOF */
qsf-1.2.7/src/mailbox/alloc.c 0000644 0000764 0000764 00000000563 10664772304 013617 0 ustar aw aw /*
* Functions for memory allocation and deallocation.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "mailbox.h"
#include "mailboxi.h"
#include
/*
* Free an mbox_t.
*/
void mbox_free(mbox_t mbox)
{
if (mbox == NULL)
return;
free(mbox->start);
free(mbox->length);
free(mbox);
}
/* EOF */
qsf-1.2.7/src/mailbox/count.c 0000644 0000764 0000764 00000000550 10664772304 013651 0 ustar aw aw /*
* Count the number of messages in the given mailbox.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "mailbox.h"
#include "mailboxi.h"
/*
* Return the number of messages in the given mailbox.
*/
size_t mbox_count(mbox_t mbox)
{
if (mbox == NULL)
return 0;
return mbox->count;
}
/* EOF */
qsf-1.2.7/src/mailbox/select.c 0000644 0000764 0000764 00000002567 10664772304 014012 0 ustar aw aw /*
* Select a particular message from a mailbox.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "mailbox.h"
#include "mailboxi.h"
#include
#include
#include
#include
/*
* Select the given message in the given mailbox by loading it into memory
* and setting the stdin-replacement stream to be an in-memory file handle.
*
* Returns non-zero on error.
*/
int mbox_select(opts_t opts, mbox_t mbox, FILE * fptr, size_t num)
{
static char *buf = NULL;
if (buf != NULL) {
free(buf);
buf = NULL;
}
if (fptr == NULL)
return 0;
if (mbox->length[num] < 1) {
fprintf(stderr, "%s: %s %d: %s\n", opts->program_name,
_("message"), (int) num + 1,
_("invalid message size"));
opts->inbuf = "";
opts->inbufsize = 1;
return 1;
}
fseek(fptr, mbox->start[num], SEEK_SET);
buf = calloc(1, mbox->length[num]);
if (buf == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
opts->inbuf = "";
opts->inbufsize = 1;
return 1;
}
if (fread(buf, mbox->length[num], 1, fptr) < 1) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("failed to read mailbox"), strerror(errno));
opts->inbuf = "";
opts->inbufsize = 1;
return 1;
}
opts->inbuf = buf;
opts->inbufsize = mbox->length[num];
return 0;
}
/* EOF */
qsf-1.2.7/src/mailbox/scan.c 0000644 0000764 0000764 00000005214 10664772304 013447 0 ustar aw aw /*
* Functions to scan a mailbox for messages.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include "config.h"
#include "mailbox.h"
#include "mailboxi.h"
#include
#include
#include
#include
void tick(void);
/*
* Scan the mailbox on the given file handle, returning an mbox_t describing
* where each message in the mailbox can be found, or NULL on error.
*/
mbox_t mbox_scan(opts_t opts, FILE * fptr)
{
char linebuf[1024]; /* RATS: ignore (size OK) */
mbox_t mbox;
int prevnewline = 0;
size_t pos, filepos;
size_t *newptr;
mbox = calloc(1, sizeof(*mbox));
if (mbox == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
return NULL;
}
mbox->start = calloc(1, sizeof(pos));
if (mbox->start == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
free(mbox);
return NULL;
}
mbox->length = calloc(1, sizeof(pos));
if (mbox->length == NULL) {
fprintf(stderr, "%s: %s: %s\n", opts->program_name,
_("calloc failed"), strerror(errno));
free(mbox->start);
free(mbox);
return NULL;
}
mbox->count = 0;
mbox->alloced = 1;
filepos = 0;
while (fgets(linebuf, sizeof(linebuf) - 1, fptr) != NULL) {
linebuf[sizeof(linebuf) - 1] = 0;
filepos = ftell(fptr);
tick();
if (strrchr(linebuf, '\n') == NULL) {
prevnewline = 0;
continue;
}
if (prevnewline && (strncmp(linebuf, "From ", 5) == 0)) {
pos = filepos - strlen(linebuf);
mbox->length[mbox->count] =
pos - mbox->start[mbox->count];
if (mbox->count >= (mbox->alloced - 1)) {
mbox->alloced += 4096;
newptr = realloc( /* RATS: ignore (OK) */
mbox->start,
sizeof(pos) *
(mbox->alloced));
if (newptr == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("realloc failed"),
strerror(errno));
free(mbox->start);
free(mbox->length);
free(mbox);
return NULL;
}
mbox->start = newptr;
newptr = realloc( /* RATS: ignore (OK) */
mbox->length,
sizeof(pos) *
(mbox->alloced));
if (newptr == NULL) {
fprintf(stderr, "%s: %s: %s\n",
opts->program_name,
_("realloc failed"),
strerror(errno));
free(mbox->start);
free(mbox->length);
free(mbox);
return NULL;
}
mbox->length = newptr;
}
mbox->count++;
mbox->start[mbox->count] = pos;
mbox->length[mbox->count] = 0;
}
prevnewline = 1;
}
mbox->length[mbox->count] = filepos - mbox->start[mbox->count];
if (mbox->length[mbox->count] != 0)
mbox->count++;
return mbox;
}
/* EOF */
qsf-1.2.7/autoconf/ 0000755 0000764 0000764 00000000000 10665021416 011741 5 ustar aw aw qsf-1.2.7/autoconf/configure.in 0000644 0000764 0000764 00000016335 10516437234 014266 0 ustar aw aw dnl Process this file with autoconf to produce a configure script.
dnl
AC_INIT(src/main/version.c)
dnl We're using C.
dnl
AC_LANG_C
dnl Output a header file.
dnl
AC_CONFIG_HEADER(src/include/config.h:autoconf/header.in)
dnl Set directory to check for Configure scripts in.
dnl
AC_CONFIG_AUX_DIR(autoconf/scripts)
dnl Read in package details.
dnl
PACKAGE=`cat $srcdir/doc/PACKAGE`
VERSION=`cat $srcdir/doc/VERSION`
UCPACKAGE=`tr a-z A-Z < $srcdir/doc/PACKAGE`
AC_SUBST(PACKAGE)
AC_SUBST(VERSION)
AC_SUBST(UCPACKAGE)
AC_DEFINE_UNQUOTED(PACKAGE, "$PACKAGE")
AC_DEFINE_UNQUOTED(PROGRAM_NAME, "$PACKAGE")
AC_DEFINE_UNQUOTED(VERSION, "$VERSION")
dnl Database backends we can use.
dnl
canuse_obtree="yes"
canuse_btree="yes"
canuse_list="yes"
canuse_gdbm="yes"
canuse_mysql="yes"
canuse_sqlite="yes"
dnl Check for compile-time options.
dnl
AC_ARG_ENABLE(debugging,
[ --enable-debugging compile with debugging symbols],
if test "$enable_debugging" = "yes"; then
CFLAGS="-g -Wall"
fi
)
AC_ARG_ENABLE(profiling,
[ --enable-profiling compile with profiling support],
if test "$enable_profiling" = "yes"; then
CFLAGS="-pg $CFLAGS"
fi
)
AC_ARG_ENABLE(static,
[ --enable-static enable static linking],
)
AC_ARG_WITH(obtree,
[ --without-obtree omit old binary tree backend support],
canuse_obtree="$with_obtree"
)
AC_ARG_WITH(btree,
[ --without-btree omit binary tree backend support],
canuse_btree="$with_btree"
)
AC_ARG_WITH(list,
[ --without-list omit list backend support],
canuse_list="$with_list"
)
AC_ARG_WITH(gdbm,
[ --without-gdbm omit GDBM backend support],
canuse_gdbm="$with_gdbm"
)
AC_ARG_WITH(mysql,
[ --without-mysql omit MySQL backend support],
canuse_mysql="$with_mysql"
)
AC_ARG_WITH(sqlite,
[ --without-sqlite omit SQLite v2.x backend support],
canuse_sqlite="$with_sqlite"
)
dnl Check for various programs.
dnl
CFLAGS=${CFLAGS-"-O2 -Wall -s"}
AC_PROG_CC
AC_PROG_CPP
AC_CHECK_TOOL(LD, ld, libtool --mode=link gcc)
AC_SUBST(LD)
AC_PROG_INSTALL
AC_PROG_MAKE_SET
AC_CHECK_PROG(DO_GZIP, gzip, gzip -f9, touch)
dnl Check for the maths library.
dnl
AC_SEARCH_LIBS(pow, m, , [AC_ERROR(maths library not found)])
dnl Check for various header files and set various other macros.
dnl
AC_DEFINE(HAVE_CONFIG_H)
AC_HEADER_STDC
AC_C_BIGENDIAN([AC_DEFINE(IS_BIG_ENDIAN)])
AC_CHECK_FUNCS(memcpy, , [AC_ERROR(the memcpy() function is required)])
AC_CHECK_FUNCS(fcntl getopt getopt_long mkstemp snprintf vsnprintf utime)
AC_CHECK_HEADERS(fcntl.h getopt.h limits.h sys/resource.h mcheck.h)
AC_FUNC_MMAP
dnl Check for backend databases and choose one.
dnl
PREVCFLAGS="$CFLAGS"
PREVCPPFLAGS="$CPPFLAGS"
PREVLIBS="$LIBS"
if test "x$canuse_mysql" = "xyes"; then
dnl
dnl First we try linking without the mysql_config libs, because on
dnl some systems that'll give us a dynamic library - the
dnl mysql_config libs often point us to static libraries.
dnl
dnl If static linking is enabled, we ONLY try the mysql_config libs.
dnl
CFLAGS=`mysql_config --cflags 2>/dev/null`
CPPFLAGS=`mysql_config --include 2>/dev/null` || CPPFLAGS="$CFLAGS"
LIBS=""
if test "x$enable_static" != "xyes"; then
AC_CHECK_HEADERS(mysql.h, [
AC_SEARCH_LIBS(mysql_real_connect, mysqlclient, [
AC_SEARCH_LIBS(mysql_real_escape_string, mysqlclient, , canuse_mysql="no")
], canuse_mysql="no")
], canuse_mysql="no")
fi
if test "x$canuse_mysql" = "xno"; then
canuse_mysql=yes
LIBS=`mysql_config --libs 2>/dev/null`
AC_CHECK_HEADERS(mysql.h, [
AC_SEARCH_LIBS(mysql_real_escape_string, mysqlclient, , canuse_mysql="no")
], canuse_mysql="no")
fi
if test "x$enable_static" = "xyes"; then
LIBS=`mysql_config --libs 2>/dev/null`
AC_CHECK_HEADERS(mysql.h, [
AC_SEARCH_LIBS(mysql_real_query, mysqlclient, , canuse_mysql="no")
], canuse_mysql="no")
fi
if test "x$canuse_mysql" = "xyes"; then
AC_CHECK_FUNCS(mysql_autocommit)
fi
MYSQLCFLAGS="$CFLAGS"
MYSQLLIBS="$LIBS"
CFLAGS="$PREVCFLAGS"
CPPFLAGS="$PREVCPPFLAGS"
LIBS="$PREVLIBS"
fi
if test "x$canuse_gdbm" = "xyes"; then
PREVLIBS="$LIBS"
LIBS=""
AC_CHECK_HEADERS(gdbm.h, [
AC_SEARCH_LIBS(gdbm_open, gdbm, [
AC_CHECK_FUNCS(gdbm_fdesc)
], canuse_gdbm="no")
], canuse_gdbm="no")
dnl
dnl This is a hideous hack to try and find ".a" (static)
dnl library replacements if --enable-static is given.
dnl
if test "x$enable_static" = "xyes"; then
LIBDIR=`echo "$LIBS" | tr ' ' '\n' | sed -n 's/^-L//p'`
test -z "$LIBDIR" && LIBDIR=/usr/lib
LIBLIST=`echo "$LIBS" | tr ' ' '\n' | sed -n "s,^-l,$LIBDIR/lib,p" | sed -e 's,$,.a,'`
NEWLIBS=""
for i in $LIBLIST; do
test -e "$i" && NEWLIBS="$NEWLIBS $i"
done
test -n "$NEWLIBS" && LIBS="$NEWLIBS"
fi
GDBMLIBS="$LIBS"
LIBS="$PREVLIBS"
fi
if test "x$canuse_sqlite" = "xyes"; then
LIBS=""
AC_CHECK_HEADERS(sqlite.h, [
AC_SEARCH_LIBS(sqlite_open, sqlite, , canuse_sqlite="no")
], canuse_sqlite="no")
dnl Hideous hack as above.
if test "x$enable_static" = "xyes"; then
LIBDIR=`echo "$LIBS" | tr ' ' '\n' | sed -n 's/^-L//p'`
test -z "$LIBDIR" && LIBDIR=/usr/lib
LIBLIST=`echo "$LIBS" | tr ' ' '\n' | sed -n "s,^-l,$LIBDIR/lib,p" | sed -e 's,$,.a,'`
NEWLIBS=""
for i in $LIBLIST; do
test -e "$i" && NEWLIBS="$NEWLIBS $i"
done
test -n "$NEWLIBS" && LIBS="$NEWLIBS"
fi
SQLITELIBS="$LIBS"
LIBS="$PREVLIBS"
fi
if test "x$canuse_list" = "xyes"; then
AC_CHECK_HEADERS(search.h, [
AC_CHECK_FUNCS(bsearch qsort, [], canuse_list="no")
], canuse_list="no")
fi
AC_MSG_CHECKING(which backend databases are available)
BACKENDS=""
if test "x$canuse_obtree" = "xyes"; then
AC_DEFINE(USING_OBTREE)
BACKENDS="$BACKENDS obtree"
fi
if test "x$canuse_btree" = "xyes"; then
AC_DEFINE(USING_BTREE)
BACKENDS="$BACKENDS btree"
fi
if test "x$canuse_list" = "xyes"; then
AC_DEFINE(USING_LIST)
BACKENDS="$BACKENDS list"
fi
if test "x$canuse_gdbm" = "xyes"; then
AC_DEFINE(USING_GDBM)
EXTRALIBS="$EXTRALIBS $GDBMLIBS"
BACKENDS="$BACKENDS GDBM"
fi
if test "x$canuse_mysql" = "xyes"; then
AC_DEFINE(USING_MYSQL)
EXTRALIBS="$EXTRALIBS $MYSQLLIBS"
CFLAGS="$CFLAGS $MYSQLCFLAGS"
BACKENDS="$BACKENDS MySQL"
fi
if test "x$canuse_sqlite" = "xyes"; then
AC_DEFINE(USING_SQLITE)
EXTRALIBS="$EXTRALIBS $SQLITELIBS"
CFLAGS="$CFLAGS $SQLITECFLAGS"
BACKENDS="$BACKENDS SQLite2"
fi
if test "x$BACKENDS" = "x"; then
AC_MSG_RESULT(none)
AC_MSG_ERROR(no usable database libraries found)
else
AC_MSG_RESULT($BACKENDS)
fi
LIBS="$LIBS $EXTRALIBS"
AC_DEFINE_UNQUOTED(BACKENDS, "$BACKENDS")
AC_SUBST(BACKENDS)
test -z "$INSTALL_DATA" && INSTALL_DATA='${INSTALL} -m 644'
AC_SUBST(INSTALL_DATA)
dnl Fudging for separate build directories.
dnl
subdirs=""
for i in `find $srcdir/src -type d -print | sed s,$srcdir/,,`; do
subdirs="$subdirs $i"
done
dnl Stitch together the Makefile fragments.
dnl
mk_segments="autoconf/Makefile.in"
for i in vars.mk package.mk filelist.mk unreal.mk modules.mk \
rules.mk link.mk depend.mk; do
mk_segments="$mk_segments:autoconf/make/$i"
done
dnl Output files (and create build directory structure too).
dnl
AC_OUTPUT(Makefile:$mk_segments doc/lsm:doc/lsm.in
doc/quickref.1:doc/quickref.1.in
doc/$PACKAGE.spec:doc/spec.in
src/.dummy:doc/NEWS,
rm -f src/.dummy
for i in $subdirs; do
test -d $i || mkdir $i
done
, subdirs="$subdirs")
dnl EOF
qsf-1.2.7/autoconf/header.in 0000644 0000764 0000764 00000003020 10554714251 013517 0 ustar aw aw /* Define if you have standard C headers. */
#undef STDC_HEADERS
/* Define if you have "config.h" (yes, you have). */
#undef HAVE_CONFIG_H
/* Define these to select database backends. */
#undef USING_OBTREE
#undef USING_BTREE
#undef USING_LIST
#undef USING_GDBM
#undef USING_MYSQL
#undef USING_SQLITE
/* Define if debugging is to be compiled in. */
#undef DEBUG
/* Define these for various system header files and functions. */
#undef HAVE_GETOPT_H
#undef HAVE_MCHECK_H
#undef HAVE_SYS_RESOURCE_H
#undef HAVE_FCNTL_H
#undef HAVE_MKSTEMP
#undef HAVE_GETOPT
#undef HAVE_FCNTL
#undef HAVE_SNPRINTF
#undef HAVE_VSNPRINTF
#undef HAVE_GETOPT_LONG
#undef HAVE_GDBM_FDESC
#undef HAVE_MMAP
#undef HAVE_UTIME
#undef HAVE_SEARCH_H
#undef HAVE_BSEARCH
#undef HAVE_QSORT
#undef HAVE_MYSQL_AUTOCOMMIT
#undef IS_BIG_ENDIAN
#ifdef ENABLE_NLS
# include
# ifdef HAVE_LOCALE_H
# include
# endif
# define _(String) gettext (String)
# define N_(String) gettext_noop (String)
#else
# define _(String) (String)
#endif
/* The name of the program. */
#define PROGRAM_NAME "progname"
/* The name of the package. */
#define PACKAGE ""
/* The current package version. */
#define VERSION "0.0.0"
/* Various identification and legal drivel. */
#define COPYRIGHT_YEAR _("2007")
#define COPYRIGHT_HOLDER _("Andrew Wood ")
#define PROJECT_HOMEPAGE "http://www.ivarch.com/programs/" PROGRAM_NAME "/"
#define BUG_REPORTS_TO _("Andrew Wood ")
/* Backends available. */
#define BACKENDS ""
/* EOF */
qsf-1.2.7/autoconf/make/ 0000755 0000764 0000764 00000000000 10665021416 012656 5 ustar aw aw qsf-1.2.7/autoconf/make/filelist.mk 0000644 0000764 0000764 00000005655 10665021314 015032 0 ustar aw aw # Automatically generated file listings
#
# Creation time: Tue Aug 28 14:27:40 BST 2007
allsrc = src/library.c \
src/md5.c \
src/db/obtree.c \
src/db/list.c \
src/db/main.c \
src/db/gdbm.c \
src/db/btree.c \
src/db/sqlite.c \
src/db/mysql.c \
src/main/help.c \
src/main/options.c \
src/main/version.c \
src/main/main.c \
src/main/log.c \
src/main/tick.c \
src/spam/update.c \
src/spam/merge.c \
src/spam/alloc.c \
src/spam/train.c \
src/spam/cksum.c \
src/spam/benchmark.c \
src/spam/check.c \
src/spam/plaintext.c \
src/spam/dump.c \
src/spam/token.c \
src/spam/db.c \
src/spam/prune.c \
src/spam/allowlist.c \
src/message/parse.c \
src/message/alloc.c \
src/message/read.c \
src/message/rfc2047.c \
src/message/header.c \
src/message/qp.c \
src/message/dump.c \
src/message/base64.c \
src/tests/gtube.c \
src/tests/main.c \
src/tests/imgcount.c \
src/tests/urls.c \
src/tests/gibberish.c \
src/tests/attached_files.c \
src/tests/html.c \
src/mailbox/alloc.c \
src/mailbox/count.c \
src/mailbox/select.c \
src/mailbox/scan.c
allobj = src/library.o \
src/md5.o \
src/db/obtree.o \
src/db/list.o \
src/db/main.o \
src/db/gdbm.o \
src/db/btree.o \
src/db/sqlite.o \
src/db/mysql.o \
src/main/help.o \
src/main/options.o \
src/main/version.o \
src/main/main.o \
src/main/log.o \
src/main/tick.o \
src/spam/update.o \
src/spam/merge.o \
src/spam/alloc.o \
src/spam/train.o \
src/spam/cksum.o \
src/spam/benchmark.o \
src/spam/check.o \
src/spam/plaintext.o \
src/spam/dump.o \
src/spam/token.o \
src/spam/db.o \
src/spam/prune.o \
src/spam/allowlist.o \
src/message/parse.o \
src/message/alloc.o \
src/message/read.o \
src/message/rfc2047.o \
src/message/header.o \
src/message/qp.o \
src/message/dump.o \
src/message/base64.o \
src/tests/gtube.o \
src/tests/main.o \
src/tests/imgcount.o \
src/tests/urls.o \
src/tests/gibberish.o \
src/tests/attached_files.o \
src/tests/html.o \
src/mailbox/alloc.o \
src/mailbox/count.o \
src/mailbox/select.o \
src/mailbox/scan.o \
src/db.o \
src/main.o \
src/spam.o \
src/message.o \
src/tests.o \
src/mailbox.o
alldep = src/library.d \
src/md5.d \
src/db/obtree.d \
src/db/list.d \
src/db/main.d \
src/db/gdbm.d \
src/db/btree.d \
src/db/sqlite.d \
src/db/mysql.d \
src/main/help.d \
src/main/options.d \
src/main/version.d \
src/main/main.d \
src/main/log.d \
src/main/tick.d \
src/spam/update.d \
src/spam/merge.d \
src/spam/alloc.d \
src/spam/train.d \
src/spam/cksum.d \
src/spam/benchmark.d \
src/spam/check.d \
src/spam/plaintext.d \
src/spam/dump.d \
src/spam/token.d \
src/spam/db.d \
src/spam/prune.d \
src/spam/allowlist.d \
src/message/parse.d \
src/message/alloc.d \
src/message/read.d \
src/message/rfc2047.d \
src/message/header.d \
src/message/qp.d \
src/message/dump.d \
src/message/base64.d \
src/tests/gtube.d \
src/tests/main.d \
src/tests/imgcount.d \
src/tests/urls.d \
src/tests/gibberish.d \
src/tests/attached_files.d \
src/tests/html.d \
src/mailbox/alloc.d \
src/mailbox/count.d \
src/mailbox/select.d \
src/mailbox/scan.d
qsf-1.2.7/autoconf/make/rules.mk 0000644 0000764 0000764 00000000576 07701413734 014356 0 ustar aw aw #
# Compilation rules.
#
#
.SUFFIXES: .c .d .o
.c.o:
$(CC) $(CFLAGS) $(CPPFLAGS) -c -o $@ $<
.c.d:
sh $(srcdir)/autoconf/scripts/depend.sh \
$(CC) $< $(<:%.c=%) $(srcdir) $(CFLAGS) $(CPPFLAGS) > $@
doc/quickref.txt: doc/quickref.1
man doc/quickref.1 | col -b | cat -s > doc/quickref.txt || :
chmod 644 doc/quickref.txt || :
qsf-1.2.7/autoconf/make/vars.mk 0000644 0000764 0000764 00000001162 10507164042 014160 0 ustar aw aw #
# Variables for Make.
#
srcdir = @srcdir@
prefix = @prefix@
exec_prefix = @exec_prefix@
bindir = @bindir@
infodir = @infodir@
mandir = @mandir@
etcdir = @prefix@/etc
datadir = @datadir@
sbindir = @sbindir@
VPATH = $(srcdir)
@SET_MAKE@
SHELL = /bin/sh
CC = @CC@
LD = @LD@
DO_GZIP = @DO_GZIP@
INSTALL = @INSTALL@
INSTALL_DATA = @INSTALL_DATA@
UNINSTALL = rm -f
LDFLAGS = -r
DEFS = @DEFS@
CFLAGS = @CFLAGS@
CPPFLAGS = @CPPFLAGS@ -I$(srcdir)/src/include -Isrc/include $(DEFS)
LIBS = @LDFLAGS@ @LIBS@
testdb := /tmp/@PACKAGE@test.db
testfile := /tmp/@PACKAGE@test.txt
testbackends := @BACKENDS@
alltarg = @PACKAGE@
# EOF
qsf-1.2.7/autoconf/make/package.mk 0000644 0000764 0000764 00000000436 10013256020 014571 0 ustar aw aw #
# Package name, version, and distribution files.
#
package = @PACKAGE@
version = @VERSION@
PACKAGE = @PACKAGE@
distfiles = \
$(srcdir)/README \
$(srcdir)/autoconf \
$(srcdir)/configure \
$(srcdir)/doc \
$(srcdir)/src \
$(srcdir)/test \
$(srcdir)/debian \
$(srcdir)/extra
# EOF
qsf-1.2.7/autoconf/make/depend.mk 0000644 0000764 0000764 00000014515 10665021321 014447 0 ustar aw aw #
# Dependencies.
#
src/library.d src/library.o: src/library.c src/include/config.h
src/md5.d src/md5.o: src/md5.c src/include/md5.h src/include/config.h
src/db/obtree.d src/db/obtree.o: src/db/obtree.c src/include/config.h src/include/database.h src/include/log.h
src/db/list.d src/db/list.o: src/db/list.c src/include/config.h src/include/database.h src/include/log.h
src/db/main.d src/db/main.o: src/db/main.c src/include/config.h src/include/database.h
src/db/gdbm.d src/db/gdbm.o: src/db/gdbm.c src/include/config.h src/include/database.h
src/db/btree.d src/db/btree.o: src/db/btree.c src/include/config.h src/include/database.h src/include/log.h
src/db/sqlite.d src/db/sqlite.o: src/db/sqlite.c src/include/config.h src/include/database.h src/include/log.h
src/db/mysql.d src/db/mysql.o: src/db/mysql.c src/include/config.h src/include/database.h src/include/log.h
src/main/help.d src/main/help.o: src/main/help.c src/include/config.h
src/main/options.d src/main/options.o: src/main/options.c src/include/config.h src/include/options.h src/include/spam.h src/include/message.h src/include/log.h
src/main/version.d src/main/version.o: src/main/version.c src/include/config.h
src/main/main.d src/main/main.o: src/main/main.c src/include/config.h src/include/options.h src/include/message.h src/include/spam.h src/include/database.h src/include/log.h
src/main/log.d src/main/log.o: src/main/log.c src/include/config.h src/include/log.h
src/main/tick.d src/main/tick.o: src/main/tick.c src/include/config.h
src/spam/update.d src/spam/update.o: src/spam/update.c src/include/config.h src/spam/spami.h src/include/options.h src/include/message.h src/include/database.h src/include/spam.h
src/spam/merge.d src/spam/merge.o: src/spam/merge.c src/include/config.h src/spam/spami.h src/include/options.h src/include/message.h src/include/database.h src/include/spam.h
src/spam/alloc.d src/spam/alloc.o: src/spam/alloc.c src/include/config.h src/spam/spami.h src/include/options.h src/include/message.h src/include/database.h src/include/spam.h
src/spam/train.d src/spam/train.o: src/spam/train.c src/include/config.h src/include/mailbox.h src/include/options.h src/spam/spami.h src/include/message.h src/include/database.h src/include/spam.h
src/spam/cksum.d src/spam/cksum.o: src/spam/cksum.c src/include/config.h src/include/md5.h
src/spam/benchmark.d src/spam/benchmark.o: src/spam/benchmark.c src/include/config.h src/include/mailbox.h src/include/options.h src/spam/spami.h src/include/message.h src/include/database.h src/include/spam.h
src/spam/check.d src/spam/check.o: src/spam/check.c src/include/config.h src/spam/spami.h src/include/options.h src/include/message.h src/include/database.h src/include/spam.h src/include/log.h
src/spam/plaintext.d src/spam/plaintext.o: src/spam/plaintext.c src/include/config.h src/spam/spami.h src/include/options.h src/include/message.h src/include/database.h src/include/spam.h src/include/log.h
src/spam/dump.d src/spam/dump.o: src/spam/dump.c src/include/config.h src/spam/spami.h src/include/options.h src/include/message.h src/include/database.h src/include/spam.h
src/spam/token.d src/spam/token.o: src/spam/token.c src/include/config.h src/spam/spami.h src/include/options.h src/include/message.h src/include/database.h src/include/spam.h
src/spam/db.d src/spam/db.o: src/spam/db.c src/include/config.h src/spam/spami.h src/include/options.h src/include/message.h src/include/database.h src/include/spam.h
src/spam/prune.d src/spam/prune.o: src/spam/prune.c src/include/config.h src/spam/spami.h src/include/options.h src/include/message.h src/include/database.h src/include/spam.h
src/spam/allowlist.d src/spam/allowlist.o: src/spam/allowlist.c src/include/config.h src/spam/spami.h src/include/options.h src/include/message.h src/include/database.h src/include/spam.h src/include/log.h
src/message/parse.d src/message/parse.o: src/message/parse.c src/include/config.h src/include/message.h src/include/options.h src/include/md5.h
src/message/alloc.d src/message/alloc.o: src/message/alloc.c src/include/config.h src/include/message.h src/include/options.h
src/message/read.d src/message/read.o: src/message/read.c src/include/config.h src/include/message.h src/include/options.h
src/message/rfc2047.d src/message/rfc2047.o: src/message/rfc2047.c src/include/config.h src/include/message.h src/include/options.h
src/message/header.d src/message/header.o: src/message/header.c src/include/config.h src/include/message.h src/include/options.h
src/message/qp.d src/message/qp.o: src/message/qp.c src/include/config.h
src/message/dump.d src/message/dump.o: src/message/dump.c src/include/config.h src/include/message.h src/include/options.h src/include/log.h
src/message/base64.d src/message/base64.o: src/message/base64.c src/include/config.h
src/tests/gtube.d src/tests/gtube.o: src/tests/gtube.c src/include/config.h src/tests/testi.h src/include/options.h src/include/message.h
src/tests/main.d src/tests/main.o: src/tests/main.c src/include/config.h src/tests/testi.h src/include/options.h src/include/message.h
src/tests/imgcount.d src/tests/imgcount.o: src/tests/imgcount.c src/include/config.h src/tests/testi.h src/include/options.h src/include/message.h
src/tests/urls.d src/tests/urls.o: src/tests/urls.c src/include/config.h src/tests/testi.h src/include/options.h src/include/message.h src/include/spam.h
src/tests/gibberish.d src/tests/gibberish.o: src/tests/gibberish.c src/include/config.h src/tests/testi.h src/include/options.h src/include/message.h
src/tests/attached_files.d src/tests/attached_files.o: src/tests/attached_files.c src/include/config.h src/tests/testi.h src/include/options.h src/include/message.h
src/tests/html.d src/tests/html.o: src/tests/html.c src/include/config.h src/tests/testi.h src/include/options.h src/include/message.h src/include/spam.h
src/mailbox/alloc.d src/mailbox/alloc.o: src/mailbox/alloc.c src/include/config.h src/include/mailbox.h src/include/options.h src/mailbox/mailboxi.h
src/mailbox/count.d src/mailbox/count.o: src/mailbox/count.c src/include/config.h src/include/mailbox.h src/include/options.h src/mailbox/mailboxi.h
src/mailbox/select.d src/mailbox/select.o: src/mailbox/select.c src/include/config.h src/include/mailbox.h src/include/options.h src/mailbox/mailboxi.h
src/mailbox/scan.d src/mailbox/scan.o: src/mailbox/scan.c src/include/config.h src/include/mailbox.h src/include/options.h src/mailbox/mailboxi.h
qsf-1.2.7/autoconf/make/unreal.mk 0000644 0000764 0000764 00000035203 10554427314 014504 0 ustar aw aw #
# Rules for all phony targets.
#
.PHONY: all help dep depend depclean make \
check test memtest benchmark bigbenchmark \
clean distclean cvsclean \
index manhtml indent indentclean \
changelog doc dist release \
install uninstall \
rpmbuild srpm rpm deb
all: $(alltarg)
help:
@echo 'This Makefile has the following utility targets:'
@echo
@echo ' all build all binary targets'
@echo ' doc regenerate text version of man page'
@echo ' install install compiled package and manual'
@echo ' uninstall uninstall the package'
@echo ' check / test run standardised tests on the compiled binary'
@echo
@echo 'Developer targets:'
@echo
@echo ' make rebuild the Makefile (after adding new files)'
@echo ' dep / depend rebuild .d (dependency) files'
@echo ' clean remove .o (object) and .c~ (backup) files'
@echo ' depclean remove .d (dependency) files'
@echo ' indentclean remove files left over from "make indent"'
@echo ' distclean remove everything not distributed'
@echo ' cvsclean remove everything not in CVS'
@echo
@echo ' index generate an HTML index of source code'
@echo ' manhtml output HTML man page to stdout'
@echo ' indent reformat all source files with "indent"'
@echo ' changelog generate doc/changelog from CVS log info'
@echo
@echo ' memtest run "make test" using valgrind to find faults'
@echo ' benchmark run benchmarking tests'
@echo ' bigbenchmark run many benchmarking tests to make a graph'
@echo
@echo ' dist create a source tarball for distribution'
@echo ' rpm build a binary RPM (passes $$RPMFLAGS to RPM)'
@echo ' srpm build a source RPM (passes $$RPMFLAGS to RPM)'
@echo ' deb build a binary Debian package'
@echo ' release dist+rpm+srpm'
@echo
@echo 'Note that "test", "memtest", and "{,big}benchmark" can be passed'
@echo 'additional environment variables: BACKENDS is a space separated'
@echo 'list of backends to test (default is to test all) and MYSQLDB'
@echo 'is a database spec for testing the MySQL backend, in the same'
@echo 'format as for @PACKAGE@ -d, eg MYSQLDB=database=foo;host=...'
@echo
make:
echo > $(srcdir)/autoconf/make/filelist.mk
echo > $(srcdir)/autoconf/make/modules.mk
cd $(srcdir); \
bash autoconf/scripts/makemake.sh \
autoconf/make/filelist.mk \
autoconf/make/modules.mk
sh ./config.status
dep depend: $(alldep)
echo '#' > $(srcdir)/autoconf/make/depend.mk
echo '# Dependencies.' >> $(srcdir)/autoconf/make/depend.mk
echo '#' >> $(srcdir)/autoconf/make/depend.mk
echo >> $(srcdir)/autoconf/make/depend.mk
cat $(alldep) >> $(srcdir)/autoconf/make/depend.mk
sh ./config.status
clean:
rm -f $(allobj)
find . -type f -name "*.c~" -exec rm -f '{}' ';'
rm -f memtest-out-*
rm -f benchmark-report benchmark-data-* benchmark-graph-*
depclean:
rm -f $(alldep)
indentclean:
cd $(srcdir) && for FILE in $(allsrc); do rm -fv ./$${FILE}~; done
distclean: clean depclean
rm -f $(alltarg) src/include/config.h
rm -rf $(package)-$(version).tar* $(package)-$(version) BUILD-DEB
rm -f *.rpm *.deb
rm -f *.html config.*
rm -f test.db trace
rm -f .test-spam .test-non-spam
rm -f .test-benchmark-spam .test-benchmark-non-spam
rm -f .testdump-a .testdump-b .testdump-c
rm -f fakemail mboxsplit gmon.out
rm Makefile
cvsclean: distclean
rm -f doc/lsm
rm -f doc/$(package).spec
rm -f doc/quickref.1
rm -f doc/quickref.txt
rm -f configure
rm -f doc/changelog ChangeLog
cat /dev/null > $(srcdir)/autoconf/make/depend.mk
cat /dev/null > $(srcdir)/autoconf/make/filelist.mk
cat /dev/null > $(srcdir)/autoconf/make/modules.mk
doc: doc/quickref.txt
index:
(cd $(srcdir); sh autoconf/scripts/index.sh $(srcdir)) > index.html
manhtml:
@man2html ./doc/quickref.1 \
| sed -e '1,/]*> ||ig' \
-e 's|]*>\([^<]*\) |\1|ig' \
-e '/\)|\1 |ig' \
-e 's/
//ig' \
-e 's/<[0-9A-Za-z_.-]\+@[0-9A-Za-z_.-]\+>//g' \
-e 's|\(http://.*\) |\1 |ig' \
| sed -e '1,/ Index/,/ /dev/null 2>&1 || ( \
echo '*** Please put cvs2cl in your PATH'; \
echo '*** Get it from http://www.red-bean.com/cvs2cl/'; \
exit 1; \
)
rm -f $(srcdir)/ChangeLog
cd $(srcdir) && cvs2cl -S -P
mv -f $(srcdir)/ChangeLog doc/changelog
dist: doc
test -d $(srcdir)/CVS && $(MAKE) changelog || :
rm -rf $(package)-$(version)
mkdir $(package)-$(version)
cp -dprf Makefile $(distfiles) $(package)-$(version)
cd $(package)-$(version); $(MAKE) distclean
-cp -dpf doc/changelog $(package)-$(version)/doc/
cp -dpf doc/lsm $(package)-$(version)/doc/
cp -dpf doc/$(package).spec $(package)-$(version)/doc/
cp -dpf doc/quickref.txt $(package)-$(version)/doc/
chmod 644 `find $(package)-$(version) -type f -print`
chmod 755 `find $(package)-$(version) -type d -print`
chmod 755 `find $(package)-$(version)/autoconf/scripts`
chmod 755 $(package)-$(version)/configure
chmod 755 $(package)-$(version)/debian/rules
rm -rf DUMMY `find $(package)-$(version) -type d -name CVS` \
`find $(package)-$(version) -type f -name .cvsignore`
tar cf $(package)-$(version).tar $(package)-$(version)
rm -rf $(package)-$(version)
-cat $(package)-$(version).tar \
| bzip2 > $(package)-$(version).tar.bz2 \
|| rm -f $(package)-$(version).tar.bz2
$(DO_GZIP) $(package)-$(version).tar
.test-spam:
$(CC) $(CFLAGS) $(CPPFLAGS) -o fakemail $(srcdir)/autoconf/scripts/fakemail.c
./fakemail \
$(srcdir)/test/tokenlist-non-spam 15 > .test-non-spam
./fakemail \
$(srcdir)/test/tokenlist-spam 15 > .test-spam
check test: $(package) .test-spam
@$(CC) -o mboxsplit $(srcdir)/autoconf/scripts/mboxsplit.c
@FAIL=0; PROG=./$(package); TESTDB=$(testdb); TESTFILE=$(testfile); \
export PROG TESTDB TESTFILE; \
test "x$$BACKENDS" = x && BACKENDS="$(testbackends)"; \
CREATEDTABLE=""; \
if test x$$MYSQLDB = x; then \
CREATEDTABLE=@PACKAGE@test$$RANDOM; \
if mysql test -Be "CREATE TABLE $$CREATEDTABLE ( key1 BIGINT UNSIGNED NOT NULL, key2 BIGINT UNSIGNED NOT NULL, token VARCHAR(64) DEFAULT '' NOT NULL, value1 INT UNSIGNED NOT NULL, value2 INT UNSIGNED NOT NULL, value3 INT UNSIGNED NOT NULL, PRIMARY KEY (key1,key2,token), KEY (key1), KEY (key2), KEY (token) );" >/dev/null 2>&1; then \
MYSQLDB="database=test;host=localhost;port=3306;user=$$USER;pass=;table=$$CREATEDTABLE;key1=0;key2=0"; \
else \
CREATEDTABLE=""; \
fi; \
fi; \
for BACKEND in $$BACKENDS; do \
SKIPMYSQL=0; \
test $$BACKEND = MySQL && test x$$MYSQLDB = x && SKIPMYSQL=1; \
test $$BACKEND = mysql && test x$$MYSQLDB = x && SKIPMYSQL=1; \
test $$SKIPMYSQL = 1 && echo \*\*\* MySQL tests disabled - define \$$MYSQLDB; \
test $$SKIPMYSQL = 1 && echo \*\*\* to enable \(see man page for spec format\); \
test $$SKIPMYSQL = 1 && continue; \
TESTDB=$(testdb); \
rm -f $$TESTDB; \
export BACKEND; \
test $$BACKEND = MySQL && TESTDB=$$MYSQLDB; \
test $$BACKEND = mysql && TESTDB=$$MYSQLDB; \
for SCRIPT in $(srcdir)/test/t[0-9]*; do \
test -f $$SCRIPT || continue; \
sed -n 's/^# *TEST: */'"$$BACKEND"': /p' < $$SCRIPT | tr "\n" ' '; \
STATUS=0; \
sh -e $$SCRIPT || STATUS=1; \
test $$STATUS -eq 1 && FAIL=1; \
test $$STATUS -eq 1 && echo FAILED || echo OK; \
done; rm -f $$TESTDB $$TESTFILE; \
done; \
test x$$CREATEDTABLE = x || mysql test -Be "DROP TABLE $$CREATEDTABLE;" >/dev/null 2>&1; \
exit $$FAIL
memtest: $(package) .test-spam
@which valgrind >/dev/null 2>/dev/null || (\
echo These tests require valgrind to be installed.; \
echo See http://valgrind.kde.org/ for details.; \
exit 1; \
)
@$(CC) -o mboxsplit $(srcdir)/autoconf/scripts/mboxsplit.c
@FAIL=0; \
PROG="valgrind --tool=memcheck --leak-check=yes --db-attach=no ./$(package)"; \
TESTDB=$(testdb); TESTFILE=$(testfile); \
export PROG TESTDB TESTFILE; \
test "x$$BACKENDS" = x && BACKENDS="$(testbackends)"; \
CREATEDTABLE=""; \
if test x$$MYSQLDB = x; then \
CREATEDTABLE=@PACKAGE@test$$RANDOM; \
if mysql test -Be "CREATE TABLE $$CREATEDTABLE ( key1 BIGINT UNSIGNED NOT NULL, key2 BIGINT UNSIGNED NOT NULL, token VARCHAR(64) DEFAULT '' NOT NULL, value1 INT UNSIGNED NOT NULL, value2 INT UNSIGNED NOT NULL, value3 INT UNSIGNED NOT NULL, PRIMARY KEY (key1,key2,token), KEY (key1), KEY (key2), KEY (token) );" >/dev/null 2>&1; then \
MYSQLDB="database=test;host=localhost;port=3306;user=$$USER;pass=;table=$$CREATEDTABLE;key1=0;key2=0"; \
else \
CREATEDTABLE=""; \
fi; \
fi; \
for BACKEND in $$BACKENDS; do \
SKIPMYSQL=0; \
test $$BACKEND = MySQL && test x$$MYSQLDB = x && SKIPMYSQL=1; \
test $$BACKEND = mysql && test x$$MYSQLDB = x && SKIPMYSQL=1; \
test $$SKIPMYSQL = 1 && echo \*\*\* MySQL tests disabled - define \$$MYSQLDB; \
test $$SKIPMYSQL = 1 && echo \*\*\* to enable \(see man page for spec format\); \
test $$SKIPMYSQL = 1 && continue; \
TESTDB=$(testdb); \
rm -f $$TESTDB; \
export BACKEND; \
test $$BACKEND = MySQL && TESTDB=$$MYSQLDB; \
test $$BACKEND = mysql && TESTDB=$$MYSQLDB; \
for SCRIPT in $(srcdir)/test/t[0-9]*; do \
test -f $$SCRIPT || continue; \
TESTOUT=memtest-out-$$BACKEND-`basename $$SCRIPT`; \
sed -n 's/^# *TEST: */'"$$BACKEND"': /p' < $$SCRIPT | tr "\n" ' '; \
STATUS=0; \
sh -e $$SCRIPT >$$TESTOUT 2>&1 || STATUS=1; \
grep '^==[0-9]\+== ERROR SUMMARY: ' $$TESTOUT \
| grep -q -v '^==[0-9]\+== ERROR SUMMARY: 0 ' && STATUS=2; \
test $$STATUS -eq 1 && FAIL=1; \
test $$STATUS -eq 2 && FAIL=1; \
test $$STATUS -eq 1 && echo FAILED, MEMTEST OK; \
test $$STATUS -eq 2 && echo FAILED MEMTEST; \
test $$STATUS -eq 0 && echo OK; \
done; rm -f $$TESTDB $$TESTFILE; \
done; \
test x$$CREATEDTABLE = x || mysql test -Be "DROP TABLE $$CREATEDTABLE;" >/dev/null 2>&1; \
exit $$FAIL
.test-benchmark-spam:
$(CC) $(CFLAGS) $(CPPFLAGS) -o fakemail $(srcdir)/autoconf/scripts/fakemail.c
./fakemail \
$(srcdir)/test/tokenlist-non-spam 1500 > .test-benchmark-non-spam
./fakemail \
$(srcdir)/test/tokenlist-spam 1500 > .test-benchmark-spam
benchmark: $(package) .test-benchmark-spam
@test "x$$BACKENDS" = x && BACKENDS="$(testbackends)"; \
CREATEDTABLE=""; \
if test x$$MYSQLDB = x; then \
CREATEDTABLE=@PACKAGE@test$$RANDOM; \
if mysql test -Be "CREATE TABLE $$CREATEDTABLE ( key1 BIGINT UNSIGNED NOT NULL, key2 BIGINT UNSIGNED NOT NULL, token VARCHAR(64) DEFAULT '' NOT NULL, value1 INT UNSIGNED NOT NULL, value2 INT UNSIGNED NOT NULL, value3 INT UNSIGNED NOT NULL, PRIMARY KEY (key1,key2,token), KEY (key1), KEY (key2), KEY (token) );" >/dev/null 2>&1; then \
MYSQLDB="database=test;host=localhost;port=3306;user=$$USER;pass=;table=$$CREATEDTABLE;key1=0;key2=0"; \
else \
CREATEDTABLE=""; \
fi; \
fi; \
for BACKEND in $$BACKENDS; do \
SKIPMYSQL=0; \
test $$BACKEND = MySQL && test x$$MYSQLDB = x && SKIPMYSQL=1; \
test $$BACKEND = mysql && test x$$MYSQLDB = x && SKIPMYSQL=1; \
test $$SKIPMYSQL = 1 && echo \*\*\* MySQL tests disabled - define \$$MYSQLDB; \
test $$SKIPMYSQL = 1 && echo \*\*\* to enable \(see man page for spec format\); \
test $$SKIPMYSQL = 1 && continue; \
TESTDB=$$BACKEND; \
test $$BACKEND = MySQL && TESTDB=mysql:$$MYSQLDB; \
test $$BACKEND = mysql && TESTDB=mysql:$$MYSQLDB; \
./$(package) -d $$TESTDB -B .test-benchmark-spam .test-benchmark-non-spam; \
done; \
test x$$CREATEDTABLE = x || mysql test -Be "DROP TABLE $$CREATEDTABLE;" >/dev/null 2>&1
bigbenchmark: $(package) .test-benchmark-spam
rm -f benchmark-report benchmark-data-* benchmark-graph-*
test "x$$BACKENDS" = x && BACKENDS="$(testbackends)"; \
echo "set title 'Training times'" > benchmark-graph-train; \
echo "set title 'Classification times'" > benchmark-graph-class; \
echo "set title 'Accuracy of trained database'" > benchmark-graph-accuracy; \
for GRAPH in train class accuracy; do \
CMD=""; \
for BACKEND in $$BACKENDS; do \
CMD="$$CMD, 'benchmark-data-$$GRAPH-$$BACKEND' title '$$BACKEND' with linespoints"; \
done; \
echo "$$CMD" | sed -e 's/^,/plot/' -e 's/SQLite2/sqlite/g' >> benchmark-graph-$$GRAPH; \
done
for NUM in 5 6 7 8 9 10 11 12 14 16 18 20 22 24 26 28 30 35 40 45 50 55 60 65 70 80 90 100 120 140 160 180 200 250 300 350 400 450 500 600 700 800 900 1000 1100 1200 1300 1400 1500; do \
./fakemail $(srcdir)/test/tokenlist-non-spam $$NUM > .test-benchmark-non-spam; \
./fakemail $(srcdir)/test/tokenlist-spam $$NUM > .test-benchmark-spam; \
$(MAKE) benchmark | tee benchmark-report; \
awk -f $(srcdir)/autoconf/scripts/benchmark.awk < benchmark-report; \
$(MAKE) benchmark | tee benchmark-report; \
awk -f $(srcdir)/autoconf/scripts/benchmark.awk < benchmark-report; \
$(MAKE) benchmark | tee benchmark-report; \
awk -f $(srcdir)/autoconf/scripts/benchmark.awk < benchmark-report; \
done
@echo
@echo 'You can now load the files "benchmark-graph-*" into "gnuplot".'
@echo
install: all doc
$(srcdir)/autoconf/scripts/mkinstalldirs \
"$(DESTDIR)/$(bindir)"
$(srcdir)/autoconf/scripts/mkinstalldirs \
"$(DESTDIR)/$(mandir)/man1"
$(INSTALL) -m 755 $(package) \
"$(DESTDIR)/$(bindir)/$(package)"
$(INSTALL) -m 644 doc/quickref.1 \
"$(DESTDIR)/$(mandir)/man1/$(package).1"
$(DO_GZIP) "$(DESTDIR)/$(mandir)/man1/$(package).1" || :
uninstall:
$(UNINSTALL) "$(DESTDIR)/$(bindir)/$(package)"
$(UNINSTALL) "$(DESTDIR)/$(mandir)/man1/$(package).1"
$(UNINSTALL) "$(DESTDIR)/$(mandir)/man1/$(package).1.gz"
rpmbuild:
echo macrofiles: `rpm --showrc \
| grep ^macrofiles \
| cut -d : -f 2- \
| sed 's,^[^/]*/,/,'`:`pwd`/rpmmacros > rpmrc
echo %_topdir `pwd`/rpm > rpmmacros
rm -rf rpm
mkdir rpm
mkdir rpm/SPECS rpm/BUILD rpm/SOURCES rpm/RPMS rpm/SRPMS
-cat /usr/lib/rpm/rpmrc /etc/rpmrc $$HOME/.rpmrc \
| grep -hsv ^macrofiles \
>> rpmrc
srpm:
-test -e $(package)-$(version).tar.gz || $(MAKE) dist
-test -e rpmrc || $(MAKE) rpmbuild
rpmbuild $(RPMFLAGS) --rcfile=rpmrc -ts $(package)-$(version).tar.bz2
mv rpm/SRPMS/*$(package)-*.rpm .
rm -rf rpm rpmmacros rpmrc
rpm:
-test -e $(package)-$(version).tar.gz || $(MAKE) dist
-test -e rpmrc || $(MAKE) rpmbuild
rpmbuild $(RPMFLAGS) --rcfile=rpmrc -tb $(package)-$(version).tar.bz2
rpmbuild $(RPMFLAGS) --rcfile=rpmrc -tb --with static $(package)-$(version).tar.bz2
mv rpm/RPMS/*/$(package)-*.rpm .
rm -rf rpm rpmmacros rpmrc
deb: dist
rm -rf BUILD-DEB
mkdir BUILD-DEB
cd BUILD-DEB && tar xzf ../$(package)-$(version).tar.gz
cd BUILD-DEB && cd $(package)-$(version) && dpkg-buildpackage -rfakeroot
mv BUILD-DEB/*.deb .
rm -rf BUILD-DEB
release: dist rpm srpm
qsf-1.2.7/autoconf/make/modules.mk 0000644 0000764 0000764 00000003652 10665021314 014662 0 ustar aw aw # Automatically generated module linking rules
#
# Creation time: Tue Aug 28 14:27:40 BST 2007
src/db.o: src/db/btree.o src/db/gdbm.o src/db/list.o src/db/main.o src/db/mysql.o src/db/obtree.o src/db/sqlite.o
$(LD) $(LDFLAGS) -o $@ src/db/btree.o src/db/gdbm.o src/db/list.o src/db/main.o src/db/mysql.o src/db/obtree.o src/db/sqlite.o
src/main.o: src/main/help.o src/main/log.o src/main/main.o src/main/options.o src/main/tick.o src/main/version.o
$(LD) $(LDFLAGS) -o $@ src/main/help.o src/main/log.o src/main/main.o src/main/options.o src/main/tick.o src/main/version.o
src/spam.o: src/spam/alloc.o src/spam/allowlist.o src/spam/benchmark.o src/spam/check.o src/spam/cksum.o src/spam/db.o src/spam/dump.o src/spam/merge.o src/spam/plaintext.o src/spam/prune.o src/spam/token.o src/spam/train.o src/spam/update.o
$(LD) $(LDFLAGS) -o $@ src/spam/alloc.o src/spam/allowlist.o src/spam/benchmark.o src/spam/check.o src/spam/cksum.o src/spam/db.o src/spam/dump.o src/spam/merge.o src/spam/plaintext.o src/spam/prune.o src/spam/token.o src/spam/train.o src/spam/update.o
src/message.o: src/message/alloc.o src/message/base64.o src/message/dump.o src/message/header.o src/message/parse.o src/message/qp.o src/message/read.o src/message/rfc2047.o
$(LD) $(LDFLAGS) -o $@ src/message/alloc.o src/message/base64.o src/message/dump.o src/message/header.o src/message/parse.o src/message/qp.o src/message/read.o src/message/rfc2047.o
src/tests.o: src/tests/attached_files.o src/tests/gibberish.o src/tests/gtube.o src/tests/html.o src/tests/imgcount.o src/tests/main.o src/tests/urls.o
$(LD) $(LDFLAGS) -o $@ src/tests/attached_files.o src/tests/gibberish.o src/tests/gtube.o src/tests/html.o src/tests/imgcount.o src/tests/main.o src/tests/urls.o
src/mailbox.o: src/mailbox/alloc.o src/mailbox/count.o src/mailbox/scan.o src/mailbox/select.o
$(LD) $(LDFLAGS) -o $@ src/mailbox/alloc.o src/mailbox/count.o src/mailbox/scan.o src/mailbox/select.o
qsf-1.2.7/autoconf/make/link.mk 0000644 0000764 0000764 00000000433 07721207662 014154 0 ustar aw aw #
# Targets.
#
#
mainobjs := src/main.o src/md5.o src/library.o src/db.o src/message.o src/mailbox.o src/spam.o src/tests.o
$(package): $(mainobjs)
$(CC) $(CFLAGS) -o $@ $(mainobjs) $(LIBS)
$(package)-static: $(mainobjs)
$(CC) $(CFLAGS) -static -o $@ $(mainobjs) $(LIBS)
# EOF
qsf-1.2.7/autoconf/Makefile.in 0000644 0000764 0000764 00000001011 10240610410 013762 0 ustar aw aw #
# Files from which this is generated (inside directory `autoconf/make'):
#
# package.mk # package name and distribution details
# vars.mk # compilation, shell and linking variables
# filelist.mk # lists of files (autogenerated)
# unreal.mk # phony targets
# modules.mk # module linking rules (autogenerated)
# rules.mk # compilation rules
# link.mk # real top-level targets
# depend.mk # dependencies (autogenerated)
#
#
qsf-1.2.7/autoconf/scripts/ 0000755 0000764 0000764 00000000000 10665021416 013430 5 ustar aw aw qsf-1.2.7/autoconf/scripts/benchmark.awk 0000755 0000764 0000764 00000000761 10516505176 016102 0 ustar aw aw #!/bin/awk -f
/Backend type:/ { backend=$NF; }
/Counting messages/ { msgcount=$NF; }
/during training/ { tnext=1; }
/during classification/ { tnext=2; }
/Total time/ {
if (tnext == 1) {
time_train=$NF;
} else {
time_class=$NF;
}
}
/Accuracy:/ {
accuracy=$2;
printf "%d %f\n", msgcount, time_train >>"benchmark-data-train-"backend;
printf "%d %f\n", msgcount, time_class >>"benchmark-data-class-"backend;
printf "%d %f\n", msgcount, accuracy >>"benchmark-data-accuracy-"backend;
}
# EOF
qsf-1.2.7/autoconf/scripts/install.sh 0000755 0000764 0000764 00000012721 07611763233 015447 0 ustar aw aw #! /bin/sh
#
# install - install a program, script, or datafile
# This comes from X11R5 (mit/util/scripts/install.sh).
#
# Copyright 1991 by the Massachusetts Institute of Technology
#
# Permission to use, copy, modify, distribute, and sell this software and its
# documentation for any purpose is hereby granted without fee, provided that
# the above copyright notice appear in all copies and that both that
# copyright notice and this permission notice appear in supporting
# documentation, and that the name of M.I.T. not be used in advertising or
# publicity pertaining to distribution of the software without specific,
# written prior permission. M.I.T. makes no representations about the
# suitability of this software for any purpose. It is provided "as is"
# without express or implied warranty.
#
# Calling this script install-sh is preferred over install.sh, to prevent
# `make' implicit rules from creating a file called install from it
# when there is no Makefile.
#
# This script is compatible with the BSD install script, but was written
# from scratch. It can only install one file at a time, a restriction
# shared with many OS's install programs.
# set DOITPROG to echo to test this script
# Don't use :- since 4.3BSD and earlier shells don't like it.
doit="${DOITPROG-}"
# put in absolute paths if you don't have them in your path; or use env. vars.
mvprog="${MVPROG-mv}"
cpprog="${CPPROG-cp}"
chmodprog="${CHMODPROG-chmod}"
chownprog="${CHOWNPROG-chown}"
chgrpprog="${CHGRPPROG-chgrp}"
stripprog="${STRIPPROG-strip}"
rmprog="${RMPROG-rm}"
mkdirprog="${MKDIRPROG-mkdir}"
transformbasename=""
transform_arg=""
instcmd="$mvprog"
chmodcmd="$chmodprog 0755"
chowncmd=""
chgrpcmd=""
stripcmd=""
rmcmd="$rmprog -f"
mvcmd="$mvprog"
src=""
dst=""
dir_arg=""
while [ x"$1" != x ]; do
case $1 in
-c) instcmd="$cpprog"
shift
continue;;
-d) dir_arg=true
shift
continue;;
-m) chmodcmd="$chmodprog $2"
shift
shift
continue;;
-o) chowncmd="$chownprog $2"
shift
shift
continue;;
-g) chgrpcmd="$chgrpprog $2"
shift
shift
continue;;
-s) stripcmd="$stripprog"
shift
continue;;
-t=*) transformarg=`echo $1 | sed 's/-t=//'`
shift
continue;;
-b=*) transformbasename=`echo $1 | sed 's/-b=//'`
shift
continue;;
*) if [ x"$src" = x ]
then
src=$1
else
# this colon is to work around a 386BSD /bin/sh bug
:
dst=$1
fi
shift
continue;;
esac
done
if [ x"$src" = x ]
then
echo "install: no input file specified"
exit 1
else
true
fi
if [ x"$dir_arg" != x ]; then
dst=$src
src=""
if [ -d $dst ]; then
instcmd=:
else
instcmd=mkdir
fi
else
# Waiting for this to be detected by the "$instcmd $src $dsttmp" command
# might cause directories to be created, which would be especially bad
# if $src (and thus $dsttmp) contains '*'.
if [ -f $src -o -d $src ]
then
true
else
echo "install: $src does not exist"
exit 1
fi
if [ x"$dst" = x ]
then
echo "install: no destination specified"
exit 1
else
true
fi
# If destination is a directory, append the input filename; if your system
# does not like double slashes in filenames, you may need to add some logic
if [ -d $dst ]
then
dst="$dst"/`basename $src`
else
true
fi
fi
## this sed command emulates the dirname command
dstdir=`echo $dst | sed -e 's,[^/]*$,,;s,/$,,;s,^$,.,'`
# Make sure that the destination directory exists.
# this part is taken from Noah Friedman's mkinstalldirs script
# Skip lots of stat calls in the usual case.
if [ ! -d "$dstdir" ]; then
defaultIFS='
'
IFS="${IFS-${defaultIFS}}"
oIFS="${IFS}"
# Some sh's can't handle IFS=/ for some reason.
IFS='%'
set - `echo ${dstdir} | sed -e 's@/@%@g' -e 's@^%@/@'`
IFS="${oIFS}"
pathcomp=''
while [ $# -ne 0 ] ; do
pathcomp="${pathcomp}${1}"
shift
if [ ! -d "${pathcomp}" ] ;
then
$mkdirprog "${pathcomp}"
else
true
fi
pathcomp="${pathcomp}/"
done
fi
if [ x"$dir_arg" != x ]
then
$doit $instcmd $dst &&
if [ x"$chowncmd" != x ]; then $doit $chowncmd $dst; else true ; fi &&
if [ x"$chgrpcmd" != x ]; then $doit $chgrpcmd $dst; else true ; fi &&
if [ x"$stripcmd" != x ]; then $doit $stripcmd $dst; else true ; fi &&
if [ x"$chmodcmd" != x ]; then $doit $chmodcmd $dst; else true ; fi
else
# If we're going to rename the final executable, determine the name now.
if [ x"$transformarg" = x ]
then
dstfile=`basename $dst`
else
dstfile=`basename $dst $transformbasename |
sed $transformarg`$transformbasename
fi
# don't allow the sed command to completely eliminate the filename
if [ x"$dstfile" = x ]
then
dstfile=`basename $dst`
else
true
fi
# Make a temp file name in the proper directory.
dsttmp=$dstdir/#inst.$$#
# Move or copy the file name to the temp name
$doit $instcmd $src $dsttmp &&
trap "rm -f ${dsttmp}" 0 &&
# and set any options; do chmod last to preserve setuid bits
# If any of these fail, we abort the whole thing. If we want to
# ignore errors from any of these, just make sure not to ignore
# errors from the above "$doit $instcmd $src $dsttmp" command.
if [ x"$chowncmd" != x ]; then $doit $chowncmd $dsttmp; else true;fi &&
if [ x"$chgrpcmd" != x ]; then $doit $chgrpcmd $dsttmp; else true;fi &&
if [ x"$stripcmd" != x ]; then $doit $stripcmd $dsttmp; else true;fi &&
if [ x"$chmodcmd" != x ]; then $doit $chmodcmd $dsttmp; else true;fi &&
# Now rename the file to the real destination.
$doit $rmcmd -f $dstdir/$dstfile &&
$doit $mvcmd $dsttmp $dstdir/$dstfile
fi &&
exit 0
qsf-1.2.7/autoconf/scripts/makemake.sh 0000755 0000764 0000764 00000006340 10466173113 015546 0 ustar aw aw #!/bin/sh
#
# Generate Makefile dependencies inclusion and module target file "depend.mk"
# by scanning the directory "src" for files ending in ".c" and ".d", and for
# subdirectories not starting with "_".
#
# Modules live inside subdirectories called [^_]* - i.e. a directory "foo" will
# have a rule created which links all code inside it to "foo.o".
#
# The directory "src/include" is never scanned; neither are CVS directories.
#
outlist=$1
outlink=$2
FIND=find
GREP=grep
which gfind 2>/dev/null | grep /gfind >/dev/null && FIND=gfind
which ggrep 2>/dev/null | grep /ggrep >/dev/null && GREP=ggrep
echo '# Automatically generated file listings' > $outlist
echo '#' >> $outlist
echo "# Creation time: `date`" >> $outlist
echo >> $outlist
echo '# Automatically generated module linking rules' > $outlink
echo '#' >> $outlink
echo "# Creation time: `date`" >> $outlink
echo >> $outlink
case `echo "testing\c"; echo 1,2,3`,`echo -n testing; echo 1,2,3` in
*c*,-n*) ECHO_N= ECHO_C='
' ECHO_T=' ' ;;
*c*,* ) ECHO_N=-n ECHO_C= ECHO_T= ;;
*) ECHO_N= ECHO_C='\c' ECHO_T= ;;
esac
echo $ECHO_N "Scanning for source files: $ECHO_C"
allsrc=`$FIND src -type f -name "*.c" -print`
allobj=`echo $allsrc | tr ' ' '\n' | sed 's/\.c$/.o/'`
alldep=`echo $allsrc | tr ' ' '\n' | sed 's/\.c$/.d/'`
echo `echo $allsrc | wc -w | tr -d ' '` found
echo $ECHO_N "Scanning for modules: $ECHO_C"
modules=`$FIND src -type d -print \
| $GREP -v '^src$' \
| $GREP -v '/_' \
| $GREP -v '^src/include' \
| $GREP -v 'CVS' \
| while read DIR; do \
CONTENT=\$(/bin/ls -d \$DIR/* \
| $GREP -v '.po$' \
| $GREP -v '.gmo$' \
| $GREP -v '.mo$' \
| $GREP -v '.h$' \
| sed -n '$p'); \
[ -n "\$CONTENT" ] || continue; \
echo \$DIR; \
done
`
echo up to `echo $modules | wc -w | tr -d ' '` found
echo "Writing module linking rules"
echo $ECHO_N "[$ECHO_C"
for i in $modules; do echo $ECHO_N " $ECHO_C"; done
echo $ECHO_N -e "]\r[$ECHO_C"
for i in $modules; do
echo $ECHO_N ".$ECHO_C"
allobj="$allobj $i.o"
deps=""
for j in $i/*.c; do
[ -f $j ] || continue
newobj=`echo $j | sed -e 's@\.c$@.o@'`
deps="$deps $newobj"
done
for j in $i/*; do
[ -d "$j" ] || continue
[ `basename $j` = "CVS" ] && continue
CONTENT=`/bin/ls -d $j/* \
| $GREP -v '.po$' \
| $GREP -v '.gmo$' \
| $GREP -v '.mo$' \
| $GREP -v '.h$' \
| sed -n '$p'`
[ -n "$CONTENT" ] || continue
deps="$deps $j.o"
done
[ -n "$deps" ] || continue
echo "$i.o: $deps" >> $outlink
echo ' $(LD) $(LDFLAGS) -o $@' "$deps" >> $outlink
echo >> $outlink
done
echo ']'
echo "Listing source, object and dependency files"
echo $ECHO_N "allsrc = $ECHO_C" >> $outlist
echo $allsrc | sed 's,src/nls/cat-id-tbl.c,,' | sed -e 's/ / \\!/g'\
| tr '!' '\n' >> $outlist
echo >> $outlist
echo $ECHO_N "allobj = $ECHO_C" >> $outlist
echo $allobj | sed -e 's/ / \\!/g' | tr '!' '\n' >> $outlist
echo >> $outlist
echo $ECHO_N "alldep = $ECHO_C" >> $outlist
echo $alldep | sed -e 's/ / \\!/g' | tr '!' '\n' >> $outlist
echo >> $outlist
echo >> $outlink
# EOF
qsf-1.2.7/autoconf/scripts/mkinstalldirs 0000755 0000764 0000764 00000001261 07611763233 016245 0 ustar aw aw #!/bin/sh
# mkinstalldirs --- make directory hierarchy
# Author: Noah Friedman
# Created: 1993-05-16
# Last modified: 1994-03-25
# Public domain
errstatus=0
for file in ${1+"$@"} ; do
set fnord `echo ":$file" | sed -ne 's/^:\//#/;s/^://;s/\// /g;s/^#/\//;p'`
shift
pathcomp=
for d in ${1+"$@"} ; do
pathcomp="$pathcomp$d"
case "$pathcomp" in
-* ) pathcomp=./$pathcomp ;;
esac
if test ! -d "$pathcomp"; then
echo "mkdir $pathcomp" 1>&2
mkdir "$pathcomp" || errstatus=$?
chmod 755 $pathcomp 2>/dev/null
fi
pathcomp="$pathcomp/"
done
done
exit $errstatus
# mkinstalldirs ends here
qsf-1.2.7/autoconf/scripts/depend.sh 0000755 0000764 0000764 00000000726 07667346351 015253 0 ustar aw aw #!/bin/sh
#
# Generate dependencies for a C source file.
CC=$1
shift
file=$1
shift
stem=$1
shift
srcdir=$1
abssrc=`echo $srcdir | sed ':1
s,^\./,,g
t1'`
shift
abssrc=`echo "$abssrc" | sed 's,\\.,\\\\.,g'`
srcdir=`echo "$srcdir" | sed 's,\\.,\\\\.,g'`
$CC -M -MG $* $file \
| sed -e 's, /[^ ]*,,g' -e "s,^.*\.o:,${stem}.d ${stem}.o:," \
-e '/^ \\$/d' -e 's/ \\$//' \
-e 's,'"$srcdir"'/,,g' -e 's,'"$abssrc"'/,,g' \
| tr '\n' ' ' \
| tr -s ' '
echo
# EOF
qsf-1.2.7/autoconf/scripts/fakemail.c 0000755 0000764 0000764 00000004700 10554714253 015356 0 ustar aw aw /*
* Small program to generate a fake email given a wordlist to choose words
* from.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include
#include
#include
#include
#include
#include "config.h"
/*
* Return a random word from the given word list.
*/
char *random_word(char **wordlist, int wordcount)
{
return wordlist[rand() % wordcount];
}
/*
* Output between "min" and "max" (inclusive) random words to stdout.
*/
void random_words(char **wordlist, int wordcount, int wmin, int wmax)
{
int numtodo, i;
numtodo = wmin + (rand() % (wmax - wmin));
for (i = 0; i < numtodo; i++) {
if (i > 0) printf(" ");
printf("%s", random_word(wordlist, wordcount));
}
}
/*
* Main program.
*/
int main(int argc, char **argv) {
char **wordlist = NULL;
char **newptr;
char *strptr;
int wordcount = 0;
char buf[1024]; /* RATS: ignore (checked) */
char *suffix[] = {
".co.uk",
".com",
".net",
".org",
".org.uk"
};
int mailnum;
time_t t;
FILE *fptr;
if (argc != 3) {
fprintf(stderr, "Usage: fakemail WORDFILE NUMMESSAGES\n");
return(1);
}
fptr = fopen(argv[1], "r");
if (!fptr) {
fprintf(stderr, "fakemail: %s: %s\n", argv[1], strerror(errno));
return(1);
}
while (fgets(buf, sizeof(buf) - 1, fptr)) {
wordcount++;
if (wordlist) {
newptr = realloc(wordlist, wordcount * sizeof(char *)); /* RATS: ignore */
} else {
newptr = malloc(wordcount * sizeof(char *));
}
if (!newptr) {
fprintf(stderr, "fakemail: %s\n", strerror(errno));
fclose(fptr);
return (1);
}
strptr = strchr(buf, '\n');
if (strptr) *strptr = 0;
wordlist = newptr;
wordlist[wordcount-1] = strdup(buf);
}
fclose(fptr);
srand(time(NULL)); /* RATS: ignore (randomness not important) */
for (mailnum = 0; mailnum < atoi(argv[2]); mailnum++) {
#ifdef HAVE_SNPRINTF
snprintf(buf, sizeof(buf),
#else
sprintf(buf, /* RATS: ignore */
#endif
"%s.%s@%s%s",
random_word(wordlist, wordcount),
random_word(wordlist, wordcount),
random_word(wordlist, wordcount),
suffix[rand()%5]);
t = time(NULL);
printf("From %s %s", buf, ctime(&t));
printf("Return-Path: <%s>\n", buf);
printf("From: <%s>\n", buf);
printf("To: you@your.com\n");
printf("Date: %s", ctime(&t));
printf("Subject: ");
random_words(wordlist, wordcount, 2, 10);
printf("\n\n");
random_words(wordlist, wordcount, 20, 400);
printf("\n\n");
}
return(0);
}
/* EOF */
qsf-1.2.7/autoconf/scripts/mboxsplit.c 0000755 0000764 0000764 00000001505 10554714254 015627 0 ustar aw aw /*
* Small program to output the Nth message from an mbox file on stdin.
*
* Copyright 2007 Andrew Wood, distributed under the Artistic License.
*/
#include
#include
#include
/*
* Main program.
*/
int main(int argc, char **argv)
{
int dispmsg, prevnl, msgnum;
char buf[1024]; /* RATS: ignore (checked) */
if (argc != 2) {
fprintf(stderr, "Usage: mboxsplit MESSAGENUM\n");
return(1);
}
dispmsg = atoi(argv[1]);
prevnl = 1;
msgnum = 0;
while (fgets(buf, sizeof(buf) - 1, stdin)) {
if (prevnl && (strncmp(buf, "From ", 5) == 0)) {
msgnum++;
prevnl = 0;
} else if (buf[0] == '\n') {
prevnl = 1;
} else if ((buf[0] == '\r') && (buf[1] == '\n')) {
prevnl = 1;
} else {
prevnl = 0;
}
if (msgnum == dispmsg)
printf("%s", buf);
}
return(0);
}
/* EOF */
qsf-1.2.7/autoconf/scripts/index.sh 0000755 0000764 0000764 00000012363 10466173207 015110 0 ustar aw aw #!/bin/ash
#
# Script to generate an HTML index of all C code from the current directory
# downwards (skipping directories ending in ~). The header comment in each
# file is listed, and each function's prototype and comment are given. A
# list of "TODO:" comments is also generated.
#
# Outputs the HTML on standard output.
#
# If a parameter is given, it is the prefix to put before any "view file"
# links, eg ".." to link to "../dir/file.c" instead of "dir/file.c".
#
# Skips any files containing the string !NOINDEX.
#
# Requires ctags and cproto.
OFFS=$1
case `echo "testing\c"; echo 1,2,3`,`echo -n testing; echo 1,2,3` in
*c*,-n*) ECHO_N= ECHO_C='
' ECHO_T=' ' ;;
*c*,* ) ECHO_N=-n ECHO_C= ECHO_T= ;;
*) ECHO_N= ECHO_C='\c' ECHO_T= ;;
esac
# Convert the given string to HTML-escaped values (<, >, & escaped) on
# stdout.
#
html_safe () {
echo "$*" \
| sed -e 's|&|\&|g;s|<|\<|g;s|>|\>|g'
}
# Convert the given string to HTML-escaped values (<, >, & escaped) on
# stdout, also adding a to the end of each line.
#
html_safebr () {
echo "$*" \
| sed -e 's|&|\&|g;s|<|\<|g;s|>|\>|g;s/$/ /'
}
ALLFILES=`find . -name '*~' -prune -o -type f -name '*.c' \
-exec grep -FL '!NOINDEX' /dev/null '{}' ';'`
CTAGDATA=`echo "$ALLFILES" \
| ctags -nRf- -L- --c-types=f \
| sed 's/ .\// /;s/;" .*$//'`
FILELIST=`echo "$CTAGDATA" | cut -d ' ' -f 2 | sort | uniq`
echo ''
echo 'Source Code Index '
echo ''
echo ''
echo '
'
echo ''
echo '
'
echo "$FILELIST" \
| sed -e \
's|^.*$|\0
|'
echo ' '
for FILE in $FILELIST; do
DIR=`dirname $FILE`
FUNCDEFS=`cproto -f1 -I. -Isrc/include -I$DIR $FILE 2>/dev/null \
| sed -n 's/^.*[ *]\([^ *(]*\)(.*$/\1/p'`
FILEHEAD="`sed -n -e \
'1,/\*\//{/\/\*/,/\*\//{s/^[\/ *]//;s/^\*[\/]*//;p;};}' \
< $FILE`"
FILESHORTDESC=`echo "$FILEHEAD" | sed -n '1,/^ *$/{/^ *[^ ]*/p;}'`
FILELONGDESC=`echo "$FILEHEAD" | sed '1,/^ *$/d'`
echo '
'
echo '
'
echo ''
echo ''"$FILE"'
'
echo ' - '
echo ''`html_safe "$FILESHORTDESC"`' '
echo '
'
echo '[View File ]
'
echo '
'
echo "`html_safebr "$FILELONGDESC"`"
echo ' '
if [ -n "$FUNCDEFS" ]; then
echo 'Functions defined:
'
echo '
'
echo "$FUNCDEFS" \
| sed 's|^.*$|\0 |' \
| sed 's/^//;s|$|
|'
echo ' '
fi
echo '['
echo 'Top |'
echo 'To Do |'
echo 'Functions ]
'
done
echo ''
echo '
'
echo "$CTAGDATA" | while read FUNC FILE LINENUM REST; do
echo $ECHO_N ''"$ECHO_C"
echo $ECHO_N ''"$FUNC"'
'"$ECHO_C"
echo '['"$FILE"'
] '
done
echo ' '
echo "$CTAGDATA" | while read FUNC FILE LINENUM REST; do
FUNCDEF=`sed -n "$LINENUM,/{/p" < $FILE \
| tr '\n' ' ' \
| tr -d '{'`
LASTCOMLINE=`sed -n '1,'"$LINENUM"'{/\/\*/=;}' < $FILE | sed -n '$p'`
[ -z "$LASTCOMLINE" ] && LASTCOMLINE=1
LASTENDFUNCLINE=`sed -n '1,'"$LINENUM"'{/}/=;}' < $FILE | sed -n '$p'`
[ -z "$LASTENDFUNCLINE" ] && LASTENDFUNCLINE=1
FUNCHEAD="`sed -n -e \
"$LASTCOMLINE,"'/\*\//{h;s/^[\/ *]//;s/^\*[\/]*//;p;x;/\*\//q;}' \
< $FILE`"
[ "$LASTCOMLINE" -le "$LASTENDFUNCLINE" ] && FUNCHEAD=""
echo '
'
echo ''
echo $ECHO_N ''"$ECHO_C"
echo $ECHO_N "$FUNC"'
'"$ECHO_C"
echo $ECHO_N '['"$ECHO_C"
echo "$FILE"'
]'
echo '
'
echo ''"`html_safe "$FUNCDEF"`"'
'
echo '
'
echo "`html_safebr "$FUNCHEAD"`"
echo ' '
echo '['
echo 'Top |'
echo 'To Do |'
echo 'Files ]
'
done
echo ''
echo '
'
for FILE in $FILELIST; do
TODOLINES=`sed -n \
-e '/\/\*.*\*\//!{/\/\*/,/\*\//{/TODO:/{=;};};}' \
-e '/\/\*.*\*\//{/TODO:/{=;};}' \
< $FILE`
[ -z "$TODOLINES" ] && continue
echo $ECHO_N ''"$ECHO_C"
echo ''"$FILE"'
'
echo ''
for NUM in $TODOLINES; do
TODO=`sed -n "$NUM"'{s/^.*TODO://;s/\*\/.*$//;p;}' < $FILE`
echo "[$NUM ] `html_safe "$TODO"` "
done
echo ' '
done
echo ''
# EOF
qsf-1.2.7/autoconf/configure.in.orig 0000644 0000764 0000764 00000015442 10470054104 015211 0 ustar aw aw dnl Process this file with autoconf to produce a configure script.
dnl
AC_INIT(src/main/version.c)
dnl We're using C.
dnl
AC_LANG_C
dnl Output a header file.
dnl
AC_CONFIG_HEADER(src/include/config.h:autoconf/header.in)
dnl Set directory to check for Configure scripts in.
dnl
AC_CONFIG_AUX_DIR(autoconf/scripts)
dnl Read in package details.
dnl
PACKAGE=`cat $srcdir/doc/PACKAGE`
VERSION=`cat $srcdir/doc/VERSION`
UCPACKAGE=`tr a-z A-Z < $srcdir/doc/PACKAGE`
AC_SUBST(PACKAGE)
AC_SUBST(VERSION)
AC_SUBST(UCPACKAGE)
AC_DEFINE_UNQUOTED(PACKAGE, "$PACKAGE")
AC_DEFINE_UNQUOTED(PROGRAM_NAME, "$PACKAGE")
AC_DEFINE_UNQUOTED(VERSION, "$VERSION")
dnl Database backends we can use.
dnl
canuse_obtree="yes"
canuse_btree="yes"
canuse_gdbm="yes"
canuse_mysql="yes"
canuse_sqlite="yes"
dnl Check for compile-time options.
dnl
AC_ARG_ENABLE(debugging,
[ --enable-debugging compile with debugging symbols],
if test "$enable_debugging" = "yes"; then
CFLAGS="-g -Wall"
fi
)
AC_ARG_ENABLE(profiling,
[ --enable-profiling compile with profiling support],
if test "$enable_profiling" = "yes"; then
CFLAGS="-pg $CFLAGS"
fi
)
AC_ARG_ENABLE(static,
[ --enable-static enable static linking],
)
AC_ARG_WITH(obtree,
[ --without-obtree omit old binary tree backend support],
canuse_obtree="$with_obtree"
)
AC_ARG_WITH(btree,
[ --without-btree omit binary tree backend support],
canuse_btree="$with_btree"
)
AC_ARG_WITH(gdbm,
[ --without-gdbm omit GDBM backend support],
canuse_gdbm="$with_gdbm"
)
AC_ARG_WITH(mysql,
[ --without-mysql omit MySQL backend support],
canuse_mysql="$with_mysql"
)
AC_ARG_WITH(sqlite,
[ --without-sqlite omit SQLite v2.x backend support],
canuse_sqlite="$with_sqlite"
)
dnl Check for various programs.
dnl
CFLAGS=${CFLAGS-"-O2 -Wall -s"}
AC_PROG_CC
AC_PROG_CPP
AC_CHECK_TOOL(LD, ld, libtool --mode=link gcc)
AC_SUBST(LD)
AC_PROG_INSTALL
AC_PROG_MAKE_SET
AC_CHECK_PROG(DO_GZIP, gzip, gzip -f9, touch)
dnl Check for the maths library.
dnl
AC_SEARCH_LIBS(pow, m, , [AC_ERROR(maths library not found)])
dnl Check for various header files and set various other macros.
dnl
AC_DEFINE(HAVE_CONFIG_H)
AC_HEADER_STDC
AC_C_BIGENDIAN([AC_DEFINE(IS_BIG_ENDIAN)])
AC_CHECK_FUNCS(memcpy, , [AC_ERROR(the memcpy() function is required)])
AC_CHECK_FUNCS(fcntl getopt getopt_long mkstemp snprintf vsnprintf utime)
AC_CHECK_HEADERS(fcntl.h getopt.h limits.h sys/resource.h mcheck.h)
AC_FUNC_MMAP
dnl Check for backend databases and choose one.
dnl
PREVCFLAGS="$CFLAGS"
PREVCPPFLAGS="$CPPFLAGS"
PREVLIBS="$LIBS"
if test "x$canuse_mysql" = "xyes"; then
dnl
dnl First we try linking without the mysql_config libs, because on
dnl some systems that'll give us a dynamic library - the
dnl mysql_config libs often point us to static libraries.
dnl
dnl If static linking is enabled, we ONLY try the mysql_config libs.
dnl
CFLAGS=`mysql_config --cflags 2>/dev/null`
CPPFLAGS=`mysql_config --include 2>/dev/null` || CPPFLAGS="$CFLAGS"
LIBS=""
if test "x$enable_static" != "xyes"; then
AC_CHECK_HEADERS(mysql.h, [
AC_SEARCH_LIBS(mysql_real_connect, mysqlclient, [
AC_SEARCH_LIBS(mysql_real_escape_string, mysqlclient, , canuse_mysql="no")
], canuse_mysql="no")
], canuse_mysql="no")
fi
if test "x$canuse_mysql" = "xno"; then
canuse_mysql=yes
LIBS=`mysql_config --libs 2>/dev/null`
AC_CHECK_HEADERS(mysql.h, [
AC_SEARCH_LIBS(mysql_real_escape_string, mysqlclient, , canuse_mysql="no")
], canuse_mysql="no")
fi
if test "x$enable_static" = "xyes"; then
LIBS=`mysql_config --libs 2>/dev/null`
AC_CHECK_HEADERS(mysql.h, [
AC_SEARCH_LIBS(mysql_real_query, mysqlclient, , canuse_mysql="no")
], canuse_mysql="no")
fi
MYSQLCFLAGS="$CFLAGS"
MYSQLLIBS="$LIBS"
CFLAGS="$PREVCFLAGS"
CPPFLAGS="$PREVCPPFLAGS"
LIBS="$PREVLIBS"
fi
if test "x$canuse_gdbm" = "xyes"; then
PREVLIBS="$LIBS"
LIBS=""
AC_CHECK_HEADERS(gdbm.h, [
AC_SEARCH_LIBS(gdbm_open, gdbm, [
AC_CHECK_FUNCS(gdbm_fdesc)
], canuse_gdbm="no")
], canuse_gdbm="no")
dnl
dnl This is a hideous hack to try and find ".a" (static)
dnl library replacements if --enable-static is given.
dnl
if test "x$enable_static" = "xyes"; then
LIBDIR=`echo "$LIBS" | tr ' ' '\n' | sed -n 's/^-L//p'`
test -z "$LIBDIR" && LIBDIR=/usr/lib
LIBLIST=`echo "$LIBS" | tr ' ' '\n' | sed -n "s,^-l,$LIBDIR/lib,p" | sed -e 's,$,.a,'`
NEWLIBS=""
for i in $LIBLIST; do
test -e "$i" && NEWLIBS="$NEWLIBS $i"
done
test -n "$NEWLIBS" && LIBS="$NEWLIBS"
fi
GDBMLIBS="$LIBS"
LIBS="$PREVLIBS"
fi
if test "x$canuse_sqlite" = "xyes"; then
LIBS=""
AC_CHECK_HEADERS(sqlite.h, [
AC_SEARCH_LIBS(sqlite_open, sqlite, , canuse_sqlite="no")
], canuse_sqlite="no")
dnl Hideous hack as above.
if test "x$enable_static" = "xyes"; then
LIBDIR=`echo "$LIBS" | tr ' ' '\n' | sed -n 's/^-L//p'`
test -z "$LIBDIR" && LIBDIR=/usr/lib
LIBLIST=`echo "$LIBS" | tr ' ' '\n' | sed -n "s,^-l,$LIBDIR/lib,p" | sed -e 's,$,.a,'`
NEWLIBS=""
for i in $LIBLIST; do
test -e "$i" && NEWLIBS="$NEWLIBS $i"
done
test -n "$NEWLIBS" && LIBS="$NEWLIBS"
fi
SQLITELIBS="$LIBS"
LIBS="$PREVLIBS"
fi
AC_MSG_CHECKING(which backend databases are available)
BACKENDS=""
if test "x$canuse_obtree" = "xyes"; then
AC_DEFINE(USING_OBTREE)
BACKENDS="$BACKENDS obtree"
fi
if test "x$canuse_btree" = "xyes"; then
AC_DEFINE(USING_BTREE)
BACKENDS="$BACKENDS btree"
fi
if test "x$canuse_gdbm" = "xyes"; then
AC_DEFINE(USING_GDBM)
EXTRALIBS="$EXTRALIBS $GDBMLIBS"
BACKENDS="$BACKENDS GDBM"
fi
if test "x$canuse_mysql" = "xyes"; then
AC_DEFINE(USING_MYSQL)
EXTRALIBS="$EXTRALIBS $MYSQLLIBS"
CFLAGS="$CFLAGS $MYSQLCFLAGS"
BACKENDS="$BACKENDS MySQL"
fi
if test "x$canuse_sqlite" = "xyes"; then
AC_DEFINE(USING_SQLITE)
EXTRALIBS="$EXTRALIBS $SQLITELIBS"
CFLAGS="$CFLAGS $SQLITECFLAGS"
BACKENDS="$BACKENDS SQLite2"
fi
if test "x$BACKENDS" = "x"; then
AC_MSG_RESULT(none)
AC_MSG_ERROR(no usable database libraries found)
else
AC_MSG_RESULT($BACKENDS)
fi
LIBS="$LIBS $EXTRALIBS"
AC_DEFINE_UNQUOTED(BACKENDS, "$BACKENDS")
AC_SUBST(BACKENDS)
test -z "$INSTALL_DATA" && INSTALL_DATA='${INSTALL} -m 644'
AC_SUBST(INSTALL_DATA)
dnl Fudging for separate build directories.
dnl
subdirs=""
for i in `find $srcdir/src -type d -print | sed s,$srcdir/,,`; do
subdirs="$subdirs $i"
done
dnl Stitch together the Makefile fragments.
dnl
mk_segments="autoconf/Makefile.in"
for i in vars.mk package.mk filelist.mk unreal.mk modules.mk \
rules.mk link.mk depend.mk; do
mk_segments="$mk_segments:autoconf/make/$i"
done
dnl Output files (and create build directory structure too).
dnl
AC_OUTPUT(Makefile:$mk_segments doc/lsm:doc/lsm.in
doc/quickref.1:doc/quickref.1.in
doc/$PACKAGE.spec:doc/spec.in
src/.dummy:doc/NEWS,
rm -f src/.dummy
for i in $subdirs; do
test -d $i || mkdir $i
done
, subdirs="$subdirs")
dnl EOF
qsf-1.2.7/debian/ 0000755 0000764 0000764 00000000000 10665021416 011345 5 ustar aw aw qsf-1.2.7/debian/copyright 0000644 0000764 0000764 00000021727 10665021215 013306 0 ustar aw aw This package was debianized by Tom Parker on
Wed, 22 Jan 2003 15:36:15 +0000.
It was downloaded from http://www.ivarch.com/programs/qsf/
Upstream Author: Andrew Wood
Copyright Holder: (C) 2003-2007 Andrew Wood
License:
This package is free software, and is being distributed under the terms
of the Artistic License 2.0.
----------------------------------------------------------
Artistic License 2.0
Copyright (c) 2000-2006, The Perl Foundation.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed. Preamble
This license establishes the terms under which a given free software
Package may be copied, modified, distributed, and/or redistributed. The
intent is that the Copyright Holder maintains some artistic control
over the development of that Package while still keeping the Package
available as open source and free software.
You are always permitted to make arrangements wholly outside of this
license directly with the Copyright Holder of a given Package. If the
terms of this license do not permit the full use that you propose to
make of the Package, you should contact the Copyright Holder and seek
a different licensing arrangement. Definitions
"Copyright Holder" means the individual(s) or organization(s) named in
the copyright notice for the entire Package.
"Contributor" means any party that has contributed code or other material
to the Package, in accordance with the Copyright Holder's procedures.
"You" and "your" means any person who would like to copy, distribute,
or modify the Package.
"Package" means the collection of files distributed by the Copyright
Holder, and derivatives of that collection and/or of those files. A given
Package may consist of either the Standard Version, or a Modified Version.
"Distribute" means providing a copy of the Package or making it accessible
to anyone else, or in the case of a company or organization, to others
outside of your company or organization.
"Distributor Fee" means any fee that you charge for Distributing this
Package or providing support for this Package to another party. It does
not mean licensing fees.
"Standard Version" refers to the Package if it has not been modified,
or has been modified only in ways explicitly requested by the Copyright
Holder.
"Modified Version" means the Package, if it has been changed, and such
changes were not explicitly requested by the Copyright Holder.
"Original License" means this Artistic License as Distributed with the
Standard Version of the Package, in its current version or as it may be
modified by The Perl Foundation in the future.
"Source" form means the source code, documentation source, and
configuration files for the Package.
"Compiled" form means the compiled bytecode, object code, binary, or any
other form resulting from mechanical transformation or translation of
the Source form. Permission for Use and Modification Without Distribution
(1) You are permitted to use the Standard Version and create and use
Modified Versions for any purpose without restriction, provided that you
do not Distribute the Modified Version. Permissions for Redistribution
of the Standard Version
(2) You may Distribute verbatim copies of the Source form of the Standard
Version of this Package in any medium without restriction, either gratis
or for a Distributor Fee, provided that you duplicate all of the original
copyright notices and associated disclaimers. At your discretion, such
verbatim copies may or may not include a Compiled form of the Package.
(3) You may apply any bug fixes, portability changes, and other
modifications made available from the Copyright Holder. The resulting
Package will still be considered the Standard Version, and as such will
be subject to the Original License. Distribution of Modified Versions
of the Package as Source
(4) You may Distribute your Modified Version as Source (either gratis
or for a Distributor Fee, and with or without a Compiled form of the
Modified Version) provided that you clearly document how it differs
from the Standard Version, including, but not limited to, documenting
any non-standard features, executables, or modules, and provided that
you do at least ONE of the following:
(a) make the Modified Version available to the Copyright Holder of the
Standard Version, under the Original License, so that the Copyright Holder
may include your modifications in the Standard Version.
(b) ensure that installation of your Modified Version does not prevent the
user installing or running the Standard Version. In addition, the Modified
Version must bear a name that is different from the name of the Standard
Version.
(c) allow anyone who receives a copy of the Modified Version to make
the Source form of the Modified Version available to others under
(i) the Original License or
(ii) a license that permits the licensee to freely copy, modify and
redistribute the Modified Version using the same licensing terms that apply
to the copy that the licensee received, and requires that the Source form of
the Modified Version, and of any works derived from it, be made freely
available in that license fees are prohibited but Distributor Fees are
allowed. Distribution of Compiled Forms of the Standard Version or Modified
Versions without the Source
(5) You may Distribute Compiled forms of the Standard Version without
the Source, provided that you include complete instructions on how to
get the Source of the Standard Version. Such instructions must be valid
at the time of your distribution. If these instructions, at any time
while you are carrying out such distribution, become invalid, you must
provide new instructions on demand or cease further distribution. If
you provide valid instructions or cease distribution within thirty days
after you become aware that the instructions are invalid, then you do
not forfeit any of your rights under this license.
(6) You may Distribute a Modified Version in Compiled form without the
Source, provided that you comply with Section 4 with respect to the
Source of the Modified Version. Aggregating or Linking the Package
(7) You may aggregate the Package (either the Standard Version or
Modified Version) with other packages and Distribute the resulting
aggregation provided that you do not charge a licensing fee for the
Package. Distributor Fees are permitted, and licensing fees for other
components in the aggregation are permitted. The terms of this license
apply to the use and Distribution of the Standard or Modified Versions
as included in the aggregation.
(8) You are permitted to link Modified and Standard Versions with other
works, to embed the Package in a larger work of your own, or to build
stand-alone binary or bytecode versions of applications that include the
Package, and Distribute the result without restriction, provided the
result does not expose a direct interface to the Package. Items That
are Not Considered Part of a Modified Version
(9) Works (including, but not limited to, modules and scripts) that
merely extend or make use of the Package, do not, by themselves, cause
the Package to be a Modified Version. In addition, such works are not
considered parts of the Package itself, and are not subject to the terms
of this license. General Provisions
(10) Any use, modification, and distribution of the Standard or Modified
Versions is governed by this Artistic License. By using, modifying or
distributing the Package, you accept this license. Do not use, modify,
or distribute the Package, if you do not accept this license.
(11) If your Modified Version has been derived from a Modified Version
made by someone other than you, you are nevertheless required to ensure
that your Modified Version complies with the requirements of this license.
(12) This license does not grant you the right to use any trademark,
service mark, tradename, or logo of the Copyright Holder.
(13) This license includes the non-exclusive, worldwide, free-of-charge
patent license to make, have made, use, offer to sell, sell, import
and otherwise transfer the Package with respect to any patent claims
licensable by the Copyright Holder that are necessarily infringed by the
Package. If you institute patent litigation (including a cross-claim or
counterclaim) against any party alleging that the Package constitutes
direct or contributory patent infringement, then this Artistic License
to you shall terminate on the date that such litigation is filed.
(14) Disclaimer of Warranty: THE PACKAGE IS PROVIDED BY THE COPYRIGHT
HOLDER AND CONTRIBUTORS "AS IS' AND WITHOUT ANY EXPRESS OR IMPLIED
WARRANTIES. THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE, OR NON-INFRINGEMENT ARE DISCLAIMED TO THE EXTENT
PERMITTED BY YOUR LOCAL LAW. UNLESS REQUIRED BY LAW, NO COPYRIGHT HOLDER
OR CONTRIBUTOR WILL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, OR
CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THE PACKAGE,
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
----------------------------------------------------------
qsf-1.2.7/debian/patches/ 0000755 0000764 0000764 00000000000 10665021416 012774 5 ustar aw aw qsf-1.2.7/debian/patches/index.html.diff 0000644 0000764 0000764 00000001233 10665021216 015675 0 ustar aw aw diff -urN old/doc/index.html new/doc/index.html
--- old/doc/index.html 2005-05-11 19:14:22.000000000 -0300
+++ new/doc/index.html 2005-05-11 19:17:46.000000000 -0300
@@ -7,13 +7,11 @@
qsf-1.2.7/debian/control 0000644 0000764 0000764 00000003467 10561647362 012773 0 ustar aw aw Source: qsf
Section: mail
Priority: optional
Maintainer: Nelson A. de Oliveira
Uploaders: Bartosz Fenski
Build-Depends: cdbs, patchutils, debhelper (>= 5), libgdbm-dev, libmysqlclient15-dev, libsqlite0-dev, bsdmainutils, man-db
Standards-Version: 3.7.2
Package: qsf
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Recommends: procmail | maildrop
Suggests: mail-transport-agent, mysql-server | sqlite, mutt, fetchmail
Description: small and fast Bayesian spam filter
Quick Spam Filter (QSF) is an Open Source email classification filter,
designed to be small, fast, and accurate, which works to classify incoming
email as either spam or non-spam.
.
QSF's targets are speed, accuracy and simplicity:
* It is small and is written in C so it starts up quickly, unlike filters
written in Perl.
* It understands MIME and HTML, so it can intelligently deal with modern
spam, unlike older Bayesian filters such as ifile.
* It runs as an inline filter rather than as a daemon, so it is simple to
install.
* It is written to do only one job - decide whether an email is spam or not
using the content of the message alone - so it is less complex than filters
such as SpamAssassin. Less complexity means bugs and security problems are
less likely.
* As well as words and word pairs, QSF also spots special patterns in email
such as runs of gibberish, HTML comments embedded in text, and other common
spam giveaways, and its flexible tokeniser allows more patterns to be added
as spammers change their tactics.
.
Homepage: http://www.ivarch.com/programs/qsf/
XB-Tag: implemented-in::c, interface::commandline, mail::filters, role::plugin, role::program, scope::application, use::checking, works-with-format::plaintext, works-with::db, works-with::mail, works-with::text
qsf-1.2.7/debian/rules 0000755 0000764 0000764 00000000475 10370214750 012431 0 ustar aw aw #!/usr/bin/make -f
include /usr/share/cdbs/1/rules/simple-patchsys.mk
include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/class/autotools.mk
DEB_INSTALL_DIRS_ALL := usr/share/lintian/overrides
binary-install/qsf::
cp debian/lintian-override $(CURDIR)/debian/qsf/usr/share/lintian/overrides/qsf
qsf-1.2.7/debian/doc-base 0000644 0000764 0000764 00000000663 10665021216 012750 0 ustar aw aw Document: qsf
Title: Quick Spam Filter Documentation Index
Author: Andrew Wood
Abstract: Quick Spam Filter (QSF) is an Open Source email classification filter, designed to be small, fast, and accurate, which works to classify incoming email as either spam or non-spam.
Section: Applications/Net
Format: text
Files: /usr/share/doc/qsf/quickref.txt.gz
Format: HTML
Index: /usr/share/doc/qsf/index.html
Files: /usr/share/doc/qsf/*.html
qsf-1.2.7/debian/changelog 0000644 0000764 0000764 00000015050 10665021215 013215 0 ustar aw aw qsf (1.2.7-1) unstable; urgency=low
* New upstream version:
- Updated license from Artistic License to Artistic License 2.0
-- Nelson A. de Oliveira Tue, 28 Aug 2007 09:35:41 -0300
qsf (1.2.6-1) unstable; urgency=medium
* New upstream version;
* Medium urgency since it fix a bug that might cause undelivered mail.
-- Nelson A. de Oliveira Mon, 5 Feb 2007 09:20:24 -0200
qsf (1.2.5-2) unstable; urgency=low
* Adding a Build-Depends on bsdmainutils and man-db, so quickref.txt gets
correctly generated.
-- Nelson A. de Oliveira Sun, 21 Jan 2007 22:02:57 -0200
qsf (1.2.5-1) unstable; urgency=medium
* New upstream version;
* New maintainer address.
-- Nelson A. de Oliveira Sun, 21 Jan 2007 15:14:49 -0200
qsf (1.2.1-1) unstable; urgency=low
* New upstream version.
-- Nelson A. de Oliveira Wed, 25 Oct 2006 12:21:30 -0300
qsf (1.2.0-1) unstable; urgency=low
* New upstream version;
* Added tags on debian/control.
-- Nelson A. de Oliveira Sun, 1 Oct 2006 20:22:09 -0300
qsf (1.1.13-1) unstable; urgency=low
* New upstream version;
* Bumped Standards-Version to 3.7.2 (no changes needed).
-- Nelson A. de Oliveira Mon, 14 Aug 2006 20:51:00 -0300
qsf (1.1.7-1) unstable; urgency=low
* New upstream version.
-- Nelson A. de Oliveira Sat, 8 Apr 2006 15:02:59 -0300
qsf (1.1.6-1) unstable; urgency=low
* New upstream version.
-- Nelson A. de Oliveira Thu, 2 Feb 2006 16:20:29 -0200
qsf (1.1.2-3) unstable; urgency=low
* Updated to debhelper compatibility level 5:
- debian/compat updated;
- debian/control: build-depends debhelper (>= 5).
* libmysqlclient transition:
- debian/control: changed from libmysqlclient14-dev to
libmysqlclient15-dev on build-depends. (Closes: #343801)
* Updated watch file;
* Added a Lintian override to fix a warning about a line too long on qsf's
manual:
- added debian/lintian-override;
- updated debian/rules to install the override.
-- Nelson A. de Oliveira Sat, 17 Dec 2005 21:30:19 -0200
qsf (1.1.2-2) unstable; urgency=low
* Fixed section of QSF - mail, instead of net. (Closes: #317267).
Thank to Laurent Fousse.
-- Nelson A. de Oliveira Fri, 08 Jul 2005 01:04:39 -0300
qsf (1.1.2-1) unstable; urgency=low
* New upstream version.
-- Nelson A. de Oliveira Wed, 06 Jul 2005 13:49:14 -0300
qsf (1.1.0-2) unstable; urgency=low
* Fix small typo in description (Closes: #317042).
Thanks to Laurent Fousse for pointing this.
-- Nelson A. de Oliveira Wed, 06 Jul 2005 02:22:08 -0300
qsf (1.1.0-1) unstable; urgency=low
* New maintainer;
* First official Debian release (Closes: #273937);
* Using CDBS now.
-- Nelson A. de Oliveira Wed, 11 May 2005 16:05:48 -0300
qsf (1.0.35-1) unstable; urgency=low
* New version (not released).
-- Andrew Wood Thu, 21 Apr 2005 12:32:16 +0100
qsf (1.0.31-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Fri, 04 Mar 2005 19:09:01 +0000
qsf (1.0.22-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Mon, 28 Feb 2005 19:50:04 +0000
qsf (1.0.18-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Sat, 19 Feb 2005 12:04:22 +0000
qsf (1.0.15-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Sat, 5 Feb 2005 18:29:08 +0000
qsf (1.0.14-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Sun, 26 Sep 2004 15:01:30 +0100
qsf (1.0.2-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Wed, 28 Apr 2004 22:08:38 +0100
qsf (1.0.1-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Sun, 14 Mar 2004 23:45:50 +0000
qsf (0.9.25-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Fri, 16 Jan 2004 00:03:06 +0000
qsf (0.9.18-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Mon, 5 Jan 2004 20:08:58 +0000
qsf (0.9.12-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Thu, 1 Jan 2004 13:35:55 +0000
qsf (0.9.9-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Sat, 29 Nov 2003 22:41:14 +0000
qsf (0.9.6-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Sat, 15 Nov 2003 01:00:58 +0000
qsf (0.9.4-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Tue, 21 Oct 2003 21:43:37 +0100
qsf (0.9.0-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Fri, 29 Aug 2003 01:36:23 +0100
qsf (0.8.1-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Thu, 21 Aug 2003 08:37:45 +0100
qsf (0.7.8-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Mon, 18 Aug 2003 21:26:22 +0100
qsf (0.7.7-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Thu, 31 Jul 2003 00:16:43 +0100
qsf (0.7.6-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Wed, 23 Jul 2003 09:58:44 +0100
qsf (0.7.4-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Tue, 8 Jul 2003 19:57:59 +0100
qsf (0.7.0-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Sat, 28 Jun 2003 16:48:39 +0100
qsf (0.5.4-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Wed, 4 Jun 2003 12:44:25 +0100
qsf (0.5.0-1) unstable; urgency=low
* Updated to newest release.
-- Andrew Wood Sat, 10 May 2003 04:22:15 +0100
qsf (0.3.4-1) unstable; urgency=low
* Reformatted, minor changes to get "make deb" working under Debian 2.2.
-- Andrew Wood Wed, 29 Jan 2003 23:12:23 +0000
qsf (0.3.1-1) unstable; urgency=low
* Initial Release.
* My first debian package.
-- Tom Parker Wed, 22 Jan 2003 15:36:15 +0000
qsf-1.2.7/debian/lintian-override 0000644 0000764 0000764 00000000041 10351131333 014526 0 ustar aw aw qsf: manpage-has-errors-from-man
qsf-1.2.7/debian/compat 0000644 0000764 0000764 00000000002 10370214750 012541 0 ustar aw aw 5
qsf-1.2.7/debian/watch 0000644 0000764 0000764 00000000056 10370214750 012375 0 ustar aw aw version=3
http://sf.net/qsf/qsf-(.*)\.tar\.gz
qsf-1.2.7/debian/install 0000644 0000764 0000764 00000000033 10240742410 012723 0 ustar aw aw extra/ /usr/share/doc/qsf/
qsf-1.2.7/debian/docs 0000644 0000764 0000764 00000000131 10415776642 012226 0 ustar aw aw README
doc/TODO
doc/NEWS
doc/quickref.txt
doc/index.html
doc/postfix-howto
doc/changelog
qsf-1.2.7/doc/ 0000755 0000764 0000764 00000000000 10665021416 010670 5 ustar aw aw qsf-1.2.7/doc/INSTALL 0000644 0000764 0000764 00000017230 07611763234 011734 0 ustar aw aw Basic Installation
==================
These are generic installation instructions.
The `configure' shell script attempts to guess correct values for
various system-dependent variables used during compilation. It uses
those values to create a `Makefile' in each directory of the package.
It may also create one or more `.h' files containing system-dependent
definitions. Finally, it creates a shell script `config.status' that
you can run in the future to recreate the current configuration, a file
`config.cache' that saves the results of its tests to speed up
reconfiguring, and a file `config.log' containing compiler output
(useful mainly for debugging `configure').
If you need to do unusual things to compile the package, please try
to figure out how `configure' could check whether to do them, and mail
diffs or instructions to the address given in the `README' so they can
be considered for the next release. If at some point `config.cache'
contains results you don't want to keep, you may remove or edit it.
The file `configure.in' is used to create `configure' by a program
called `autoconf'. You only need `configure.in' if you want to change
it or regenerate `configure' using a newer version of `autoconf'.
The simplest way to compile this package is:
1. `cd' to the directory containing the package's source code and type
`./configure' to configure the package for your system. If you're
using `csh' on an old version of System V, you might need to type
`sh ./configure' instead to prevent `csh' from trying to execute
`configure' itself.
Running `configure' takes awhile. While running, it prints some
messages telling which features it is checking for.
2. Type `make' to compile the package.
3. Optionally, type `make check' to run any self-tests that come with
the package.
4. Type `make install' to install the programs and any data files and
documentation.
5. You can remove the program binaries and object files from the
source code directory by typing `make clean'. To also remove the
files that `configure' created (so you can compile the package for
a different kind of computer), type `make distclean'. There is
also a `make maintainer-clean' target, but that is intended mainly
for the package's developers. If you use it, you may have to get
all sorts of other programs in order to regenerate files that came
with the distribution.
Compilers and Options
=====================
Some systems require unusual options for compilation or linking that
the `configure' script does not know about. You can give `configure'
initial values for variables by setting them in the environment. Using
a Bourne-compatible shell, you can do that on the command line like
this:
CC=c89 CFLAGS=-O2 LIBS=-lposix ./configure
Or on systems that have the `env' program, you can do it like this:
env CPPFLAGS=-I/usr/local/include LDFLAGS=-s ./configure
Compiling For Multiple Architectures
====================================
You can compile the package for more than one kind of computer at the
same time, by placing the object files for each architecture in their
own directory. To do this, you must use a version of `make' that
supports the `VPATH' variable, such as GNU `make'. `cd' to the
directory where you want the object files and executables to go and run
the `configure' script. `configure' automatically checks for the
source code in the directory that `configure' is in and in `..'.
If you have to use a `make' that does not supports the `VPATH'
variable, you have to compile the package for one architecture at a time
in the source code directory. After you have installed the package for
one architecture, use `make distclean' before reconfiguring for another
architecture.
Installation Names
==================
By default, `make install' will install the package's files in
`/usr/local/bin', `/usr/local/man', etc. You can specify an
installation prefix other than `/usr/local' by giving `configure' the
option `--prefix=PATH'.
You can specify separate installation prefixes for
architecture-specific files and architecture-independent files. If you
give `configure' the option `--exec-prefix=PATH', the package will use
PATH as the prefix for installing programs and libraries.
Documentation and other data files will still use the regular prefix.
In addition, if you use an unusual directory layout you can give
options like `--bindir=PATH' to specify different values for particular
kinds of files. Run `configure --help' for a list of the directories
you can set and what kinds of files go in them.
If the package supports it, you can cause programs to be installed
with an extra prefix or suffix on their names by giving `configure' the
option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.
Optional Features
=================
Some packages pay attention to `--enable-FEATURE' options to
`configure', where FEATURE indicates an optional part of the package.
They may also pay attention to `--with-PACKAGE' options, where PACKAGE
is something like `gnu-as' or `x' (for the X Window System). The
`README' should mention any `--enable-' and `--with-' options that the
package recognizes.
For packages that use the X Window System, `configure' can usually
find the X include and library files automatically, but if it doesn't,
you can use the `configure' options `--x-includes=DIR' and
`--x-libraries=DIR' to specify their locations.
Specifying the System Type
==========================
There may be some features `configure' can not figure out
automatically, but needs to determine by the type of host the package
will run on. Usually `configure' can figure that out, but if it prints
a message saying it can not guess the host type, give it the
`--host=TYPE' option. TYPE can either be a short name for the system
type, such as `sun4', or a canonical name with three fields:
CPU-COMPANY-SYSTEM
See the file `config.sub' for the possible values of each field. If
`config.sub' isn't included in this package, then this package doesn't
need to know the host type.
If you are building compiler tools for cross-compiling, you can also
use the `--target=TYPE' option to select the type of system they will
produce code for and the `--build=TYPE' option to select the type of
system on which you are compiling the package.
Sharing Defaults
================
If you want to set default values for `configure' scripts to share,
you can create a site shell script called `config.site' that gives
default values for variables like `CC', `cache_file', and `prefix'.
`configure' looks for `PREFIX/share/config.site' if it exists, then
`PREFIX/etc/config.site' if it exists. Or, you can set the
`CONFIG_SITE' environment variable to the location of the site script.
A warning: not all `configure' scripts look for a site script.
Operation Controls
==================
`configure' recognizes the following options to control how it
operates.
`--cache-file=FILE'
Use and save the results of the tests in FILE instead of
`./config.cache'. Set FILE to `/dev/null' to disable caching, for
debugging `configure'.
`--help'
Print a summary of the options to `configure', and exit.
`--quiet'
`--silent'
`-q'
Do not print messages saying which checks are being made. To
suppress all normal output, redirect it to `/dev/null' (any error
messages will still be shown).
`--srcdir=DIR'
Look for the package's source code in directory DIR. Usually
`configure' can determine that directory automatically.
`--version'
Print the version of Autoconf used to generate the `configure'
script, and exit.
`configure' also accepts some other, not widely useful, options.
qsf-1.2.7/doc/COPYING 0000644 0000764 0000764 00000021253 10664536761 011743 0 ustar aw aw This package is free software, and is being distributed under the terms
of the Artistic License 2.0.
----------------------------------------------------------
Artistic License 2.0
Copyright (c) 2000-2006, The Perl Foundation.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed. Preamble
This license establishes the terms under which a given free software
Package may be copied, modified, distributed, and/or redistributed. The
intent is that the Copyright Holder maintains some artistic control
over the development of that Package while still keeping the Package
available as open source and free software.
You are always permitted to make arrangements wholly outside of this
license directly with the Copyright Holder of a given Package. If the
terms of this license do not permit the full use that you propose to
make of the Package, you should contact the Copyright Holder and seek
a different licensing arrangement. Definitions
"Copyright Holder" means the individual(s) or organization(s) named in
the copyright notice for the entire Package.
"Contributor" means any party that has contributed code or other material
to the Package, in accordance with the Copyright Holder's procedures.
"You" and "your" means any person who would like to copy, distribute,
or modify the Package.
"Package" means the collection of files distributed by the Copyright
Holder, and derivatives of that collection and/or of those files. A given
Package may consist of either the Standard Version, or a Modified Version.
"Distribute" means providing a copy of the Package or making it accessible
to anyone else, or in the case of a company or organization, to others
outside of your company or organization.
"Distributor Fee" means any fee that you charge for Distributing this
Package or providing support for this Package to another party. It does
not mean licensing fees.
"Standard Version" refers to the Package if it has not been modified,
or has been modified only in ways explicitly requested by the Copyright
Holder.
"Modified Version" means the Package, if it has been changed, and such
changes were not explicitly requested by the Copyright Holder.
"Original License" means this Artistic License as Distributed with the
Standard Version of the Package, in its current version or as it may be
modified by The Perl Foundation in the future.
"Source" form means the source code, documentation source, and
configuration files for the Package.
"Compiled" form means the compiled bytecode, object code, binary, or any
other form resulting from mechanical transformation or translation of
the Source form. Permission for Use and Modification Without Distribution
(1) You are permitted to use the Standard Version and create and use
Modified Versions for any purpose without restriction, provided that you
do not Distribute the Modified Version. Permissions for Redistribution
of the Standard Version
(2) You may Distribute verbatim copies of the Source form of the Standard
Version of this Package in any medium without restriction, either gratis
or for a Distributor Fee, provided that you duplicate all of the original
copyright notices and associated disclaimers. At your discretion, such
verbatim copies may or may not include a Compiled form of the Package.
(3) You may apply any bug fixes, portability changes, and other
modifications made available from the Copyright Holder. The resulting
Package will still be considered the Standard Version, and as such will
be subject to the Original License. Distribution of Modified Versions
of the Package as Source
(4) You may Distribute your Modified Version as Source (either gratis
or for a Distributor Fee, and with or without a Compiled form of the
Modified Version) provided that you clearly document how it differs
from the Standard Version, including, but not limited to, documenting
any non-standard features, executables, or modules, and provided that
you do at least ONE of the following:
(a) make the Modified Version available to the Copyright Holder of the
Standard Version, under the Original License, so that the Copyright Holder
may include your modifications in the Standard Version.
(b) ensure that installation of your Modified Version does not prevent the
user installing or running the Standard Version. In addition, the Modified
Version must bear a name that is different from the name of the Standard
Version.
(c) allow anyone who receives a copy of the Modified Version to make
the Source form of the Modified Version available to others under
(i) the Original License or
(ii) a license that permits the licensee to freely copy, modify and
redistribute the Modified Version using the same licensing terms that apply
to the copy that the licensee received, and requires that the Source form of
the Modified Version, and of any works derived from it, be made freely
available in that license fees are prohibited but Distributor Fees are
allowed. Distribution of Compiled Forms of the Standard Version or Modified
Versions without the Source
(5) You may Distribute Compiled forms of the Standard Version without
the Source, provided that you include complete instructions on how to
get the Source of the Standard Version. Such instructions must be valid
at the time of your distribution. If these instructions, at any time
while you are carrying out such distribution, become invalid, you must
provide new instructions on demand or cease further distribution. If
you provide valid instructions or cease distribution within thirty days
after you become aware that the instructions are invalid, then you do
not forfeit any of your rights under this license.
(6) You may Distribute a Modified Version in Compiled form without the
Source, provided that you comply with Section 4 with respect to the
Source of the Modified Version. Aggregating or Linking the Package
(7) You may aggregate the Package (either the Standard Version or
Modified Version) with other packages and Distribute the resulting
aggregation provided that you do not charge a licensing fee for the
Package. Distributor Fees are permitted, and licensing fees for other
components in the aggregation are permitted. The terms of this license
apply to the use and Distribution of the Standard or Modified Versions
as included in the aggregation.
(8) You are permitted to link Modified and Standard Versions with other
works, to embed the Package in a larger work of your own, or to build
stand-alone binary or bytecode versions of applications that include the
Package, and Distribute the result without restriction, provided the
result does not expose a direct interface to the Package. Items That
are Not Considered Part of a Modified Version
(9) Works (including, but not limited to, modules and scripts) that
merely extend or make use of the Package, do not, by themselves, cause
the Package to be a Modified Version. In addition, such works are not
considered parts of the Package itself, and are not subject to the terms
of this license. General Provisions
(10) Any use, modification, and distribution of the Standard or Modified
Versions is governed by this Artistic License. By using, modifying or
distributing the Package, you accept this license. Do not use, modify,
or distribute the Package, if you do not accept this license.
(11) If your Modified Version has been derived from a Modified Version
made by someone other than you, you are nevertheless required to ensure
that your Modified Version complies with the requirements of this license.
(12) This license does not grant you the right to use any trademark,
service mark, tradename, or logo of the Copyright Holder.
(13) This license includes the non-exclusive, worldwide, free-of-charge
patent license to make, have made, use, offer to sell, sell, import
and otherwise transfer the Package with respect to any patent claims
licensable by the Copyright Holder that are necessarily infringed by the
Package. If you institute patent litigation (including a cross-claim or
counterclaim) against any party alleging that the Package constitutes
direct or contributory patent infringement, then this Artistic License
to you shall terminate on the date that such litigation is filed.
(14) Disclaimer of Warranty: THE PACKAGE IS PROVIDED BY THE COPYRIGHT
HOLDER AND CONTRIBUTORS "AS IS' AND WITHOUT ANY EXPRESS OR IMPLIED
WARRANTIES. THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE, OR NON-INFRINGEMENT ARE DISCLAIMED TO THE EXTENT
PERMITTED BY YOUR LOCAL LAW. UNLESS REQUIRED BY LAW, NO COPYRIGHT HOLDER
OR CONTRIBUTOR WILL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, OR
CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THE PACKAGE,
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
----------------------------------------------------------
qsf-1.2.7/doc/index.html 0000644 0000764 0000764 00000000773 07611763233 012703 0 ustar aw aw
Documentation Index
qsf-1.2.7/doc/VERSION 0000644 0000764 0000764 00000000006 10664771531 011745 0 ustar aw aw 1.2.7
qsf-1.2.7/doc/spec.in 0000644 0000764 0000764 00000014003 10664771512 012160 0 ustar aw aw Summary: Quick Spam Filter
%if %{?_with_static:1}0
Name: @PACKAGE@-static
%else
Name: @PACKAGE@
%endif
Version: @VERSION@
Release: 1%{?dist}
License: Artistic 2.0
Group: Development/Tools
Source: http://www.ivarch.com/programs/sources/@PACKAGE@-@VERSION@.tar.bz2
URL: http://www.ivarch.com/programs/@PACKAGE@.shtml
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
%if %{?_with_static:1}0
Obsoletes: @PACKAGE@
%else
Obsoletes: @PACKAGE@-static
%endif
Provides: @PACKAGE@ = @VERSION@-1
%description
Quick Spam Filter (@PACKAGE@) is a small, fast spam filter that works by
learning to recognise the words that are more likely to appear in spam than
non-spam. It is intended to be used in a procmail recipe to mark email as
being possible spam.
Available rpmbuild rebuild options:
--without: gdbm mysql sqlite
--with: static
%prep
%setup -q -n @PACKAGE@-@VERSION@
%build
CFLAGS="$RPM_OPT_FLAGS" sh ./configure \
%if %{?_without_gdbm:1}0
--without-gdbm \
%endif
%if %{?_without_mysql:1}0
--without-mysql \
%endif
%if %{?_without_sqlite:1}0
--without-sqlite \
%endif
%if %{?_with_static:1}0
--enable-static \
%endif
--prefix=/usr \
--infodir=/usr/share/info \
--mandir=/usr/share/man \
--sysconfdir=/etc
make %{?_smp_mflags}
%install
[ -n "$RPM_BUILD_ROOT" -a "$RPM_BUILD_ROOT" != / ] && rm -rf "$RPM_BUILD_ROOT"
[ -e "$RPM_BUILD_ROOT" ] || mkdir -m 755 "$RPM_BUILD_ROOT"
make install DESTDIR="$RPM_BUILD_ROOT"
chmod 755 "$RPM_BUILD_ROOT"/usr/bin/@PACKAGE@*
%clean
[ -n "$RPM_BUILD_ROOT" -a "$RPM_BUILD_ROOT" != / ] && rm -rf "$RPM_BUILD_ROOT"
%files
%defattr(-, root, root)
/usr/bin/@PACKAGE@
%docdir /usr/share/man/man1
/usr/share/man/man1/*
%doc README doc/NEWS doc/TODO doc/COPYING doc/postfix-howto extra/*.sh
%changelog
* Tue Aug 28 2007 Andrew Wood 1.2.7-1
- Changed to Artistic License 2.0.
- Removed "-l" option.
* Sun Feb 4 2007 Andrew Wood 1.2.6-1
- Removed locking from MySQL as it makes it too slow.
* Sun Jan 21 2007 Andrew Wood 1.2.5-1
- Major bugfix in the "list" backend to fix the random deletion of tokens.
- Improved MySQL support.
* Wed Oct 25 2006 Andrew Wood 1.2.1-1
- Concurrent updates now work correctly on all database backends.
* Mon Oct 2 2006 Andrew Wood
- A new database backend called "list". New options to set value of X-Spam
- header, keep a plaintext mapping of hashes to tokens, and maintain a
- deny-list. Allow and deny lists can now list domains as well as individual
- email addresses.
* Mon Aug 14 2006 Andrew Wood
- Code cleanup and fixes for various non-i386-Linux problems.
* Sat Apr 8 2006 Andrew Wood
- Addresses from the Return-Path: header are now also checked against the
- allow-list in addition to those from the From: header.
* Thu Feb 2 2006 Andrew Wood
- Tokenisation fixes for URLs at the start of messages and for nested
- attachments.
* Thu Jul 7 2005 Andrew Wood
- Allow list matching is now case insensitive;
- a btree database's last-modification is updated after any modification;
- the spec file was fixed to work with Fedora Core 4.
* Thu May 12 2005 Andrew Wood
- Tokens now have an age marker;
- additional token types were added;
- the database pruning algorithm was improved;
- the binary tree backend has had some speed enhancements.
* Fri Mar 4 2005 Andrew Wood
- Moved all internal db functions to one file;
- all backends can now be compiled into the same binary;
- some cleanup of code;
- benchmarking has been improved;
- the RPM can now be built with statically linked backends.
* Mon Feb 28 2005 Andrew Wood
- Fixed the documentation of the "--dump" option;
- checked where "--dump" is dumping data to;
- no longer dump large messages in non-filtering mode;
- reporting of database backend in verbose mode.
* Sat Feb 19 2005 Andrew Wood
- A new option to skip short messages was added;
- an option to tune the extent of database pruning was added;
- and the tokeniser was improved.
* Sat Feb 5 2005 Andrew Wood
- Bug fixes when building RPMs, and added support for "rpmbuild --with".
* Sun Sep 26 2004 Andrew Wood
- Code cleanup, and new routines to decode character-encoded headers.
* Wed Sep 22 2004 Andrew Wood
- A new database backend based on SQLite was added.
* Tue Jun 22 2004 Andrew Wood
- A new verbosity option to add errors and information as message headers;
- an option to output stars like SpamAssassin was added;
- the allow-list can be queried using an address read from an email; and
- a system-wide filtering HOWTO was added.
* Tue Apr 27 2004 Andrew Wood
- More explanation of tokenisation, and a new troubleshooting section added
to the manual.
* Fri Mar 12 2004 Andrew Wood
- Code cleanup, many new filters, and some command line syntax improvements.
* Fri Jan 16 2004 Andrew Wood
- A new option to query and update the allow-list directly was added;
- the spam threshold level can be now altered;
- a second "global" database can now be used; and
- some minor bug fixes were made.
* Mon Jan 5 2004 Andrew Wood
- The tokeniser was improved further to recognise distinct URLs and compress whitespace.
- Additional filters for IP-based URLs and virus attachments were added.
* Sat Dec 27 2003 Andrew Wood
- Minor cosmetic fixes were made for non-Linux systems.
- Speed improvements have been made in the binary tree backend database.
* Fri Nov 14 2003 Andrew Wood
- Added "-mysql" subpackage for optional MySQL backend
* Thu Aug 21 2003 Andrew Wood
- Added package description
* Sat Jan 11 2003 Andrew Wood
- First draft of spec file created
qsf-1.2.7/doc/qsf.spec 0000644 0000764 0000764 00000013661 10665021365 012347 0 ustar aw aw Summary: Quick Spam Filter
%if %{?_with_static:1}0
Name: qsf-static
%else
Name: qsf
%endif
Version: 1.2.7
Release: 1%{?dist}
License: Artistic 2.0
Group: Development/Tools
Source: http://www.ivarch.com/programs/sources/qsf-1.2.7.tar.bz2
URL: http://www.ivarch.com/programs/qsf.shtml
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
%if %{?_with_static:1}0
Obsoletes: qsf
%else
Obsoletes: qsf-static
%endif
Provides: qsf = 1.2.7-1
%description
Quick Spam Filter (qsf) is a small, fast spam filter that works by
learning to recognise the words that are more likely to appear in spam than
non-spam. It is intended to be used in a procmail recipe to mark email as
being possible spam.
Available rpmbuild rebuild options:
--without: gdbm mysql sqlite
--with: static
%prep
%setup -q -n qsf-1.2.7
%build
CFLAGS="$RPM_OPT_FLAGS" sh ./configure \
%if %{?_without_gdbm:1}0
--without-gdbm \
%endif
%if %{?_without_mysql:1}0
--without-mysql \
%endif
%if %{?_without_sqlite:1}0
--without-sqlite \
%endif
%if %{?_with_static:1}0
--enable-static \
%endif
--prefix=/usr \
--infodir=/usr/share/info \
--mandir=/usr/share/man \
--sysconfdir=/etc
make %{?_smp_mflags}
%install
[ -n "$RPM_BUILD_ROOT" -a "$RPM_BUILD_ROOT" != / ] && rm -rf "$RPM_BUILD_ROOT"
[ -e "$RPM_BUILD_ROOT" ] || mkdir -m 755 "$RPM_BUILD_ROOT"
make install DESTDIR="$RPM_BUILD_ROOT"
chmod 755 "$RPM_BUILD_ROOT"/usr/bin/qsf*
%clean
[ -n "$RPM_BUILD_ROOT" -a "$RPM_BUILD_ROOT" != / ] && rm -rf "$RPM_BUILD_ROOT"
%files
%defattr(-, root, root)
/usr/bin/qsf
%docdir /usr/share/man/man1
/usr/share/man/man1/*
%doc README doc/NEWS doc/TODO doc/COPYING doc/postfix-howto extra/*.sh
%changelog
* Tue Aug 28 2007 Andrew Wood 1.2.7-1
- Changed to Artistic License 2.0.
- Removed "-l" option.
* Sun Feb 4 2007 Andrew Wood 1.2.6-1
- Removed locking from MySQL as it makes it too slow.
* Sun Jan 21 2007 Andrew Wood 1.2.5-1
- Major bugfix in the "list" backend to fix the random deletion of tokens.
- Improved MySQL support.
* Wed Oct 25 2006 Andrew Wood 1.2.1-1
- Concurrent updates now work correctly on all database backends.
* Mon Oct 2 2006 Andrew Wood
- A new database backend called "list". New options to set value of X-Spam
- header, keep a plaintext mapping of hashes to tokens, and maintain a
- deny-list. Allow and deny lists can now list domains as well as individual
- email addresses.
* Mon Aug 14 2006 Andrew Wood
- Code cleanup and fixes for various non-i386-Linux problems.
* Sat Apr 8 2006 Andrew Wood
- Addresses from the Return-Path: header are now also checked against the
- allow-list in addition to those from the From: header.
* Thu Feb 2 2006 Andrew Wood
- Tokenisation fixes for URLs at the start of messages and for nested
- attachments.
* Thu Jul 7 2005 Andrew Wood
- Allow list matching is now case insensitive;
- a btree database's last-modification is updated after any modification;
- the spec file was fixed to work with Fedora Core 4.
* Thu May 12 2005 Andrew Wood
- Tokens now have an age marker;
- additional token types were added;
- the database pruning algorithm was improved;
- the binary tree backend has had some speed enhancements.
* Fri Mar 4 2005 Andrew Wood
- Moved all internal db functions to one file;
- all backends can now be compiled into the same binary;
- some cleanup of code;
- benchmarking has been improved;
- the RPM can now be built with statically linked backends.
* Mon Feb 28 2005 Andrew Wood
- Fixed the documentation of the "--dump" option;
- checked where "--dump" is dumping data to;
- no longer dump large messages in non-filtering mode;
- reporting of database backend in verbose mode.
* Sat Feb 19 2005 Andrew Wood
- A new option to skip short messages was added;
- an option to tune the extent of database pruning was added;
- and the tokeniser was improved.
* Sat Feb 5 2005 Andrew Wood
- Bug fixes when building RPMs, and added support for "rpmbuild --with".
* Sun Sep 26 2004 Andrew Wood
- Code cleanup, and new routines to decode character-encoded headers.
* Wed Sep 22 2004 Andrew Wood
- A new database backend based on SQLite was added.
* Tue Jun 22 2004 Andrew Wood
- A new verbosity option to add errors and information as message headers;
- an option to output stars like SpamAssassin was added;
- the allow-list can be queried using an address read from an email; and
- a system-wide filtering HOWTO was added.
* Tue Apr 27 2004 Andrew Wood
- More explanation of tokenisation, and a new troubleshooting section added
to the manual.
* Fri Mar 12 2004 Andrew Wood
- Code cleanup, many new filters, and some command line syntax improvements.
* Fri Jan 16 2004 Andrew Wood
- A new option to query and update the allow-list directly was added;
- the spam threshold level can be now altered;
- a second "global" database can now be used; and
- some minor bug fixes were made.
* Mon Jan 5 2004 Andrew Wood
- The tokeniser was improved further to recognise distinct URLs and compress whitespace.
- Additional filters for IP-based URLs and virus attachments were added.
* Sat Dec 27 2003 Andrew Wood
- Minor cosmetic fixes were made for non-Linux systems.
- Speed improvements have been made in the binary tree backend database.
* Fri Nov 14 2003 Andrew Wood
- Added "-mysql" subpackage for optional MySQL backend
* Thu Aug 21 2003 Andrew Wood
- Added package description
* Sat Jan 11 2003 Andrew Wood
- First draft of spec file created
qsf-1.2.7/doc/postfix-howto 0000644 0000764 0000764 00000004736 10066123031 013446 0 ustar aw aw This document was mainly written by M. Kölbl, and details the steps that can
be taken to implement QSF as a system-wide mail filter.
1. Create a user "filter" with password "*" and usergroup "nogroup".
2. Make the dir "/var/spool/filter"
3. Do "> /var/spool/filter/.qsfdb" (i.e. create an empty file).
4. Do "chown -R filter:nogroup /var/spool/filter"
5. Do "chmod -R 700 /var/spool/filter"
6. Create a file "/usr/bin/qsf-postfix" with the following content:
#### BEGIN ####
#!/bin/sh
# Simple shell-based filter. It is meant to be invoked as follows:
# /path/to/script -f sender recipients...
# Localize these.
INSPECT_DIR=/var/spool/filter
SENDMAIL="/usr/sbin/sendmail -i"
QSF="/usr/bin/qsf -r"
# Exit codes from
EX_TEMPFAIL=75
EX_UNAVAILABLE=69
# Clean up when done or when aborting.
trap "rm -f in.$$; rm -f out.$$" 0 1 2 3 15
# Start processing.
cd $INSPECT_DIR || {
echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
cat >in.$$ || {
echo Cannot save mail to file; exit $EX_TEMPFAIL; }
# Specify your content filter here.
$QSF out.$$|| {
echo Message content rejected; exit $EX_UNAVAILABLE; }
$SENDMAIL "$@"