Log file DNS resolver with disc cache

Description

Cachelogres is like Apache's logres but with a disc cache which caches look up results.
When a lookup fails logres will try over and over again. Possibly with a long timeout for each try. Sometimes taking more than 24 hours to process a log.
Cachelogres on the other hand, also caches negative results. The positive (a host name was found) and negative (no host name was found) TTLs can be set on the command line and default to four and one weeks resp.

Just like logres, cachelogres expects the first string in a line of text to be an IP address followed by a space. It reads from standard input and writes to standard output, replacing this IP address by a host name when it can be found. To make sure that the host name is correct, it will do AAAA and A (IP addresses) lookups after the PTR (reverse) look up;

10.20.30.40	→	example.org
example.org	→	10.20.30.40

This is like the logres '-c' option.

Files

Cache file format

       0        1        2        3        4        5        6        7
     ┌────────┬────────┬────────┬────────┬────────┬────────┬────────┬────────┐
   0 │ IP Address                                                            │
     ├────────┼────────┼────────┼────────┼────────┼────────┼────────┼────────┤
   8 │                                                                       │
     ├────────┼────────┼────────┼────────┼────────┼────────┼────────┼────────┤
  16 │ Host name                                                             │
     ├────────┼────────┼────────┼────────┼────────┼────────┼────────┼────────┤
     ≈                                                                       ≈
     ├────────┼────────┼────────┼────────┼────────┼────────┼────────┼────────┤
 264 │                                                                  0    │
     ├────────┼────────┼────────┼────────┼────────┼────────┼────────┼────────┤
 272 │ Epoch                                                                 │
     └────────┴────────┴────────┴────────┴────────┴────────┴────────┴────────┘

Each entry is 280 bytes:

16 Byte IP address. IPv4 Addresses are stored as IPv4-mapped IPv6 address.
256 Byte host name. NULL Terminated string (with zero padding). Max 255 bytes.
The maximum allowed host name size is 253 bytes. So this adds a little margin.
8 Byte epoch.

Files

hash0, hash1, hash2, ... hashe, hashf

1 to 16 Sparse files.
(The lower 16 to 23 bits of) a 24-Bit CRC of the IP address is used as a hash. The hash is multiplied by the record size (280 bytes) and this is used as a file offset (number of bytes from start of file).
hash0 is tried first. In case of a hash clash (an other IP address having the same hash) the next hash file is used.
If all of the hash files clash the lin file is used.

lin

Regular file.
Used for hash clashes in the above files.
File size: 0 or more times 280 bytes.
This file uses a linear search, which is relatively slow. So it is important to keep this file small.

Cache lay-out

The number of hash files can be 1 to 16 (Default: 1).
The maximum number of entries per hash file can be 64k to 16M (Default: 4M or 4194304 entries).
The cache is generated by 'gendnscac'. Hash file size and number of hash files can be set on the command line;

Hash file size
Hash Bits	Max Entries	Max Hash	Hash file size
16	64k	0x00FFFF	17920 kB
17	128k	0x01FFFF	35 MB
18	256k	0x03FFFF	70 MB
19	512k	0x07FFFF	140 MB
20	1M	0x0FFFFF	280 MB
21	2M	0x1FFFFF	560 MB
22	4M	0x3FFFFF	1120 MB
23	8M	0x7FFFFF	2240 MB
24	16M	0xFFFFFF	4480 MB

The '-b' option sets the number of hash bits. The '-f' option the number of hash files.

'gendnscac' expects data with the following format on standard input:

 Epoch <Tab or space> IP Address <Tab or space> Host name <Line feed>

Note that this is Tab or single space!
If you don't feed 'gendnscac' any data it will just produce an empty cache (press <Enter> to feed it an empty line).
You can also use 'gendnscac' to add data to an a existing cache, in which case the '-b' and '-f' options are ignored.

Changing the cache lay-out

A big lin file slows things down.
If the lin file gets too big:

Dump the cache to text files with 'dumpdnscac'.
This will produce hash0.txt, hash1.txt, hash2.txt, ... hashe.txt, hashf.txt and lin.txt
You can merge these into one file with 'gen-cache-txt.sh'.
Remove the old cache files.
Generate a new cache from the text dump with 'gendnscac', increasing the number of hash files, their size or both.

TTL

The TTL or Time To Live determines how long data is stored in the cache. To this end the moment when an entry is updated is stored in the last eight bytes of a record in the form of Unix time. (An epoch of 0 indicates free space.)
A positive TTL is the time a look up result with host name is allowed to stay in the cache.
A negative TTL is the time a look up result without a host name is allowed to stay in the cache.
Both can be specified in seconds, hours, days, weeks months or years.
A negative TTL defaults to one week. A positive TTL defaults to four weeks.

An entry that is too old is called stale.
Stale entries are removed from the lin file whenever the lin file is searched.
A stale entry is only removed from a hash file when the hash matches.
In both cases is removal is done by overwriting the old data with NULL bytes. So files don't get smaller by removal of old records.

Programs

cachlogres: Logfile DNS resolver with cache.
chkdnslin: Check lin file.
clrdnscac: Removes stale entries from the DNS cache by overwriting these with null bytes.
When followed by 'fallocate -d Hash_File_Name', this may actually free disk space.
dnscac2ana: Convert cache text dump to file suitable for analog.
dnscacstats: Show cache statistics.
dumpdnscac: Dump cache to text files.
The script 'gen-cache-txt.sh' can combine these into one file.
gendnscac: Generate cache.
mvdnscaclin: Moves entries from lin file to free space in hash files.
After a successful move, the old entry in the lin file is overwritten with null bytes.
Use this after 'clrdnscac'.
rmdnscacip: Remove IP address from cache.

Download and install

Current version of tar: 2026-10-05 08:08:00 UTC

Download: cachelogres.tar.gz
Extract tar: 'tar xvfz cachelogres.tar.gz'.
Cd to cachelogres/.
Run: 'make'.
As root, Run: 'make install'.

The install script installs binaries in /usr/local/bin/, man pages in /usr/local/share/man/man1/, documentation in /usr/local/share/doc/cachelogres/ (directories are created when they do not exist) and creates a directory /var/local/cache/dns/.