The batch integration of Kippo text logs is moving along, but has pointed out some annoying bugs in my rebuild code. The actual integration of the data into my log files is done (and mostly tested). The problem is that I have been creating lists of IP addresses, Usernames, and Passwords based on “ALL” the old data, and then using those files (passwords.all, for instance) to determine what passwords are new for that day.
Sooooo, I need to nuke those files, and then iterate through the days in order and recreate those files to get new passwords for that day. This is also important for my Trends anaylsis.
Thankfully I already had “rebuild” code in place, so I just need to do the analysis in that section of code.
Easy, but non trivial (I hope).
Version 2.0 of LongTail will handle batch AND live feeds from Kippo.