My actual goal is to support streaming .gz log file support. I saw an input plugin that essentially gunzipped the file to scratch but that's not going to work out for me. I'm not sure if a streaming version of that plug-in could be made or not.
In the meantime I got the idea to try using UnZipPipe (a named pipe that streams and gunzips) from here - http://jcarlossaez.spaces.live.com/blog/cns!B3378F057444B65C!108.entry
start /MIN UnZipPipe logpipe D:\06-16-09.gz
logparser -i:IISW3C -o:CSV file:testpipe.sql -headers:ON -fileMode:1 -q:OFF > zippipetest.csv
testpipe.sql:
SELECT TO_STRING(TO_LOCALTIME(TO_TIMESTAMP(date,time)), 'hh') as hour,ADD(QUANTIZE(TO_INT(TO_STRING(time, 'mm')),15),15) AS minutestart, STRCAT(TO_STRING(sc-status), STRCAT('.', TO_STRING(sc-substatus))) AS Status,
COUNT(*) AS Total
FROM \\.\pipe\logpipe
GROUP BY hour,minutestart,Status
ORDER BY hour,minutestart,Status
LogParser appears to not support named pipes though. It complains:
Cannot open <from-entity>: Error opening files: Error searching for files in
folder \\.\pipe: The network path was not found.
I can pipe to logparsers STDIN cat \\.\pipe\logpipe | logparser but that doesn't work out for me because I need to build reports based on multiple files and that would only let me pipe one.
Any chance LogParser can be updated to work with named pipes or other tricks anyone can think ok?