zipped csv files as input
Last post Sep 11, 2008 08:39 AM by gluegood
Jun 19, 2008 06:53 AM|juergmaier|LINK
Any plugin available that will allow to parse zipped files?
CSV and TSV input formats
Jun 19, 2008 08:32 AMemail@example.com|LINK
I haven't heard of any, but why not run a batch file that unzips the file, LogParses it then rezips it?
Jun 19, 2008 11:53 AM|juergmaier|LINK
Thanks, but I am looking at 100'000 possible files in the TB range and I have no room to unzip them all before parsing
Jun 24, 2008 07:52 AM|Zhao Ji Ma - MSFT|LINK
Log Parser supports reading input data from the input stream, for example.
type filename.csv | LogParser "SELECT * FROM STDIN" -i:CSV
Perhaps you can output csv data to the I/O pipe since unzipping those data requires large disk space for storing before parsing.
Jun 24, 2008 11:36 AM|juergmaier|LINK
I can't howevr yet not figure out how it could help me. My files are zipped csv log files and it would be very handy to e.g. count the number of special events logged in the files.
The files have talking names so to analyze a certain month I could use:
SELECT count(*) from *200805*.zip WHERE Field1="011" group by SUBSTR(Filename,1,5)
I agree that I could first use a -i:FS run to get the list of files, run an unzipper with these files, then run log parser and thereafter clean up but - in this scenario - how much more efficient would it be to be able to specify -i:CSVZIP or some other
The ultimate solution would of course be that logparser recognizes zipped files automatically and unpacks them before processing?
As of my knowledge zip/unzip libraries are available but would need to be used from within logparser.
Sep 11, 2008 08:39 AM|gluegood|LINK
See my post (or
blog site) in the LogParser Advanced forum regarding a VB.NET freeware Log Parser custom input plug-in that can run Log Parser queries inside of ZIP files.
Should address your needs.