« Previous Next »

• 11-10-2006, 1:11 AM

  NCSA Input format and the same as Output? Reply Contact Hi I have a little Problem. My Web-Server make Logfiles in NCSA-Format, there are some Logfiles which are bigger then 2GB ( Yes it was my fault o_O ) and so my Web-Analyser isn´t able to evaluate the File because it´s too big. Now I have written an sql File to select the Data out of the Logfile, but my Web-Analyser isn´t able to analyse the Outputformats from the LogParser. Now my Question: Is it possible to make only a cut out of the NCSA File and write exactly the selected entrys in a new file that is looking like the NCSA Format? In precept I Just want to make a cut from Logfile and paste in another File which has exactly the same Style. Well I think I had to make my own Outputformat but I don´t know how! Maybe some one inside here could explain it to me. For example a line out of my Logfile: 142.160.52.98 - Anonymous [10/Nov/2006:01:10:29 +0200] "GET /Flottenkdo/Service/telefonbuc HTTP/1.1" 500 16592 "" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" And the text in my sql File: select * into oktober.log from access311006.log WHERE DateTime > TO_TIMESTAMP('2006-10-15 00:00:00', 'yyyy-MM-dd hh:mm:ss') My command in commandline: LogParser -i:NCSA -o:W3C file:formattest.sql The W3C Format was just a testrun to check if my Web-Analyser is able to evaluate the format. I hope that is not too much information ;-) And so on I want to excuse me for my bad english....it isn´t my native language and I hope you understand what I want :-) Greetz NickSoft

  Tags: NCSA input format

• 11-10-2006, 10:53 AM

  In reply to

  RE: NCSA Input format and the same as Output? Mark as Not AnswerMark as Answer... Reply Contact If you just want to cut the file into smaller chunks so your web-analyzer is able to understand them then I wouldn't even bother with LP. There are plenty of programs that can chunk data like that or you could just write a simple Perl/Python/Ruby/Bash script to do it. If you want to use LP, I'd suggest actually treating the files as TEXTLINE instead of NCSA. Use string parsing to do the timestamp comparison you want to do and just write out the Text field to the new file. This will be easier than trying to craft a new output format and it will also let you use the multiplexing feature if you want to automatically break the big files into month or week or daily chunks. Good luck!

  Tags: NCSA input format

• 11-13-2006, 2:10 AM

  In reply to

  RE: NCSA Input format and the same as Output? Mark as Not AnswerMark as Answer... Reply Contact Thx this sounds good Sometimes is it better then see it from another view....the new problem how I will give only the textline out? The Input data in textline is no problem also the text comparison but in which format I can make the output....the output formats are already the same then Input in NCSA. Or is there any possibility to put the line as textformat in an file out? For sure there is an posibility...but im too stupid to find it o_O Well I will search for it today and will report in but if you have an solution I will be happy. Thx and Greetz Nicki

  Tags: NCSA input format

• 11-13-2006, 3:40 AM

  In reply to

  RE: NCSA Input format and the same as Output? Mark as Not AnswerMark as Answer... Reply Contact Hi me again I think I´ve found the problem why ma Web-Analyser isn´t able do evaluate the Information. It doesn´t matter which Outputformat I choose the LogParser always write something at the beginning of each line. The things he wrote in front of each line are: C:\Programme\Log Parser 2.2\wiseaccess311006.log 7479960 The number seems to be an counter the number is rising. Is there an possibility that the LogParser didn´t write this Line?? Then I´m thinking this is se easiest way to make it understandable for my Web-Analyser. Thx and Greetz Nicki

  Tags: NCSA input format

• 11-13-2006, 12:34 PM

  In reply to

  RE: NCSA Input format and the same as Output? Mark as Not AnswerMark as Answer... Reply Contact If you use SELECT *, LP is going to output the filename and row number in addition to the other fields defined as NCSA. Try using this query and see if it works for you: logparser -i:TEXTLINE -o:NAT -headersFF "SELECT FileNum, Text USING ADD(1, DIV(Index, 1000)) AS FileNum INTO outfile-*.log FROM infile.log"

  Tags: NCSA input format

• 11-14-2006, 2:43 AM

  In reply to

  RE: NCSA Input format and the same as Output? Mark as Not AnswerMark as Answer... Reply Contact Sounds good but doesn´t work Well I just try it and the only think is that LP is wrote a number in front of Line. The Line looks like this now: 1 C:\Programme\Log Parser 2.2\z2.log, 18547, 142.160.44.19, -, Anonymous, 11/14/2006, 5:45:01, GET /UNIFIL/gpers_img HTTP/1.0, 200, 479, http://www.flottenkdo.mar/UNIFIL, Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0), -, I try it also in another input formats. Anyone another idea? I just need the part after and incl. "Anonymous". Thx and Greetz

  Tags: NCSA input format

• 11-14-2006, 9:13 AM

  In reply to

  	RE: NCSA Input format and the same as Output? Mark as Not AnswerMark as Answer... Reply Contact Oops, NAT doesn't support multiplexing. Try using the TSV output instead. As long as you are only outputting one field, it shouldn't put any delimiters in and you should be alright. Are you sure you used the select statement I suggested instead of a *? SELECT FileNum, Text USING ...
  	Tags: NCSA input format

• 11-16-2006, 3:07 AM

  In reply to

  RE: NCSA Input format and the same as Output? Mark as Not AnswerMark as Answer... Reply Contact Hi I did it....everything now is clear and the Logfile did it the way I want. For information the way I did it The sql-file: select RemoteHostName, UserName, DateTime, Request, StatusCode, BytesSent into oktober.log from wiseaccess311006.log WHERE DateTime > TO_TIMESTAMP('2006-10-15 00:00:00', 'yyyy-MM-dd hh:mm:ss') The Command Line: LogParser -i:NCSA -o:TSV file:selall.sql This is the way I wanna do it Many Thanks for help Daniel !!! Greetz and cu soon NickSoft

  Tags: NCSA input format