Advanced Topics

How To Output the Filename Where Match Occurs?

mOrrIsEmSaBeR — Thu, 12 Nov 2009 16:58:33 GMT

Hello,

I'm running logparser in a C# application, and I'm using the ExecuteBatch method.

This runs nicely (I think) because I do get the output file, but the method returns false.

Now the main question I wanted to ask is.

since my query is searching in a bunch of files, I was wondering if it's possible to output the file name where the match occurs?

e.g.

Query: SELECT DESC INTO OutPutFile.csv FROM C:\\Somefolder\\*.* HAVING DESC LIKE '%My Match%'

the output file will only contain the DESC Field.

And I wanted to have the DESC Field and also the FILENAME.

Thanks in advance,

André Pedro Peixoto

Urgent: Can not impersonate for MSutil with ASP.NET

tuantomy — Mon, 09 Nov 2009 08:46:10 GMT

I created a website using ASP.NET and imported MSUtil as a reference. MSUtil.Interop was installes into GAC.

Then I create a code like this:

LogQueryClassClass logQuery = new LogQueryClassClass();

COMEventLogInputContextClassClass evtLog = new COMEventLogInputContextClassClass();

ILogRecordset iLogRecordSet = logQuery.Execute("SELECT top 10 FROM \\abc\security", evtLog);

---------------

I publish this web application on IIS local, identity of this website is a domain account but this account did not belong to Local Admin Group.

I rewrite the code using:

using(Impersonate imp = new Impersonate("account_A","domain","pass"))

{

///My code here

}

account_A is a domain account with full rights on that computer and belongs to Local Admin Group.

But I still get an error, I mean I could not impersonate my code to admin account when I run the application with non-local admin account as identity.

Please help me fix that.

Thanks

how to get the # of records from ILogRecordset?

jdelinger — Fri, 16 Oct 2009 19:48:01 GMT

hello all,

How Do I get the number of records using the code below? rsLP does not have a method for that :(

LogQueryClassClass LogParser = new LogQueryClassClass(); COMIISW3CInputContextClassClass pObjInput = new COMIISW3CInputContextClassClass();

ILogRecordset rsLP = LogParser.Execute(query, pObjInput);

Thank you

How do you convert date format from dd/mm/yyyy to yyMMdd?

thaichi23 — Thu, 22 Oct 2009 08:42:46 GMT

Hello all,

I have my IIS to record daily so the log name is ex091022.log. I want logparser to find the log that has the current date similar to "select * from ex%date%.log"

The only problem is that %date% returns 10/22/09 and that will not automatically grab the right log. I want to grab the date in the yyMMdd format. Is this possible?

My logparser statement works great but this is the only problem. I am going to save the logparser query as a .bat file and let task schedular run this everday at a certain time so that way it automatically grabs the current date's log and slaps it into SQL.

64 Bit version of LogParser.dll

usmanjamil — Tue, 27 Jan 2009 11:29:57 GMT

In previous posts I've found out that 64 bit version of LogParser.dll has not be released yet. Many of the applications that are using LogParser.dll fail to run on 64 bit machine. Most of the web hosting industry is already migrating to 64 bit machine and all these applications will certainly fail to work in such environments. If Microsoft cannot find time to compile the code for 64 bit, I being a developer too, volunteer to do it for them for 64 bit if they can provide source code of it. It is really becoming a necessity for all the applications migrating on 64 bit platform and there's no news of its official release from Microsoft. I guess there's no other option for users but to compile it for themselves. I Hope, Microsoft takes a seriuos notice of it and either provide the 64 bit version of Logparser.dll themselves or let me do it for them (Free of cost offcourse) :)

Regards

Usman

LogParser through Windows Service

Ya Ya — Mon, 07 Apr 2008 04:38:08 GMT

Hi all, I am working on small project using C# (2005) to use LogParser to query a particular server to grab it's log and transfer the log to SQL2005.

I am able to do so if I access the LogParser through IE (through interface like letting the user select the server, date/time to parse etc). But when I transfer the entire script into Windows Service, I cannot get anything at all.

Below is part of the codes that I have. For the Windows Service part, if I were to comment off the line LogRecordSet oRecordSet = oLogQuery.Execute( query, oEVTInputFormat ); it works (as in if I were to .write the query string out, I am able to) but when I put that line in, nothing happens although the service is started. Hope some experts here can help me! Thanks in advance!

LogQuery oLogQuery = new LogQuery();
EventLogInputFormat oEVTInputFormat = new EventLogInputFormat();
oEVTInputFormat.direction = "BW";
string query = @"SELECT * FROM ";
if ( _ip != null )
{
if ( _ip != "local" )
{
query = query + "\\\\" + _ip + "\\SYSTEM";
}
}
query = query + " WHERE (1 = 1)";
if ( _lastmessageon != null )
{
query = query + " AND (TO_DATE(TimeGenerated) > TO_DATE(TIMESTAMP( '" + Common.MyFunctions.FormatDateTime( _lastmessageon, "yyyy-MM-dd hh:mm:ss" ) + "', 'yyyy-MM-dd hh:mm:ss')))";
}
LogRecordSet oRecordSet = oLogQuery.Execute( query, oEVTInputFormat );

.... (codes to insert into SQL2005)

oRecordSet.close();

Using LogParser v2.2 to query registry.

gcasella — Tue, 13 Oct 2009 20:28:57 GMT

Hello Everyone,

i'm having a little issue with LogParser, i am able to do my query through the command line and recieve an appropriate and the correct result, however when i try implimenting it into my vb.net application written in visual studio i can't seem to determine if the data is going to the application mainly because i don't know how to display it. I've tried many examples but not a lot on vb.net for this situation.

Any clues or insight would be awsome.

Thanks in advance.

Regards,

gcasella

IIS Log parse user column empty

togo413 — Tue, 06 Oct 2009 03:50:47 GMT

Dear everyone,

I got some question regard iis log need to ask.

I got a web application which require windows authentication to log in to access to the system.

Anonymous is not allowed.

But when I parse the log or open the log to read it, I still see a lot of record with empty user column. Please advice why this could happen.

Thanks

iCheckPoint issue

jwaz1999 — Wed, 23 Sep 2009 17:39:49 GMT

Hello,

I'm having an issue when using the checkpoint functionality of the log parser. The subsequent calls to the updated file always skip the first record. This happens every time.

It could be a format issue of my file where the last record is thought to exist even though it is incomplete.

Does anyone know a way around this?

Thanks,

Jeff

LogRecordSet.atEnd() bug

Undutchable — Mon, 18 May 2009 19:19:57 GMT

Hi, In Execute Method, the LogRecordSet.atEnd(), is never true, with ETW input in live session (real time mode). For example tracelog and logman have the real time option. It works very well with LogParser but the recordSet never reaches the end! Which makes it almost impossible to script something unless you set system timers to exit the enumeration. Awesome tool besides that. 5 stars! Cheers!

Merging IIS logs

bigluz — Sat, 08 Aug 2009 20:19:47 GMT

I need to write a script to merge two or more IIS logs. Any help will be greatly appreciated.

GZip / Named Pipe Support?

sideout — Sat, 04 Jul 2009 11:05:52 GMT

My actual goal is to support streaming .gz log file support. I saw an input plugin that essentially gunzipped the file to scratch but that's not going to work out for me. I'm not sure if a streaming version of that plug-in could be made or not.

In the meantime I got the idea to try using UnZipPipe (a named pipe that streams and gunzips) from here - http://jcarlossaez.spaces.live.com/blog/cns!B3378F057444B65C!108.entry

start /MIN UnZipPipe logpipe D:\06-16-09.gz

logparser -i:IISW3C -o:CSV file:testpipe.sql -headers:ON -fileMode:1 -q:OFF > zippipetest.csv

testpipe.sql:

SELECT TO_STRING(TO_LOCALTIME(TO_TIMESTAMP(date,time)), 'hh') as hour,ADD(QUANTIZE(TO_INT(TO_STRING(time, 'mm')),15),15) AS minutestart, STRCAT(TO_STRING(sc-status), STRCAT('.', TO_STRING(sc-substatus))) AS Status,
COUNT(*) AS Total
FROM \\.\pipe\logpipe
GROUP BY hour,minutestart,Status
ORDER BY hour,minutestart,Status

LogParser appears to not support named pipes though. It complains:

Cannot open <from-entity>: Error opening files: Error searching for files in
folder \\.\pipe: The network path was not found.

I can pipe to logparsers STDIN cat \\.\pipe\logpipe | logparser but that doesn't work out for me because I need to build reports based on multiple files and that would only let me pipe one.

Any chance LogParser can be updated to work with named pipes or other tricks anyone can think ok?

LogParser Script to capture logon/logoff info on a remote computer

BCStryker — Mon, 10 Aug 2009 18:50:45 GMT

Hi all,

I am hoping to use Logparser to capture logon/logoff user info (ID's 528 and 540) from the security logs of a Windows 2003 fileserver for only certain machine names using wildcards in a certain date range say weekly or monthly.

Ideally the script would be run remotely and would on a weekly schedule connect to the fileserver, capture only logon and logoff events of computer workstations with names like *wkstn and *srvr from the previous week, and would then dump the info into a CSV file into a remote file share somewhere for analysis.

Is that possible to do with Logparser and if so has something been already written out there that I could modify and use?

Thanks for any help!

Scripts to capture logon/logoff works fine-does not work if run as a group policy logon/logoff script

BobJ — Tue, 04 Mar 2008 04:12:27 GMT

I created a process to capture specific logon/logoff information - including date, time, user name, ip address, mac address, computer name, and some other site specific information. This all runs as part of a batch file that collects the data and ultimately echos all captured data to a csv file.

I want to import this data at the time it is captured to a SQL database, so I modified my script to run logparser and read the csv file in order to import it directly to the SQL DB. All data is initially stored on the local machine of the user logging on. There do not appear to be any issues with creating the text files in that directory. If I manually run the batch file, everything works as planned - CSV is created, read and imported into SQL. Works Great!

But....if I set the batch files to run as the logon and logoff scripts in group policy (user), the CSV file is created, but the data is not imported into SQL. I tried to pipe the output of the SQL command to a text file for debugging and along with the statistics, it reports "task aborted". That's it. I couldn't find anything else anywhere.

This is the statement I'm trying to execute:

\\mydomain\NETLOGON\logparser.exe "select * INTO LogonLogoff from c:\logdata\DataToImport.txt" -i:CSV -o:SQL -server:MySQLServer\ThisInstance -database:LogonLogoffDB -driver:"SQL Server" -username:loguser -password:logpassword -createtable:OFF -headerrow:off

I'd certainly appreciate any suggestions anyone may have.

Can we parse Event description (Message) into only what we want? RESOLVED!!!

wonderboyme — Mon, 03 Nov 2008 17:39:57 GMT

Hi Everyone! I'm newer to Log Parser and have been wracking my fingers over this. I'm looking to parse Windows Event Log files and I'm hoping to find a method using Log Parser that allows me to customize (filter/parse) the Message field in the output file generated.

For example, if I want logon records 528 I use:

"SELECT Message INTO report.csv FROM SecEvent.Evt WHERE EventID=528"

But I don't want everything in the description field for the record. I only want: time generated, user name and domain. So I modify my command to:

"SELECT TimeGenerated, EXTRACT_TOKEN(Strings, 0, '|')AS UserName, EXTRACT_TOKEN(Strings, 1, '|')AS Domain INTO report.csv FROM SecEvent.Evt WHERE EventID=528"

The above gives me my CSV file with the time generated, the user account and domain. But the key piece of information I want is the first line of the Message (description) field in the event record: that of "Successful Logon:".

Thus far I have not found a way to parse the Message field to pull this first line. 0 is for the user name, 1 for the domain, 2 for the logon ID, etc., in the Security event log.

Is there a way to use Log Parser to parse the Message field in the event log to pull this "first" line? I quote first because it's the very first line of text displayed in the event viewer event properties window, followed by the user name, domain, etc.

Currently I'm outputting the entire Message field to a CSV file, then using OpenOffice to find and replace all the information I don't want. Not slow, but would like to know if Log Parser can write out what I want and only what I want.

Has anyone ideas or know how to get that first bit of information from the Message (description) field of event logs using Log Parser (along with the other two fields (strings) I'm interested in?

Cheers!

not able to read .CSV file using LogParser

maximusdm — Thu, 16 Jul 2009 18:05:16 GMT

Hi all,

I managed to create a .CSV file using the COMIISW3CInputContextClassClass to read an IIS log, however I have a CSV file which I am trying to read and is giving me the error below:

{"Error parsing query: SELECT clause: Syntax Error: unknown field 'date1' [Record field does not exist.]"}

Sample code:
--------------------
query="select date1, date2 INTO C:\\A.csv FROM c:\\B.log"

public static void GetResultsFromBatch_CSV(string query)
{
LogQueryClassClass LogParser = new LogQueryClassClass();
COMCSVInputContextClassClass pObjInput = new COMCSVInputContextClassClass();
COMCSVOutputContextClassClass pObjOutput = new COMCSVOutputContextClassClass();

try
{
LogParser.ExecuteBatch(query, pObjInput, pObjOutput);
}
catch (Exception e)
{
}
}

Sample source CSV file:
------------------------
#2009:07:15 11:00:26:23 - 2009:07:15 12:00:00:00
date1,date2,id1,status1,sink1,sink2,sink3,id2,id3,status2
2009:07:15 16:00:26:23,2009:07:15 11:00:26:23T-5,65535,Debug,LoggingTrace,AgentService.exe,,5564,4,[Debug] Init agent.^

Any ideas? Thank you!

Query works via logparser window but won't work via COM programming

maximusdm — Fri, 17 Jul 2009 17:18:01 GMT

Hi all,

The query below works fine when I ran it off the logparser window:

logparser "select date1,date2,id1 INTO C:\Sink.csv FROM C:\Sink.log WHERE date1 > '2009:07:15 16:30:31:36'"
-i:CSV -iHeaderFile:"c:\Header.csv" -nSkipLines:2 -iTsFormat:"yyyy-MM-dd?hh:mm:ss"

Using C# and StringBuilder class I have the following output string:

SELECT date1,date2 INTO 'C:\Sink.csv' FROM c:\Sink.log
WHERE date1 > '2009-07-17 16:42:17' -i:CSV -iHeaderFile:"c:\Header.csv" -nSkipLines:2
-iTsFormat:"yyyy-MM-dd?hh:mm:ss"

also tried with single quotes:

SELECT date1,date2 INTO 'C:\Sink.csv' FROM c:\Sink.log
WHERE date1 > '2009-07-17 16:50:36' -i:CSV -iHeaderFile:'c:\Header.csv' -nSkipLines:2
-iTsFormat:'yyyy-MM-dd?hh:mm:ss'

When I call "LogParser.ExecuteBatch(query, pObjInput, pObjOutput)" I get the error:

{"Error parsing query: Syntax Error: extra token(s) after query: '-i:CSV' [SQL query syntax invalid or unsupported.]"}
System.Exception {System.Runtime.InteropServices.COMException}

Any ideas?

Thank you!

Creating a new Input Format in C#

ph — Fri, 17 Jul 2009 15:00:44 GMT

Hello,

I was looking at the different ways to write a new Input Format to be able to analyse my own log files with Log Parser.

So I wrote a scriptlet, I included it in my C# project and it is working well.

Now I'm trying to do the same, but by writting a C# class. I though I just had then to compile (csc command) this C# class to generate a dll, and then to include it in my .net project, and to use it as an Input Format when executing my sql query with the LogQuery execute Method.

But it seems it's not the right way to do that because i get the message The specified input object is not a built-in Log Parser input format, nor a custom ILogParserInputContext COM plugin [The parameter is incorrect.].

I don't know where I'm wrong, in the dll generation or in the class itself (I inspired me of the XML COM Input Format sample to write the class)

Some idea? Somewhere some skeletton and the steps to follow to do that?

Thanks a lot!

How do i get Log Parser to release its file handle so i can delete a .CSV file?

maximusdm — Tue, 14 Jul 2009 19:12:02 GMT

Hi. I am using Log Parser 2.0 COM to parse a IISW3C log file to csv output format from C#.NET. If an error is encountered whilst parsing i catch the error and i want to delete the file. However when i try and delete the csv file i get another error saying that i cannot delete it because

"The process cannot access the file xxx.yyy because it is being used by another process".

How do i get Log Parser to release its file handle so i can delete the csv file?

-----here is the code-----
public static void GetResultsFromBatch(string query)
{
  LogQueryClassClass LogParser = new LogQueryClassClass();
  COMIISW3CInputContextClassClass pObjInput = new COMIISW3CInputContextClassClass();
  COMCSVOutputContextClassClass pObjOutput = new COMCSVOutputContextClassClass();

  try
  {
   LogParser.ExecuteBatch(query, pObjInput, pObjOutput);
  }
  catch (Exception e)
  {
  }
  finally
  {
   //LogParser - need to find a way to release the handle or dispose this obj
  }
}

Error: [System.IO.IOException] = {"The process cannot access the file 'C:\\test.csv' because it is being used by another process."}

Someone posted the same question here sometime ago but nobody was able to find a solution for this issue :(

Thank you

Error opening event log "server\Application": Access is denied.

kantravi — Tue, 07 Jul 2009 00:48:41 GMT

Using Log Parser and I am getting this error when I am trying to read the Application Events log remotely using the following query.

Cannot open <from-entity>: Error opening event log "server\Application":
Access is denied.

EXEC xp_cmdshell 'C:\Parser\LogParser.EXE " SELECT 1,[EventID], [EventType], [SourceName], [TimeGenerated], [TimeWritten],[Message] INTO TempEvents12 FROM \\server\Application " -i:EVT -o:SQL -server:servername -database:dbname -driver:"SQL Server" -createTable:ON -stats OFF '

I am running the query from Sql Server Management Studio. And the environment is Windows Server Enterprise - 2008.

The user I am using is the member of: [Administrators], [Event Log Readers] and [Remote Desktop Users] Groups. And also I have provided the read access to the log file directory. But still I am getting the Access is denied error.

Is Anyone has faced this isssue before ??

which log should i concern about

gwl7810 — Wed, 24 Jun 2009 03:56:59 GMT

which log should i conern about? can someone tell me?

thx in advance .

How can i make a good Printer Log (read system event log)

tuxus — Fri, 26 Jun 2009 14:50:52 GMT

Hi all

I want to make a good printer log. A printer log who count only the true page who was sent to the printer, but with this script when a print job is cancelled the page is counted.

I don't know how i can substract the page who has cancelled (eventID=13)

here my script

SELECT
EXTRACT_TOKEN(Strings,2,'|') As UserName,
COUNT (*) As Docs ,
SUM(TO_INT(EXTRACT_TOKEN(Strings,6,'|'))) As Pages
FROM System
Where EventID=10 AND timegenerated > '2009-06-25 00:00:00' and timegenerated < '2009-06-30 23:59:00'
GROUP BY UserName ORDER By Pages DESC

Thanks for your help

Logparser to syslog errors in the date

Ag3NT01 — Tue, 23 Jun 2009 17:33:37 GMT

I have a script to export the w3svc log files to a syslog server. I actually found this on this forum and changed what I needed. I have a problem with this in that some of my logs from 2008 are being changed to 2009 when being exported. I do not see this behavior when exporting to a csv. Can anyone tell me why this is happening?

Script to syslog:

LogParser" "SELECT TO_TIMESTAMP(date,time), CASE QNTFLOOR_TO_DIGIT(sc-status, 1) WHEN 500 THEN 'emerg' WHEN 400 THEN 'err' ELSE 'info' END AS MySeverity, s-computername AS MyHostname, s-ip, cs-method, cs-uri-stem, c-ip, cs(User-Agent), sc-status, sc-substatus INTO @xxx.xxx.xxx.xxx FROM <1> WHERE sc-status >= 400" -o:SYSLOG -severity:$MySeverity -hostName:$MyHostname -iCheckpoint:MyCheckpoint.lpc

Script to csv:

LogParser" "SELECT TO_TIMESTAMP(date,time), CASE QNTFLOOR_TO_DIGIT(sc-status, 1) WHEN 500 THEN 'emerg' WHEN 400 THEN 'err' ELSE 'info' END AS MySeverity, s-computername, s-ip, cs-method, cs-uri-stem, c-ip, cs(User-Agent), sc-status, sc-substatus INTO c:\test.csv FROM <1> WHERE sc-status >= 400" -o:CSV -iCheckpoint:test.lpc

LogRecordSet.getRecord() doesn't get whole message

ytsejam1 — Fri, 24 Apr 2009 23:59:59 GMT

Hello,

From some reason the LogRecordSet.getRecord() method doesn't collecting the whole message.
It collect only a part of the string , this is a problem when i have long messages with Exceptions.

Example:

Date -- Service -- Type -- Sub -- Text

2009-04-17 01:15:15,875 -- SeedRetrival -- ERROR -- Inner Error -- Cannot insert duplicate key row in object 'dbo.Tbl_Movie_Frames_1' with unique index 'IX_Frames1'.The statement has been terminated. .Net SqlClient Data Provider at Video.NDS.BL.MovieServiceNew.StoreMovieFrames(String p_MetaId, IList p_movieFrames, String p_Source, String p_Query, Boolean SigIsInBinaryFormat, Single FrameRate) in C:\WebHaystack\102
at SeedsRetrivalService.ItemsProcessor.ProcessSingleItem(ItemToProcess item) in C:\SeedsRetrivalService\ItemsProcessor.cs:line 144

For this message the getRecord() got only the following string:
2009-04-17 01:15:15,875 -- SeedRetrival -- ERROR -- Inner Error -- Cannot insert duplicate key row in object 'dbo.Tbl_Movie_Frames_1' with unique index 'IX_Frames1'

Thanks

Chart generation fails with COM object but not from command line

kjmatt — Thu, 07 May 2009 18:05:15 GMT

I've got a program that works perfectly on two different machines. Both have VS 2008 installed. On a new test server my program works perfectly generating .CSV files and using .TPL files to generate web pages but as soon as it attempts to generate the first chart it crashes the server.

The program hangs for about 30 seconds and then the server states that the connection has been lost and ultimately logs me out, closing all my work. My error log shows no errors and the event log shows nothing either.

Has anyone else seen anything like this?

Windows server 2003 Enterprise SP 2 (Blade virtual server with clean install)
.NET Framework 3.5
Microsoft Office 2003 Excel & Powerpoint
Office Web Components
Logparser.exe

I have LogParser.exe, LogParser.dll and Interop.MSUtil.dll in the same folder as my program. Am I missing something? Is it permission related?

This is driving me crazy.