Output Targets

LP Stats

hg363 — Tue, 24 Nov 2009 21:57:44 GMT

Good evening to all LP forum members. My question. When a query is executed via the command console LP will generate some stats after executing the query. For example

Statistics:
-----------
Elements processed: 2411
Elements output: 0
Execution time: 1.20 seconds

C:\Program Files\Log Parser 2.2>

How would one get these stats if the query was run from a vbscript. In other words if I run this script

Set objShell = CreateObject("WScript.Shell")

objShell.CurrentDirectory = "\\h505050\C$\Program Files\Log Parser 2.2\"

string_is = "-i:EVT -o:DATAGRID " & """SELECT sourcename, timegenerated AS EventTime, message AS Message FROM \\h101010\system WHERE sourcename like 'USER32%'"""

objshell.run ("LogParser " & string_is)

How after LP has generated the output can I also see the stats as shown above?

Kind regards Ghelani UK

Send raw data input as a column for output

TazzyTazzy — Fri, 20 Nov 2009 23:11:29 GMT

Kind of a wierd request, considering logparser is meant to split up stuff..

I have various log files where the datetime is always the first two columns. I've managed to get that into a single timestamp column. However, the number of columns varies from log file to log file. I want to simply extract the date/time (done), and send the remaining columns as a single column out in CSV (or '|') form.

Example:

Log 1

Date|Time|NID|Some Header|Another Header
11/20/2009|12:29:45|864|Something|or another
11/20/2009|12:29:46|964|Something|or another
11/20/2009|12:29:55|692|Something|or another

Log 2

Here's what I have so far:

select TO_TIMESTAMP(MyDate, MyTime) as DateTime
USING TO_TIMESTAMP(Date, 'M/d/yyyy') as MyDate,
TO_TIMESTAMP(Time, 'hh:mm:ss') as MyTime

So, what can I use in the SELECT section to get the remainder headers? I would like to use the same script across all log files. The log files can change, only thing that stays constant is the Date/Time.

OR...

How do send the entire input line out within a colum. Something like:

SELECT MyDate, * as SomeRow

Obviously, that's not going to work, but that's essentially what I'd like to get done. Thanks.

-Mitch

Personal Daily Log parse

sunchess — Fri, 13 Nov 2009 16:28:53 GMT

Hi! I search a way to parse my personal daily log. I write it in a text file in this way: 13.11.2009 // 17:24:12,98 // Make some staff, and fix the roof 13.11.2009 // 17:24:25,92 // Paint door red (echo %date% // %time% // %*>>D:\users\dailylog.txt) Now I want to view my log like a styled html file. How can i do this? I read manual, and there is extract_token can help to cut out date and leave text only, but cant make a correct query (but im close =) And, more complicated and hard question — can i do an html output with date headers, to view my log by days? Like so: 13.11.2009 first log string second log string etc... 12.11.2009 first log string second log string Thanks!

COMIISOutputContextClassClass uses commas as delimiter? I was expecting the IIS format :(

maximusdm — Tue, 11 Aug 2009 23:43:02 GMT

Hi all,

My log output is using commas instead of using the IIS format. What am I doing wrong?
Here is my code:

public static bool GetResultsFromBatch(string query)
{
LogQueryClassClass LogParser = new LogQueryClassClass();
COMIISW3CInputContextClassClass pObjInput = new COMIISW3CInputContextClassClass();
COMIISOutputContextClassClass pObjOutput = new COMIISOutputContextClassClass(); //comma and without header

try
{
  LogParser.ExecuteBatch(query, pObjInput, pObjOutput);
  return true;
}
catch (Exception e)
{
  return false;
}
}

Thank you!

JScript Template Output

ghostamorx — Tue, 02 Jun 2009 07:56:26 GMT

When creating template output format what are the parameters to show it where the template file is

var oOutputFormat = new ActiveXObject("MSUtil.LogQuery.TemplateOutputFormat");

in logparser command I can execute it like this -o:tpl -tpl:TEMPLATE.tpl>TEMPLATE.htm but how to write it in jscript

thanks

Active Directory Output Problem

hg363 — Tue, 27 Oct 2009 19:02:14 GMT

Hi to all forum members. Please can some one kindly help me with this slight problem.

I am running a query using LP and it works fine, except that the data when written to a CSV file is all on one line. Here is the query

output_path = "'C:\WINDOWS\Temp\joop\" & user_name & "\Group Membership.csv'"

Set objinputformat = CreateObject("MSUtil.LogQuery.ADSInputFormat")

Set objoutputformat = CreateObject("MSUtil.LogQuery.CSVoutputFormat")

objinputformat.objClass = "user"

strQuery = "SELECT cn, memberOf INTO " & output_path & " FROM 'Domain Name' where sAMAccountName='HG363'"

objLogParser.ExecuteBatch strQuery, objInputFormat , objOutputFormat

Please can some one show me what I need to do to fix this, kind regards HG363 London UK

Bug with LogParser - Truncates event log Message string in Datagrid

agc1 — Thu, 22 Oct 2009 19:32:21 GMT

It appears that LogParser truncates the event log message string if using either the DataGrid or SQL outputs (I haven't tested other outputs yet)

Is this a bug with LP or is there a setting that I haven't seen to enable the entire message to be sent to the DataGrid or SQL?

Thanks

SQL output question.

monty124 — Mon, 19 Oct 2009 03:31:18 GMT

HI,I have been browsing the forum for a while and it has helped me in putting together a script to collect Event Logs and dump them into a SQL 2005 database.My question is, am I able to include a custom field to be entered into a custom column in SQL? Basically I want to be able to assign a variable, and write that variable to an additional column in the SQL table along with the data in the event logs.Is this possible and how can I do it?

Thanks.

How to append data

adoyt — Mon, 11 May 2009 16:16:11 GMT

I'm using logparser to upload data to a SQL DB and my DBA has some audit elements that they require be populated. So, I need to add my username and the current time to the list of fields being populated.

TYIA

Code for using COM object for C# .Net with SQL Output

Solburn17 — Mon, 08 Jun 2009 20:45:18 GMT

Here's my code that works. I actually dumped log files from System, Application and Security logs into the database.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

// Log Parser 2.2
using MSUtil;

namespace LogParser2 {
    public partial class _Default :System.Web.UI.Page {
        // constants
        const string iCHECKPOINT_FILE_LOCATION = @"c:\checkpoint.lpc"; // iCheckpoint allows incremental parsing

        protected void Page_Load(object sender, EventArgs e) {
            GetSystemEventLogToSql();
        }

        private void GetSystemEventLogToSql() {
            try {
                // Instantiate the LogQuery object
                LogQueryClassClass LogQuery = new LogQueryClassClass();

                // Instantiate the Event Log Input Format object
                COMEventLogInputContextClassClass EvtInputFormat = new COMEventLogInputContextClassClass();

                // Set Input Parameters.
                EvtInputFormat.direction = "BW";

                // iCheckpoint parses input incrementally. Give it a location.
                EvtInputFormat.iCheckpoint = @"c:\checkpoint.lpc";

                // Instantiate the Event Log Output Format object
                COMSQLOutputContextClassClass EvtOutputFormat = new COMSQLOutputContextClassClass();

                // Set Output Parameters
                EvtOutputFormat.driver = "SQL Server";
                EvtOutputFormat.server = "ServerName";
                EvtOutputFormat.database = "DatabaseName";
                EvtOutputFormat.username = "UserName";
                EvtOutputFormat.password = "Password";

                // I created the tables and added an ID Column. It needs to be ignored during the insert.
                EvtOutputFormat.ignoreIdCols = true;

                // Create the query
                // tblLogWebSystem is the name of the table I am inserting these records into in my DB
                string Query = @"SELECT * FROM TO tblLogWebSystem";

                // Execute the query into SQL
                LogQuery.ExecuteBatch(Query, EvtInputFormat, EvtOutputFormat);

                Response.Write("Done");
            }
            catch(System.Runtime.InteropServices.COMException exc) {
                Response.Write("Unexpected error: " + exc.Message);
            }
        }
    }
}

parameter substitution in vbscript

pilmart — Thu, 01 Oct 2009 11:03:47 GMT

Hi there my query is this, when using logparser at the command-line it's possible to pass DOS variables into template files so that those output variables are 'copied in' to the final output file.

For instance if I set server=server1, I create a variable called server with the value server1 Then in my template file I reference it as %server%.

So when the template file is rendered the value "server1" is substituted Now what I would like to know is can this be replicated when calling logparser using VBScript and if so how ???

Many Thanks

CHART config (value labels)

zumsel — Mon, 07 Sep 2009 13:35:08 GMT

Hi there

I'm using a simple js to pimp my charts. Unfortunately I found out that its not possible to change the angle of X axis labels.

But is it possible to change the color, font and size of the lable on top of columns when using -values:ON

Regards

Trouble with iCheckpoint IISW3C to SQL

jhpope — Thu, 27 Aug 2009 18:30:52 GMT

I am using Logparser to parse IIS logs into SQL. I am trying to use iCheckpoint to avoid duplicate entries. The query works, but when I run the query again, the SQL table doubles and the same entries are added. So it seems it is not picking up from where it left off. I'm just using test logs so nothing is being added to them so I assume no entries should be added to the table.

Here is my query

LogParser "SELECT TO_TIMESTAMP(date, time) AS date, SUBSTR(cs-uri-query, 8) AS csuriquery, cs(Referer) INTO TestTable FROM C:\TestLogs\u_ex*.log WHERE cs-uri-stem = '/Index.Aspx' AND cs-method <> 'POST' AND cs(User-Agent) NOT IN (SELECT cs(User-Agent) FROM C:\TestLogs\bots.list)" -i:IISW3C -iCheckPoint:C:\TestLogs\checkPoint.lpc -o:SQL -server:testServer -database:TestDB

Thanks

SQLXML 3.0 on 64BIT 2008

blyons86 — Thu, 27 Aug 2009 13:39:34 GMT

I have SQLXML 3.0 installed on my 64bit 2008 server. Every time I open Configure IIS Support, I receive an error "An error occurred connecting to 'server'. This server does not exist or is unavailable". Any one have an idea as to what could be causing this? I have 32bit applications in IIS set to 'True'. It works fine on my 32bit 2008 server.

Why is the SQL output target so slow?

PWills — Sun, 09 Aug 2009 03:34:08 GMT

Frustrated with the speed of the SQL output directives, I just ran a little experiment. I used -o:sql to dump a log query on a 10MB IIS file. It took 2:30 (I'm using the COM wrapper). I then used the "interactive mode" in COM, assembled a System.Data.DataTable by hand, and then used System.Data.SqlClient.SqlBulkCopy to bulk insert to the database. It took 18ms. I don't understand why LogParser's -o:sql output directive is over 8000x slower than my crappy code. Am I doing something wrong, or is this a known issue? I'm using SQL Server 2008, if that makes any difference.

Bugreport: XML output with oCodepage=0 produces incorrect XML

suhanovsergey — Thu, 06 Aug 2009 12:28:29 GMT

Hello!

When oCodepage is set to 0 XMLOutputFormat writes an XML with no encoding attribute in <?xml..> section. According to standard that must be treated as UTF-8 by a parser which might result to pasring errors on non-english characters. XMLOutputFormat must write the name of the current system encoding, for instance

<?xml version="1.0" encoding="windows-1251" standalone="yes" ?>

Seems that ETW-live works only with NAT

suhanovsergey — Thu, 06 Aug 2009 11:56:13 GMT

Hello, I want to dump events from live ETW session into XML file at realtime. Another tool is going to parse this file as it is being written. I tried this command: "LogParser "SELECT * INTO out.xml FROM ExampleTrace" -i:ETW -o:XML". That created zero-size file 1.xml and the file was empty even after I had interrupted LogParser with Ctrl+C. I would be pretty ok with any other text-based output format but I couldn't make any output format work as I want. It seems that ETW-live input work ok only with NAT output with default options (-rtp:10) which makes it useless for realtime dumping purposes. "LogParser "SELECT * FROM ExampleTrace" -i:ETW" works OK, "LogParser "SELECT * FROM ExampleTrace" -i:ETW -rtp:-1" does nothing. Of course there is a workaround: use LogQuery and write the data however/whereever I want. Is it a bug or I am doing something wrong?

Generate line chart from a CSV file using numbers as X/Y axes (use two or more measure units for Y axis)

Sean_Zhang — Fri, 24 Jul 2009 17:22:54 GMT

I have a CSV file below:

sequence,value,increment

1,1000000,0

2,1000010,10

3,1000030,20

4,1000060,30

I want to generate a line chart w/ field 'sequence' as X-axis and field 'value'/'increment' as Y-axis.

How to do that?

It is quite simple if the value for field a is string rather than numbers.

Besides, I want to use different measure units for field 'value' and 'increment' as the value of 'increment' is far less than that of 'value'

Thanks in advance~~~

Addational line + data

hg363 — Fri, 17 Jul 2009 15:46:16 GMT

Hi please kindly advise. I am running this query

Set objinputFormat = CreateObject("MSUtil.LogQuery.TEXTLINEInputFormat")Set objOutputFormat = CreateObject("MSUtil.LogQuery.TSVOutputFormat")objOutputFormat.fileMode = 0

objOutputFormat.headers = "Auto"

strQuery = "select top 1 text INTO '\\h505050\C$\wamp\www\helpdesk\nw_audits\" & remote_host & ".txt'" & " from '\\h505050\C$\wamp\www\helpdesk\pc_audits\telent_mac_list\*.txt' where text like '%" & mac_add & "%'"

And it works fine, expect that for some reason, ach time I run the query I get this

Text
212,00:17:a4:4e:41:6a,---,1/9,Level 1 Parkwood House @ HH, 192.168.215.149

The word Text appears + the line of data I actually want. Can some one please tell me how to stop the 1st line TEXT appearing in the output, thank you Regards hg363 UK

Change Line thickness of -o:CHART

pentiumeric — Mon, 06 Jul 2009 04:43:01 GMT

For the chart output, how would i change the thickness of the chart?

Generate Line Chart (3 lines) from a CSV file

pentiumeric — Thu, 02 Jul 2009 02:00:00 GMT

Hello,

CSV file A

Date, Place, Level

12/6, A site, 12.435

13/6, A site, 53.454

14/6, A site, 1.545

12/6, B site, 43.456

13/6, B site, 14.542

14/6, B site, 21.512

12/6, C site, 12.212

13/6, C site, 2.321

14/6, C site, 32.514

I have a CSV file like the one above.

I would like output a line chart, with 3 lines, one of A, one for B and one for C

X-axis would be Date, and Y-axis would be Level.

If the CSV file is organized as below, I could generate the chart I want easily

CSV file B

Date, A Site, B Site C Site

12/6, 12.435, 43.456, 12.212

13/6, 53.454, 14.542, 2.321

14/6, 1.545, 21.512, 32.514

Would anyone advise on plotting the graph directly from CSV file A, or a method to convert file A to B?

I am a newbie of Log Parser, just use it for one week

Regards,

Eric

Trying to get the last 15 min from a hourly IIS log

maximusdm — Tue, 09 Jun 2009 19:24:40 GMT

Hello all,

The query below always returns "zero" elements. What am I doing wrong?
The records on the file below range from 2009-05-30 14:59:59 --> 2009-05-30 15:59:59

logparser "SELECT time, cs-uri-stem INTO C:\Temp\15min.csv FROM C:\Temp\ex09053010.log" -minDateMod:"2009-05-30 15:45:00" -o:csv

Thank you

Inserting higher resolution dates into SQL Server 2008 datetime2 type

MikeGale — Thu, 14 May 2009 06:35:12 GMT

I have some time data in logs with a sub-second time resolution.

I can get this safely into log parser using the iTsFormat setting.

When I put that into SQL Server 2008 I have (so far) gotten the times in seconds (loss of resolution). The target field is datetime2 (not datetime).

How do I preserve the resolution when inserting the data?

Looking for someone to help me with this please

stranger_09 — Tue, 12 May 2009 05:19:59 GMT

I want to create XSLT stlyesheet (task1.xslt) that results in a bilingual dictionary (task1.xml) in XML format. <?xml version="1.0" encoding="UTF‐8"?>

<!DOCTYPE wordList [

<!ELEMENT wordList (wordEntry+)>

<!ELEMENT wordEntry (word+)>

<!ELEMENT word (#PCDATA)>

<!ATTLIST word language (en|es|it|nl) #REQUIRED>

.. .. ..

<word language="it">Caino</word>

<word language="it">Abele</word>

</wordEntry>

<word language="it">Caino Bibbia</word>

<word language="it">Abele Bibbia</word>

<word language="nl">Abel Hebreeuwse bijbel</word>

</wordEntry>

<word language="en">Adagio in G minor</word>

<word language="it">Adagio in Sol minore Albinoni</word>

</wordEntry>

.. .. ..

</wordList>

This is an example of the desired output task1.xml (if we used the example from the previous

section as the input):

<?xml version="1.0" encoding="UTF‐8"?><Dictionary xmlns:xsi="http://www.w3.org/2001/XMLSchema‐instance"

xsi:noNamespaceSchemaLocation="task1.xsd"

from="nl"

to="es">

.. .. ..

</dictionaryEntry>

</dictionaryEntry>

</dictionaryEntry>

Make a webpage showing FTP logs

ghostamorx — Tue, 28 Apr 2009 20:42:15 GMT

I have a problem. I'd like to create a page that shows FTP logs on it. What I mean is that if a client uploads files to ftp server, the page shows exatcly which folder and what time the files were uploaded. I could use LogParser somehow to make SQL queries to change the way the page shows information. I just don't know how or where to start.