Previous Next

• 02-27-2008, 8:15 AM

  kewlapps Joined on 12-20-2007, 9:40 AM Posts 19

  (Near) Real Time Log viewer Reply Contact Hi, I recently developed a small python program which displays the last 100 line in the log file by piping the output of IIS log file using tail cmd and filter the data for aspx pages using grep and awk and display the output in a HTML table. It also has few features like color coded table row based on predefined threshold for response times and auto-page-refresh every 15 sec. The whole purpose of this is for the support tems to quickly understand the status/responsiveness of out website with out drilling much in the website when come enquires about the web site status. All of this was running my local machine using UniformServer in the intranet. The idea was well recieved and now the management team now wants something in asp.net and hosted on a std. server farm. The challenge I encountered in porting is that I cannot use the unxutils I use on my desktop like tail, grep, awk etc as it is a security issue. So I turned to user LP and wrote query like "select top 100 time as [time], time-taken as RT from \\server\ex080227.log order by [time] desc" to get the last 100 response time entries.  Our log files grow to 300+MB by mid-day so while running this query i noticed that the the entire logfile is downloaded local and read to display the output. So you can imagine the network utilization and performance hit it multiple users use the site. My question is "does logparser have something like select tail 100 time as [time], time-taken as RT from \\server\ex080227.log order by [time] desc" or some better way that I get the last 100 entries without reading the entire log file. Thanks in advance.

• 02-28-2008, 12:17 AM

  In reply to

  qbernard Joined on 03-25-2003, 10:12 PM Malaysia Posts 2,443	Re: (Near) Real Time Log viewer Reply Contact Mm... well if you really want to do so.. you can select top 100 of coz order by the date time in reverse order. no time to test this :) theoretically it should works. Test it out and let us know. Cheers, Bernard Cheah

• 03-13-2008, 6:31 AM

  In reply to

  kewlapps Joined on 12-20-2007, 9:40 AM Posts 19

  Re: (Near) Real Time Log viewer Reply Contact Thanks for the reply. It sure works but as I mentioned before, the query parsers the entire log file, sort all entries and pick the top count, which creates a network overhead for every request as the log files are huge and not on the local drive but on a mapped network drive.  A feature like "tail" similar to "top" but with out the sorting will help. Any ideas on how this can be achieved with minimal network overhead.  Thanks.

• 03-17-2008, 2:21 AM

  In reply to

  qbernard Joined on 03-25-2003, 10:12 PM Malaysia Posts 2,443

  Re: (Near) Real Time Log viewer Reply Contact Mm... don't parse it over network then ? or parse it locally then output to a smaller file, then view it at remote? there's no tail command afaik. mm... what about -i ? checkpoint ? check point will tell the last point / row of process and will not read whatever that processed before ...  try it out and let us know again :)  sorry man.. been lazy lately. Cheers, Bernard Cheah

• 08-20-2008, 3:35 PM

  In reply to

  deathwagon Joined on 08-20-2008, 7:27 PM Posts 3	Re: (Near) Real Time Log viewer Reply Contact I'm not sure I fully understand what you are after here however it sounds like the -iCheckpoint switch may be of use to you. "Logparser -h -i:IISW3C"

• 08-21-2008, 2:43 AM

  In reply to

  yellowdog.dave Joined on 07-18-2008, 7:17 AM Johannesburg, South Africa Posts 81

  Re: (Near) Real Time Log viewer Reply Contact Hi There,  Why don't you look at using psexec, part of the PSTOOLS suite http://technet.microsoft.com/en-us/sysinternals/bb896649.aspx to execute your query on the remote system and pipe the results back to your local system? Doing this, you will minimise your network traffic both ways, combine it with the iCheckpoint parameter and you should be good to go.  Alternatively, as the logs are already on an IIS server, why not consider publishing a little page on that server for monitoring using a TPL (template file) and colour coding your information. Take a look here for some great examples http://blogs.iis.net/chrisad/archive/2006/07/13/Chris.aspx Then everything stays on the server and you generate no traffic.  If you didn't know, the PSTOOLS suite is another awesome little collection of free utilities that should, in my opinion,  be part of every Windoze technicians arsenal. Hope this helps.  Good luck. Cheers, Dave  Yes, dear

• 09-04-2008, 1:21 AM

  In reply to

  kewlapps Joined on 12-20-2007, 9:40 AM Posts 19	Re: (Near) Real Time Log viewer Reply Contact Thanks Dave for the reply. Yes I am aware of pstools, they are awesome utilities. But in my case the web servers are on a managed environment and so and so no access to web serve m/c to run these utilities. Also the logfiles are dumped onto a share drive and are made available as url's for access. Many Thanks for the info.