reading a log file one line at a time depending on timestamp

[eon201]

Hi,

Currently I am attepting to create a stats package for my companies website (yes my life would be 100% stress free if I was allowed to use a standard/commercial stats package ).

So far things are all good, and ive progressed quite far. But ive hit a snag.

At the moment im reading in the WHOLE log file and sorting it into an array and passing the neccesary data to mysql. Easy peasy.

But my boss has pointed out to me that as the site gets more and more traffic the log gets bigger, and if it gets to big the servers memory will overload, and we all know what happens then!

So what I need to do is read in the file one line at a time. and compare the timestamp to the last time the script was run. If the timestamp in the log is less than the last timestamp for when the file was run dont read it. If it is more than defiently read it.

so far my code looks like so...

PHP: Select all

$filename = '../../logs\W3SVC10019'. '/ex'. date('ymd'). '.log'; // Filename Checker
            
    if (file_exists($filename)) // Does the corresponding filename exist - if yes then
    {
        $fp = fopen($filename, "r"); //Open the server log
        $content = fread($fp, filesize($filename));     // Read the server log    
        $content = explode("\n", $content); // explode into array    
        $content  = array_reverse($content ); // reverse the array
        foreach($content as $key=>$value)    {

//This is where im sorting all the array data
} 

So what needs to happen is that the script reads each line which look like...

2007-11-22 23:59:59 W3SVC10019 **.***.***.*** GET /images/e-br...

then grab '2007-11-22 23:59:59' and convert to timestamp ( strtotime )
then compare it to the mysql stored last run time of the script.

If the timestamp is less than the mysql version. Ignore
If it is more read it in.

How do I read one line at a time??
But only the first date/time part, and then figure out true/false??


I hope this all makes sense. Ive tried to explain this fully. If it all still does not make sense, please ask for more info!

Ok thanks in advance! Eon201

[Rakuli]

To read one file at a time use

fgets(); and check if you have reached the end of the file using feof

PHP: Select all

while (!feof($file)) {

        $line = fgets($file, 4096);

        // do your date and time checking here. If you want to stop the running through of the file

// just use break;

    }