one. Memory overflow solution
When doing data statistics analysis, I often encounter large arrays and memory overflow may occur. Here I will share my solution. Let’s use examples to illustrate this problem, as follows:
Assuming that the number of records stored in the log is 500,000, the solution is as follows:
Copy the codeThe code is as follows:
ini_set('memory_limit','64M'); //Reset the memory size that can be used by php is 64M. Generally, files cannot be modified on remote hosts and can only be set through the program. Note: In safe_mode (safe mode), ini_set fails
set_time_limit(600);//Set the timeout limit to 6 minutes
$farr = $Uarr = $Marr = $IParr = $data = $_sub = array();
$spt = ”$@#!$”;
$root = ”/Data/webapps/VisitLog”;
$path = $dpath = $fpath = NULL;
$path = $root.”/”.date(“Y-m”,$timestamp);
$dpath = $path.”/”.date(“m-d”,$timestamp);
for($j=0;$j<24;$j++){
$v = ($j < 10) ? ”0″.$j : $j;
$gpath = $dpath.”/”.$v.”.php”;
if(!file_exists($gpath)){
continue;
} else {
$arr = file($gpath);//// Read the file into the array
array_shift($arr);//Move out of the first unit-》<?php exit;?>
$farr = array_merge($farr,$arr);
unset($arr);
}
}
if(empty($this->farr)){
echo ”<p><center>No relevant records!</center></p>”;
exit;
}
while(!empty($farr)){
$_sub = array_splice($farr, 0, 10000); // Take out 1000 of $farr each time
for($i=0,$scount=count($_sub);$i<$scount;$i++){
$arr = explode($spt,$_sub[$i]);
$Uarr[] = $arr[1]; //vurl
$Marr[] = $arr[2]; //vmark
$IParr[] = $arr[3].” |$nbsp;”.$arr[1]; //IP
}
unset($_sub);//Destroy it in time after use
}
unset($farr);
Here, it is not difficult to see that on the one hand, we need to increase the available memory size of PHP, and on the other hand, as long as we find a way to batch the array, divide and conquer it, and destroy used variables in time, there will generally be no overflow problem.
In addition, in order to save PHP program memory loss, we should minimize the use of static variables as much as possible. When data reuse is required, we can consider using references (&). Another point is: after the database operation is completed, the connection must be closed immediately; after an object is used, the destructor (__destruct()) must be called in time.
two. Unset destroys variables and frees memory issues
PHP's unset() function is used to clear and destroy variables. We can use unset() to destroy variables. But sometimes, using unset() cannot achieve the memory occupied by the variable! Let's first look at an example:
Copy the codeThe code is as follows:
<?php
$s=str_repeat('1',255); //Create a string composed of 255 1s
$m=memory_get_usage(); //Get the current memory occupied
unset($s);
$mm=memory_get_usage(); //unset() before checking the current memory occupied
echo $m-$mm;
?>
Finally, the memory is occupied before unset() is used. If it is a positive number, it means that unset($s) has destroyed $s from memory (or in other words, the memory usage is reduced after unset()). However, the result I got in PHP5 and Windows platforms is: 0. Does this mean that unset($s) does not play the role of destroying the memory occupied by the variable $s? Let's make the following example:
Copy the codeThe code is as follows:
<?php
$s=str_repeat('1',256); //Create a string composed of 256 1s
$m=memory_get_usage(); //Get the current memory occupied
unset($s);
$mm=memory_get_usage(); //unset() before checking the current memory occupied
echo $m-$mm;
?>
This example is almost the same as the above example. The only difference is that $s consists of 256 1s, that is, there is one more 1 than the first example, and the result is: 272. Does this mean that unset($s) has destroyed the memory occupied by $s?
Through the above two examples, we can draw the following conclusions:
Conclusion 1. The unset() function can only free up memory space when the variable value occupies more than 256 bytes of memory space.
So can I use unset to free up memory space as long as the variable value exceeds 256? Let's test it with another example:
Copy the codeThe code is as follows:
<?php
$s=str_repeat('1',256); //This is exactly the same as the second example
$p=&$s;
$m=memory_get_usage();
unset($s); //Destroy $s
$mm=memory_get_usage();
echo $p.'<br />';
echo $m-$mm;
?>
Refreshing the page, we see that the first line has 256 1s and the second line is 0. Logically speaking, we have destroyed $s, and $p is just a variable that references $s, so there should be no content. In addition, the memory usage before and after unset($s) has not changed! Now let's do the following example:
Copy the codeThe code is as follows:
<?php
$s=str_repeat('1',256); //This is exactly the same as the second example
$p=&$s;
$m=memory_get_usage();
$s=null; //Set $s to null
$mm=memory_get_usage();
echo $p.'<br />';
echo $m-$mm;
?>
Now refreshing the page, we see that the output $p is no longer content, and the difference in memory usage before and after unset() is 272, that is, the memory occupied by the variable has been cleared. In this example, $s=null can also be replaced with unset(), as follows:
Copy the codeThe code is as follows:
<?php
$s=str_repeat('1',256); //This is exactly the same as the second example
$p=&$s;
$m=memory_get_usage();
unset($s); //Destroy $s
unset($p);
$mm=memory_get_usage();
echo $p.'<br />';
echo $m-$mm;
?>
We use unset() to destroy both $s and $p. At this time, we look at the difference in memory usage is 272, which means that this can also free up memory. Then, we can get another conclusion:
Conclusion 2. Memory will be released only when all variables (such as reference variables) pointing to the variable are destroyed.