This article describes the method of using curl_multi_select to solve the problem of curl_multi web page fake death. Share it for your reference, as follows:
curl_multi can batch process transactions and bring great convenience to web programming. However, in the process of using curl_multi, we will encounter a headache, that is, when there are too many concurrent transactions, the CPU will be too high and the web page will be faked, which cannot be ignored.
Today, by querying relevant information and testing, we finally found a solution to the problem.
Under normal circumstances, this is how we use itcurl_multiof.
Example code:
$connomains = array(
"///",
"http:///",
"/"
);
$mh = curl_multi_init();
foreach ($connomains as $i => $url) {
$conn[$i]=curl_init($url);
curl_setopt($conn[$i],CURLOPT_RETURNTRANSFER,1);
curl_multi_add_handle ($mh,$conn[$i]);
}
do { $n=curl_multi_exec($mh,$active); } while ($active);
foreach ($connomains as $i => $url) {
$res[$i]=curl_multi_getcontent($conn[$i]);
curl_close($conn[$i]);
}
print_r($res);
This example code has a fatal weakness, which is that in the do loop, it is a dead loop during the entire url request period, which will easily cause the CPU to occupy a high level and the web page will be in a fake death state.
After testing, we can use it ingeniouslycurl_multi_select()Functions to solve this problem.
The method is as follows:
Bundle
do { $n=curl_multi_exec($mh,$active); } while ($active);
Change to
do {
$mrc = curl_multi_exec($mh,$active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active and $mrc == CURLM_OK) {
if (curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
Because $active has to wait until all url data is accepted before it becomes false, so herecurl_multi_execThe return value determines whether there is still data, and it keeps calling when there is datacurl_multi_exec, if there is no data for the time being, it will enter the select stage. As soon as the new data comes, it can be awakened to continue execution. The advantage here is that the unnecessary consumption of the CPU is gone.
Another possible problem:
Controls the timeout time of each request, incurl_multi_add_handlePreviously passedcurl_setoptGo to do:
curl_setopt($ch, CURLOPT_TIMEOUT, $timeout);
Determine whether the timeout or other errors,curl_multi_getcontentPreviously used:
curl_error($conn[$i]);
Understand multi interface
When the program needs to make multiple curl concurrent requests, the multi interface provided by curl comes in handy. The smoothness is roughly like this:
1）、curl_multi _initInitialize amulti curlObject. In order to perform concurrent access to multiple curls at the same time, we need to initialize multiple easy curl objects, usingcurl_easy_setoptMake relevant settings.
2) Callcurl_multi _add_handleAdd the easy curl object to the multi curl object.
3) Execute after addingcurl_multi_performMethod for concurrent access.
4) After the visitcurl_multi_remove_handleRemove the relevant easy curl object,curl_easy_cleanupClear the easy curl object.
5) Lastcurl_multi_cleanupClear the multi curl object.
A simple and clear PHP usecurl_multi_add_handleParallel processing example
<?php
// Create a pair of cURL resources$ch1 = curl_init();
$ch2 = curl_init();
// Set URL and corresponding optionscurl_setopt($ch1, CURLOPT_URL, "///");
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch2, CURLOPT_URL, "/");
curl_setopt($ch2, CURLOPT_HEADER, 0);
// Create a batch cURL handle$mh = curl_multi_init();
// Add 2 handlescurl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);
$running=null;
// Execute batch handledo {
curl_multi_exec($mh,$running);
} while($running > 0);
// Close all handlescurl_multi_remove_handle($mh, $ch1);
curl_multi_remove_handle($mh, $ch2);
curl_multi_close($mh);
?>
For more information about PHP related content, please check out the topic of this site:Summary of the usage of php curl》、《Summary of PHP network programming skills》、《Complete collection of PHP array (Array) operation techniques》、《Summary of usage of php strings》、《PHP data structure and algorithm tutorial"and"Summary of data operation techniques for json format in PHP》
I hope this article will be helpful to everyone's PHP programming.