Don't foget to curl_close($ch); Even if curl_errno($ch) != 0
Because if you don't - on Windows this will produce windows-error-report (Program terminated unexpectedly)
Don't foget to curl_close($ch); Even if curl_errno($ch) != 0
Because if you don't - on Windows this will produce windows-error-report (Program terminated unexpectedly)
This is sample script to use curl, Just input curl_setopt,
exp :
curlsetop[0] ==> name : CURLOPT_URL ; value : http://amazon.com
curlsetop[1] ==> name : CURLOPT_RETURNTRANSFER ; value : true
curlsetop[2] ==> name : CURLOPT_FOLLOWLOCATION ; value : true
You can add form input.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title> New Document </title>
<meta name="Generator" content="">
<meta name="Author" content="Helmi Anwar">
<meta name="Keywords" content="">
<meta name="Description" content="">
</head>
<body>
<form method="post" action="">
<table>
<tr class="rowid_add">
<td>curl_setopt [0]</td>
<td>Name : <input type="text" size="50" name="setopt_name[]"></td>
<td>Value :<input type="text" size="50" name="setopt_value[]"></td>
</tr>
<tr class="rowid_add">
<td>curl_setopt [0]</td>
<td>Name : <input type="text" size="50" name="setopt_name[]"></td>
<td>Value :<input type="text" size="50" name="setopt_value[]"></td>
</tr>
<tr class="rowid_add">
<td>curl_setopt [0]</td>
<td>Name : <input type="text" size="50" name="setopt_name[]"></td>
<td>Value :<input type="text" size="50" name="setopt_value[]"></td>
</tr>
<tr>
<td><input type="submit" name="submit_yes" value="EXECUTE"></td>
</tr>
</table>
</form>
<?php
function curl_test($setopt_content)
{
$ch = curl_init();
curl_setopt_array($ch, $setopt_content);
$result_data = curl_exec($ch);
curl_close($ch);
return $result_data;
}
if($_REQUEST['submit_yes']=="EXECUTE")
{
foreach ($_REQUEST['setopt_name'] as $k => $index_content)
{
$value_content=$_REQUEST['setopt_value'][$k];
$index_content =strtoupper($index_content);
eval('$index_content = '.$index_content.';');
//echo ($index_content);
if($index_content!='')
{
if(strtoupper($value_content)=='TRUE')
{$setopt_content[$index_content]=TRUE;}
elseif(strtoupper($value_content)=='FALSE')
{$setopt_content[$index_content]=FALSE;}
else
{$setopt_content[$index_content]=$value_content;}
}
}
$info=curl_test($setopt_content);
}
?>
<textarea name="result" rows="25" cols="100"><?php echo htmlspecialchars($info);?></textarea>
</body>
</html>
Note, if you will ever need to get/set unique handle for Curl-object, you might need to use CURL_PRIVATE property for each instace
https://www.php.net/manual/en/function.curl-setopt.php
A note of warning for PHP 5 users: if you try to fetch the CURLINFO_CONTENT_TYPE using curl_getinfo when there is a connect error, you will core dump PHP. I have informed the Curl team about this, so it will hopefully be fixed soon. Just make sure you check for an error before you look for this data.
Although it has been noted that cURL outperforms both file_get_contents and fopen when it comes to getting a file over a HTTP link, the disadvantage of cURL is that it has no way of only reading a part of a page at a time.
For example, the following code is likely to generate a memory limit error:
<?php
$ch = curl_init("http://www.example.com/reallybigfile.tar.gz");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
$output = curl_exec($ch);
$fh = fopen("out.tar.gz", 'w');
fwrite($fh, $output);
fclose($fh);
?>
While this, on the other hand, wouldn't
<?php
$hostfile = fopen("http://www.example.com/reallybigfile.tar.gz", 'r');
$fh = fopen("out.tar.gz", 'w');
while (!feof($hostfile)) {
$output = fread($hostfile, 8192);
fwrite($fh, $output);
}
fclose($hostfile);
fclose($fh);
?>
For anyone trying to use cURL to submit to an ASP/ASPX page that uses an image as the submit button.
Make sure that you have 'button_name.x' and 'button_name.y' in the post fields. PHP names these fields 'button_name_x' and 'button_name_y', while ASP uses a dot.
Also, as noted above, be sure to include the '__VIEWSTATE' input field in your post request.
It took me quite some to to figure out how to get Curl (with SSL), OpenSSL and PHP to play nicely together.
After reinstalling MS-VC7 and compiling OpenSSL to finally realise this was'nt nesscary.
If your like me and like *Nix systems more than Windows then you'll most probly have similar problems.
I came across this, on a simple google with the right keywords.
http://www.tonyspencer.com/journal/00000037.htm
I read thru that and found my mistake.
Its just a small list of notes, I found them to be the best I've found on the subject and the most simplist.
Dont forget to add a simple line like this into your scripts to get them working on Win32.
<?php
if($WINDIR) curl_setopt($curl, CURLOPT_CAINFO, "c:\\windows\\ca-bundle.crt");
?>
Last note: ca-bundle.crt file is located in the Curl download. I stored mine in the windows directory and apache/php can access it fine.
All the best and I hope this helps.
Simon Lightfoot
vHost Direct Limited
In recent versions of php, CURLOPT_MUTE has (probably) been deprecated. Any attempt of using curl_setopt() to set CURLOPT_MUTE will give you a warning like this:
PHP Notice: Use of undefined constant CURLOPT_MUTE - assumed 'CURLOPT_MUTE' in ....
If you wish tu silence the curl output, use the following instead:
<?php
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
?>
And then,
<?php
$curl_output=curl_exec($ch);
?>
The output of the curl operation will be stored as a string in $curl_output while the operation remains totally silent.
Fixed bugs in the function posted earlier (better javascript redirect following and now supports HTTPS)
<?php
/*==================================
Get url content and response headers (given a url, follows all redirections on it and returned content and response headers of final url)
@return array[0] content
array[1] array of response headers
==================================*/
function get_url( $url, $javascript_loop = 0, $timeout = 5 )
{
$url = str_replace( "&", "&", urldecode(trim($url)) );
$cookie = tempnam ("/tmp", "CURLCOOKIE");
$ch = curl_init();
curl_setopt( $ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1" );
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_COOKIEJAR, $cookie );
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_ENCODING, "" );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_AUTOREFERER, true );
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false ); # required for https urls
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, $timeout );
curl_setopt( $ch, CURLOPT_TIMEOUT, $timeout );
curl_setopt( $ch, CURLOPT_MAXREDIRS, 10 );
$content = curl_exec( $ch );
$response = curl_getinfo( $ch );
curl_close ( $ch );
if ($response['http_code'] == 301 || $response['http_code'] == 302)
{
ini_set("user_agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1");
if ( $headers = get_headers($response['url']) )
{
foreach( $headers as $value )
{
if ( substr( strtolower($value), 0, 9 ) == "location:" )
return get_url( trim( substr( $value, 9, strlen($value) ) ) );
}
}
}
if ( ( preg_match("/>[[:space:]]+window\.location\.replace\('(.*)'\)/i", $content, $value) || preg_match("/>[[:space:]]+window\.location\=\"(.*)\"/i", $content, $value) ) &&
$javascript_loop < 5
)
{
return get_url( $value[1], $javascript_loop+1 );
}
else
{
return array( $content, $response );
}
}
?>
Using curl to take snapshots of the current page for emailing the HTML is a clever little idea. (ie: Email this page to a friend)
<?php
//to be explained below!
session_write_close();
$pageurl = "http://www.site.com/content.php?PHPSESSID=123XYZ
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_URL, $pageurl );
$html = curl_exec ( $ch );
curl_close($ch);
//then you need to fix pathing to absolute
$search = "/(src|href|background)=\"[^:,^>,^\"]*\"/i";
preg_match_all ( $search, $html, $a_matches );
//you can figure out the rest ! but thought the reg expression is useful as well
?>
But here is the catch, you may want to make sure curl connects to the server under the same session as the browser. So naturally you pass the session cookie through the curl system either by the cookie jar system, or through the query string in the path.
This is where you will get stuck. PHP will need write access to the same session file simultaneously!! causing serious hanging issues!
This is why you should close off your session before you make curl take a page snapshot!
Beware of any extra spaces in the URL. A trailing space in the URL caused my script to fail with the message "empty reply from server".
I had the following experience when harvesting urls with a get variable from a page using cUrl. HTML pages will output ampersands as & when the page is read by the curl function.
If you code a script to find all hyperlinks, it will use & instead of &, especially using a regular expression search.
It is hard to detect because when you output the url to the browser it renders the html. To fix, add a line to replace the & with &.
<?php
function processURL($url){
$url=str_replace('&','&',$url);
$ch=curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
$xml = curl_exec ($ch);
curl_close ($ch);
echo $xml;
}
?>
Note that on Win32 this documentation can get a little confusing.
In order to get this to work you need to:
1) Be sure that the folder where libeay32.dll and ssleay32.dll - tipically C:\\PHP - is present on the PATH variable.
2) Uncomment - remove the semi-colon - the line that says "extension=php_curl.dll" from php.ini
3) Restart the webserver (you should already know this one, but...)
It took me some time to realize this, since this page doesn't mention the need to uncomment that php.ini's line.
This is sample script to use curl, Just input curl_setopt,
exp :
curlsetop[0] ==> name : CURLOPT_URL ; value : https://amzn.to/3njlWW6
curlsetop[1] ==> name : CURLOPT_RETURNTRANSFER ; value : true
curlsetop[2] ==> name : CURLOPT_FOLLOWLOCATION ; value : true
You can add form input.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title> New Document </title>
<meta name="Generator" content="">
<meta name="Author" content="Helmi Anwar">
<meta name="Keywords" content="">
<meta name="Description" content="">
</head>
<body>
<form method="post" action="">
<table>
<tr class="rowid_add">
<td>curl_setopt [0]</td>
<td>Name : <input type="text" size="50" name="setopt_name[]"></td>
<td>Value :<input type="text" size="50" name="setopt_value[]"></td>
</tr>
<tr class="rowid_add">
<td>curl_setopt [0]</td>
<td>Name : <input type="text" size="50" name="setopt_name[]"></td>
<td>Value :<input type="text" size="50" name="setopt_value[]"></td>
</tr>
<tr class="rowid_add">
<td>curl_setopt [0]</td>
<td>Name : <input type="text" size="50" name="setopt_name[]"></td>
<td>Value :<input type="text" size="50" name="setopt_value[]"></td>
</tr>
<tr>
<td><input type="submit" name="submit_yes" value="EXECUTE"></td>
</tr>
</table>
</form>
<?php
function curl_test($setopt_content)
{
$ch = curl_init();
curl_setopt_array($ch, $setopt_content);
$result_data = curl_exec($ch);
curl_close($ch);
return $result_data;
}
if($_REQUEST['submit_yes']=="EXECUTE")
{
foreach ($_REQUEST['setopt_name'] as $k => $index_content)
{
$value_content=$_REQUEST['setopt_value'][$k];
$index_content =strtoupper($index_content);
eval('$index_content = '.$index_content.';');
//echo ($index_content);
if($index_content!='')
{
if(strtoupper($value_content)=='TRUE')
{$setopt_content[$index_content]=TRUE;}
elseif(strtoupper($value_content)=='FALSE')
{$setopt_content[$index_content]=FALSE;}
else
{$setopt_content[$index_content]=$value_content;}
}
}
$info=curl_test($setopt_content);
}
?>
<textarea name="result" rows="25" cols="100"><?php echo htmlspecialchars($info);?></textarea>
</body>
</html>
If you have upgraded to using thread safe PHP (with apache 2 MPM=worker) note that
CURLOPT_COOKIEJAR / CURLOPT_COOKIEFILE both need an absolute path set for the cookie file location and no longer take a relative path.
(As before, also remember to have set correct permissions to allow a writeable cookie file/dir by apache)
[php 4.3.7/apache v2.0.49]