PHP cURL,写入文件

Question

我想尝试连接到远程文件并将输出从那里写入本地文件,这是我的功能:

function get_remote_file_to_cache()
{

$the_site="http://facebook.com";

    $curl = curl_init();
    $fp = fopen("cache/temp_file.txt", "w");
    curl_setopt ($curl, CURLOPT_URL, $the_site);
    curl_setopt($curl, CURLOPT_FILE, $fp);

    curl_setopt($curl,  CURLOPT_RETURNTRANSFER, TRUE);


    curl_exec ($curl);



    $httpCode = curl_getinfo($curl, CURLINFO_HTTP_CODE);
    if($httpCode == 404) {
        touch('cache/404_err.txt');
    }else
    {

    touch('cache/'.rand(0, 99999).'--all_good.txt');
    }


    curl_close ($curl);
}

它在"cache"目录中创建了两个文件,但问题是它没有将数据写入"temp_file.txt",为什么会这样？

谢谢!
[R

Answer 1

实际上,使用fwrite是部分正确的.为了避免大文件的内存溢出问题(超过PHP的最大内存限制),您需要设置一个回调函数来写入该文件.

注意:我建议专门创建一个类来处理文件下载和文件句柄等,而不是使用全局变量,但是对于本示例的目的,下面显示了如何启动和运行.

所以,请执行以下操作:

# setup a global file pointer
$GlobalFileHandle = null;

function saveRemoteFile($url, $filename) {
  global $GlobalFileHandle;

  set_time_limit(0);

  # Open the file for writing...
  $GlobalFileHandle = fopen($filename, 'w+');

  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_FILE, $GlobalFileHandle);
  curl_setopt($ch, CURLOPT_HEADER, 0);
  curl_setopt($ch, CURLOPT_USERAGENT, "MY+USER+AGENT"); //Make this valid if possible
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); # optional
  curl_setopt($ch, CURLOPT_TIMEOUT, -1); # optional: -1 = unlimited, 3600 = 1 hour
  curl_setopt($ch, CURLOPT_VERBOSE, false); # Set to true to see all the innards

  # Only if you need to bypass SSL certificate validation
  curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
  curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);

  # Assign a callback function to the CURL Write-Function
  curl_setopt($ch, CURLOPT_WRITEFUNCTION, 'curlWriteFile');

  # Exceute the download - note we DO NOT put the result into a variable!
  curl_exec($ch);

  # Close CURL
  curl_close($ch);

  # Close the file pointer
  fclose($GlobalFileHandle);
}

function curlWriteFile($cp, $data) {
  global $GlobalFileHandle;
  $len = fwrite($GlobalFileHandle, $data);
  return $len;
}

您还可以创建一个进度回调来显示您下载的速度/速度,但这是另一个例子,因为在输出到CLI时它可能很复杂.

从本质上讲,这将下载每个数据块,并立即将其转储到文件中,而不是先将ENTIRE文件下载到内存中.

这样做更安全!当然,您必须确保URL正确(将空格转换为%20等)并且本地文件是可写的.

干杯,詹姆斯.

Answer 2

让我们尝试发送GET请求http://facebook.com:

$ curl -v http://facebook.com
* Rebuilt URL to: http://facebook.com/
* Hostname was NOT found in DNS cache
*   Trying 69.171.230.5...
* Connected to facebook.com (69.171.230.5) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: facebook.com
> Accept: */*
> 
< HTTP/1.1 302 Found
< Location: https://facebook.com/
< Vary: Accept-Encoding
< Content-Type: text/html
< Date: Thu, 03 Sep 2015 16:26:34 GMT
< Connection: keep-alive
< Content-Length: 0
< 
* Connection #0 to host facebook.com left intact

发生了什么？似乎Facebook将我们重定向http://facebook.com到安全https://facebook.com/.注意什么是响应体长:

Content-Length: 0

这意味着将写入零字节xxxx--all_good.txt.这就是文件保持空白的原因.

您的解决方案绝对正确:

$fp = fopen('file.txt', 'w');
curl_setopt($handle, CURLOPT_FILE, $fp);
curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);

您需要做的就是将URL更改为https://facebook.com/.

关于其他答案:

• @JonGauthier:没有,没有必要使用fwrite()后curl_exec()
• @doublehelix:不,你不需要CURLOPT_WRITEFUNCTION将内容复制到文件这么简单的操作.
• @ScottSaunders:touch()如果不存在,则创建空文件.我认为这是OP的意图.

说真的,三个答案和每个答案都是无效的？

Answer 3

您需要使用显式写入文件fwrite,并将其传递给您之前创建的文件句柄:

if ( $httpCode == 404 ) {
    ...
} else {
    $contents = curl_exec($curl);
    fwrite($fp, $contents);
}

curl_close($curl);
fclose($fp);

Answer 4

在你的问题中，你有

    curl_setopt($curl, CURLOPT_FILE, $fp);

    curl_setopt($curl,  CURLOPT_RETURNTRANSFER, TRUE);

但是来自 PHP 的 curl_setopt 文档说明...

It appears that setting CURLOPT_FILE before setting CURLOPT_RETURNTRANSFER doesn't work, presumably because CURLOPT_FILE depends on CURLOPT_RETURNTRANSFER being set.

So do this:

<?php
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FILE, $fp);
?>

not this:

<?php
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
?>

...声明“ CURLOPT_FILE 取决于设置的 CURLOPT_RETURNTRANSFER ”。

参考：https : //www.php.net/manual/en/function.curl-setopt.php#99082