为什么Perl HTTP :: Response没有解码这个撇号？

Question

我正在使用

my $ua = new LWP::UserAgent;
$ua->agent("Mozilla/5.0 (Windows NT 6.1; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 ");
my $url = "http://somedomain.com/page/";
my $req = new HTTP::Request 'GET' => $url;
$req->header('Accept' => 'text/html');
my $response = $ua->request($req);
my $html = $response->decoded_content;

获得一个网页.在此页面上,Abobo's Big Adventure出现.在$request->content和中$request->decoded_content,这显示为Abobo's Big Adventure.

我可以做些什么来正确解码吗？

Answer 1

为什么,这是完全有效的HTML!但是,您可以使用HTML::EntitiesCPAN 解码实体.

use HTML::Entities;

...;
my $html = $response->decoded_content;
my $decoded_string = decode_entities($html);

HTTP::Response::decoded_content声明Content-encoding和charsets颠倒过来的文档,而不是HTML实体(HTML/XML语言功能,不是真正的编码).

编辑:

但是,正如ikegami指出的那样,立即解码实体可能会使HTML无法解析.因此,最好先解析HTML(例如使用HTML::Tree),然后在需要时仅解码文本节点.

use HTML::TreeBuilder;

my $url = ...;
my $tree = HTML::TreeBuilder->new_from_url($url);    # invokes LWP automatically
my $decoded_text = decode_entities($tree->as_text);  # dumps the tree as flat text, then decodes.