谷歌的字符编码是什么？

Question

显然，Google 的编码是 UTF-8，正如它在 html 元标记中所述。但是当我使用 ASP WebRequest.GetResponse() 打开 scharfes+s 的搜索页面时，它充满了无法识别的字符。有人知道那里发生了什么吗？

为了您的方便，代码粘贴在下面

Asp 页面

<form id="form1" runat="server">
<div>
    <div runat="server" id="output"/>
</div>
</form>

代码隐藏

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Net;
using System.IO;
using System.Text;

public partial class SearchEngineCaller : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
        HttpWebRequest queryPage = (HttpWebRequest)WebRequest.Create("https://www.google.com/search?q=scharfes+s");
        queryPage.Credentials = CredentialCache.DefaultCredentials;

        HttpWebResponse response = (HttpWebResponse)queryPage.GetResponse();

        Stream receiveStream = response.GetResponseStream();
        StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8);
        output.InnerHtml = readStream.ReadToEnd();
    }
}

我应该使用什么编码？

Answer 1

您必须为该HttpWebRequest对象设置一些 HTTP 标头：

HttpWebRequest queryPage = (HttpWebRequest)WebRequest.Create("https://www.google.com/search?q=scharfes+s");
queryPage.Credentials = CredentialCache.DefaultCredentials;
queryPage.Accept = "text/html";
queryPage.Headers["Accept-Charset"] = "utf-8";
queryPage.UserAgent = "Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/21.0";

重要提示：设置Accept-Charset还不够，设置 也很重要（我从此处User-Agent复制了上面的用户代理字符串）。我尝试了这个解决方案，它对我有用（测试代码）。