从网站获取请求并检索响应?

Jon*_*orm 5 javascript jquery greasemonkey

我正在尝试监控一个网站(www.bidcactus.com)。在网站上,我打开 Firebug,转到网络选项卡,然后单击 XHR 选项卡。

我想获取请求的响应并将其保存到 mySql 数据库(我的计算机上运行着本地数据库(XAMPP)。

有人告诉我主要使用 jQuery 或 JavaScript 来做各种事情,但我也没有经验,所以我想知道是否有人可以帮助我。

有人向我建议了这个链接 Using Greasemonkey and jQuery to截取来自页面的JSON/AJAX数据,并处理它

它也使用 Greasemonkey,我对此也不太了解......

预先感谢您的任何帮助

示例/更多细节:
在监视发送的请求时(通过 firebug)我看到下面

http://www.bidcactus.com/CactusWeb/ItemUpdates?rnd=1310684278585
The response of this link is the following:
{"s":"uk5c","a":[{"w":"MATADORA","t":944,"p":5,"a":413173,"x":10},   
{"w":"1000BidsAintEnough","t":6,"p":863,"a":413198,"x":0}, 
{"w":"YourBidzWillBeWastedHere","t":4725,"p":21,"a":413200,"x":8}, 
{"w":"iwillpay2much","t":344,"p":9,"a":413201,"x":9}, 
{"w":"apcyclops84","t":884,"p":3,"a":413213,"x":14}, 
{"w":"goin_postal","t":165,"p":5,"a":413215,"x":12}, 
{"w":"487951","t":825,"p":10,"a":413218,"x":6}, 
{"w":"mishmash","t":3225,"p":3,"a":413222,"x":7}, 
{"w":"CrazyKatLady2","t":6464,"p":1,"a":413224,"x":2}, 
{"w":"BOSS1","t":224,"p":102,"a":413230,"x":4}, 
{"w":"serbian48","t":62,"p":2,"a":413232,"x":11}, 
{"w":"Tuffenough","t":1785,"p":1,"a":413234,"x":1}, 
{"w":"apcyclops84","t":1970,"p":1,"a":413240,"x":13}, 
{"w":"Tuffenough","t":3524,"p":1,"a":413244,"x":5}, 
{"w":"Cdm17517","t":1424,"p":1,"a":413252,"x":3}],"tau":"0"}
Run Code Online (Sandbox Code Playgroud)

我了解这些信息的含义,并且我认为我可以自己对其进行格式化,但是该网站会随机创建新的请求。
示例http://www.bidcactus.com/CactusWeb/ItemUpdates?rnd=XXXXXXXXXXXX
,我不确定它是如何创建它们的。

因此,我需要获取所有项目更新请求的响应,并将信息发送到 mysql 数据库。

Bro*_*ams 5

好的,这是工作代码,针对该网站进行了一些调整(仅首页,无帐户)。

\n

使用说明:

\n
    \n
  1. 安装GM脚本。请注意,目前仅限 Firefox。

    \n
  2. \n
  3. 观察它在 Firebug 控制台中的运行,并调整过滤器部分(清楚标记),以定位您感兴趣的数据。(也许是整个a数组?)

    \n

    请注意,打印“Script Start”后可能需要几秒钟的时间才能启动 ajax 拦截。

    \n
  4. \n
  5. 设置您的 Web 应用程序和服务器以接收数据。该脚本发布 JSON,因此 PHP 会抓取数据,如下所示:

    \n
     $jsonData   = json_decode ($HTTP_RAW_POST_DATA);\n
    Run Code Online (Sandbox Code Playgroud)\n
  6. \n
  7. 将脚本指向您的服务器。

    \n
  8. \n
  9. 瞧\xc3\xa0。她完成了。

    \n
  10. \n
\n
\n
/******************************************************************************\n*******************************************************************************\n**  This script intercepts ajaxed data from the target web pages.\n**  There are 4 main phases:\n**      1)  Intercept XMLHttpRequest's made by the target page.\n**      2)  Filter the data to the items of interest.\n**      3)  Transfer the data from the page-scope to the GM scope.\n**          NOTE:   This makes it technically possibly for the target page's\n**                  webmaster to hack into GM's slightly elevated scope and\n**                  exploit any XSS or zero-day vulnerabilities, etc.  The risk\n**                  is probably zero as long as you don't start any feuds.\n**      4)  Use GM_xmlhttpRequest () to send the data to our server.\n*******************************************************************************\n*******************************************************************************\n*/\n// ==UserScript==\n// @name            _Record ajax, JSON data.\n// @namespace       stackoverflow.com/users/331508/\n// @description     Intercepts Ajax data, filters it and then sends it to our server.\n// @include         http://www.bidcactus.com/*\n// ==/UserScript==\n\nDEBUG   = true;\nif (DEBUG)  console.log ('***** Script Start *****');\n\n\n/******************************************************************************\n*******************************************************************************\n**  PHASE 1 starts here, this is the XMLHttpRequest intercept code.\n**  Note that it will not work in GM's scope.  We must inject the code to the\n**  page scope.\n*******************************************************************************\n*******************************************************************************\n*/\nfunkyFunc   = ( (<><![CDATA[\n\n    DEBUG           = false;\n    //--- This is where we will put the data we scarf. It will be a FIFO stack.\n    payloadArray    = [];   //--- PHASE 3a\n\n    (function (open) {\n        XMLHttpRequest.prototype.open = function (method, url, async, user, pass)\n        {\n            this.addEventListener ("readystatechange", function (evt)\n            {\n                if (this.readyState == 4  &&  this.status == 200)  //-- Done, & status "OK".\n                {\n                    var jsonObj = null;\n                    try {\n                        jsonObj = JSON.parse (this.responseText);   // FF code.  Chrome??\n                    }\n                    catch (err) {\n                        //if (DEBUG)  console.log (err);\n                    }\n                    //if (DEBUG)  console.log (this.readyState, this.status, this.responseText);\n\n                    /******************************************************************************\n                    *******************************************************************************\n                    **  PHASE 2:    Filter as much as possible, at this stage.\n                    **              For this site, jsonObj should be an object like so:\n                    **                  { s="1bjqo", a=[15], tau="0"}\n                    **              Where a is an array of objects, like:\n                    **                  a   417387\n                    **                  p   1\n                    **                  t   826\n                    **                  w   "bart69"\n                    **                  x   7\n                    *******************************************************************************\n                    *******************************************************************************\n                    */\n                    //if (DEBUG)  console.log (jsonObj);\n                    if (jsonObj  &&  jsonObj.a  &&  jsonObj.a.length > 1) {\n                        /*--- For demonstration purposes, we will only get the 2nd row in\n                            the `a` array. (Probably stands for "auction".)\n                        */\n                        payloadArray.push (jsonObj.a[1]);\n                        if (DEBUG)  console.log (jsonObj.a[1]);\n                    }\n                    //--- Done at this stage!  Rest is up to the GM scope.\n                }\n            }, false);\n\n            open.call (this, method, url, async, user, pass);\n        };\n    } ) (XMLHttpRequest.prototype.open);\n]]></>).toString () );\n\n\nfunction addJS_Node (text, s_URL)\n{\n    var scriptNode                      = document.createElement ('script');\n    scriptNode.type                     = "text/javascript";\n    if (text)  scriptNode.textContent   = text;\n    if (s_URL) scriptNode.src           = s_URL;\n\n    var targ    = document.getElementsByTagName('head')[0] || d.body || d.documentElement;\n    targ.appendChild (scriptNode);\n}\n\naddJS_Node (funkyFunc);\n\n\n/******************************************************************************\n*******************************************************************************\n**  PHASE 3b:\n**  Set up a timer to check for data from our ajax intercept.\n**  Probably best to make it slightly faster than the target's\n**  ajax frequency (about 1 second?).\n*******************************************************************************\n*******************************************************************************\n*/\ntimerHandle = setInterval (function() { SendAnyResultsToServer (); }, 888);\n\nfunction SendAnyResultsToServer ()\n{\n    if (unsafeWindow.payloadArray) {\n        var payload     = unsafeWindow.payloadArray;\n        while (payload.length) {\n            var dataRow = JSON.stringify (payload[0]);\n            payload.shift ();   //--- pop measurement off the bottom of the stack.\n            if (DEBUG)  console.log ('GM script, pre Ajax: ', dataRow);\n\n            /******************************************************************************\n            *******************************************************************************\n            **  PHASE 4: Send the data, one row at a time, to the our server.\n            **  The server would grab the data with:\n            **      $jsonData   = json_decode ($HTTP_RAW_POST_DATA);\n            *******************************************************************************\n            *******************************************************************************\n            */\n            GM_xmlhttpRequest ( {\n                method:     "POST",\n                url:        "http://localhost/db_test/ShowJSON_PostedData.php",\n                data:       dataRow,\n                headers:    {"Content-Type": "application/json"},\n                onload:     function (response) {\n                                if (DEBUG)  console.log (response.responseText);\n                            }\n            } );\n        }\n    }\n}\n\n\n//--- EOF\n
Run Code Online (Sandbox Code Playgroud)\n
\n
\n

其他注意事项:

\n
    \n
  1. 我在该网站的主页上测试了它,没有登录(我不打算在那里创建帐户)。

    \n
  2. \n
  3. 我测试了AdBlockFlashBlockNoSCriptRequestPolicy的全部效果。bidcactus.com的 JS 已打开(必须如此),但其他的则没有。重新打开所有这些内容应该不会导致副作用——但如果确实如此,我不会对其进行调试。

    \n
  4. \n
  5. 必须针对该站点以及您浏览该站点的方式来调整这样的代码1。这取决于你。希望代码有足够的自我记录。

    \n
  6. \n
  7. 享受!

    \n
  8. \n
\n
\n
\n

1主要是:@include@exclude指令,JSON数据选择和过滤,以及iFrame是否需要阻塞。另外,建议在调整完成时DEBUG设置 2 个变量(一个用于 GM 范围,一个用于页面范围) 。false

\n