ColdFusion的cfquery默默地失败了

Question

我有一个检索大量数据的查询.

<cfsetting requesttimeout="9999999" >

<cfquery name="randomething" datasource="ds" timeout="9999999" >
    SELECT
        col1,
        col2
    FROM
        table
</cfquery>

<cfdump var="#randomething.recordCount#" /> <!---should be about 5 million rows --->

我可以使用python的cx_Oracle成功检索数据,并sys.getsizeof在python列表上使用返回22621060,所以大约21兆字节.

ColdFusion不会在页面上返回错误,我在任何日志中都找不到任何内容.为什么cfdump不显示行数？

附加信息

这样做的原因是因为我有大约8000个较小的查询来运行randomthing查询.换句话说,当我对数据库运行这8000个查询时,该过程需要数小时才能完成.我怀疑这是因为我与其他几个数据库用户竞争,数据库陷入困境.

8000个较小的查询在col2期间获得col1的计数.

SELECT 
    count(col1) as count
WHERE 
    col2 < 20121109 
AND 
    col2 > 20121108

根据亚当卡梅隆的 建议.

• cflog建议查询没有完成.

• 我尝试在代码和 CFIDE /管理员中更改查询超时,显然CF9不再考虑超时属性,无论我尝试了什么我都无法使查询超时.

我也开始玩这个maxrows属性,看看我是否可以通过这种方式识别任何信息.

• 当maxrows设置为1300000时,一切正常
• 当maxrows为1400000或更高时,我收到此错误
• 当maxrows是2000000时,我会观察到我原来的问题

更新

所以这不是cfquery的限制.通过使用QueryNew然后循环它来添加数据,我可以很好地超过200万标记没有任何问题.

我还使用此问题中的信息创建了一个ThinClient数据源,我没有观察到行为的任何变化.

数据库端的消息是

来自客户端的SQL*Net消息

和

SQL*Net向客户端提供更多数据

我刚刚发现通过使用瘦客户端,blockfactor1="100"我可以检索更多行(appx.3000000).

Answer 1

数据库端是否有记录任何内容？

I wonder if the timeout is not being respected, and JDBC is "hanging up" on the DB whilst it's working. That's a wild guess. What if you set a very low timeout - eg: 5sec - does it error after 5sec, or what?

The browser could be timing out too. What say you write something to a log before and after the <cfquery> block, with <cflog>. To see if the query is eventually finishing.

I have to wonder what it is you intend to do with these 22M records once you get them back to CF. Whatever it is, it sounds to me like CF is the wrong place to be doing whatever it is: CF ain't for heavy data processing, it's for making web pages. If you need to process 22M records, I suspect you should be doing it on the database. That said, I'm second-guessing what you're doing with no info to go on, so I presume there's probably a good reason to be doing it.