为什么代码中的"失乐园",在py2.7中运行了几个月,现在np.ndarray()在命名空间中找不到,而[import numpy as np]被执行了？
use*_*197 -1 numpy namespace-organisation python-import python-2.7 anaconda
在2.7.9和2.7.11之间是否有任何命名空间处理变化？
TL; DR:在Stack Overflow上提供450多个答案之后.我确实知道Stack Overflow规则并努力缩小/隔离问题,但出于所有下面提到的原因,实际上不可能发布公平的MCVE.
下面发布的体内诊断证明,np.ndarray调用没有命名空间问题,直到尝试访问/使用numpy(np)符号第14644行(orignal编号中的14630)的第一行
aDSegFLOAT = np.ndarray( ( 80, 7 ), ...
其中一个工作numpy呼叫突然提出异常:
UnboundLocalError:赋值前引用的局部变量'np'
简介:
为了进行定量建模,python在用于技术分析和预测的分布式处理框架中重复使用相当广泛的模块代码.由于一个事实,定量模型(用于预测)是以小数十个CPU核心小时的显着成本生成的,因此部署了一些允许有状态模块的附加措施reload( QuantFX ),以便这些保护已经精心设计的量子模型实例,但允许模块功能在运行中进行更新,调整和重新测试.
事实:
import QuantFX停止工作并抛出numpy未修改的呼叫异常:
- 大约过去12个月使用了模块代码,只更新[QUANT-TOOLS],[MODEL]部分
- 代码是多方分布式处理的Python 2.7部分,因此MCVE的尝试永远不会反映/复制生态系统行为
当问题开始时:为主机
加载最近的Anaconda(miniconda4.0.0/4.0.5)软件包管理器后VM02/wXP,还python重新装入了2.7.11和一个名为的模块QuantFX.py,直到此更新运行时其他Anaconda没有任何问题python中的2.2.0(32位)安装2.7.9停止使用奇怪的异常
什么表明运作不良的运作状态:一周一周的十几个月表现
非常import QuantFX顺利.对QuantFX.aMiniRESPONDER()现在的调用导致异常/产生a Traceback,好像numpy根本没有import-ed(参见第221行)但是假设是某个局部变量.
...
[aMiniRESPONDER]: messaging & signalling sockets RTO:<infrastructure-setup>
START:
Sun Apr 17 20:19:33 2016 ...
[aMiniRESPONDER]: final attempt to de-block a remote-processing failed at aXmitSOCKET.send()
SIG->: Sun Apr 17 20:19:33 2016 PUB-lished a <_|SIG_EXIT|_> signal to peers, will sleep( 3 ) to allow for reception
ZMQ:
Sun Apr 17 20:19:36 2016 Graceful Exit Done. RET(0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "QuantFX.py", line 14630, in aMiniRESPONDER
aDSegFLOAT= np.ndarray( ( 80, 7 ),
UnboundLocalError: local variable 'np' referenced before assignment
(而手动调用非常相同的numpy上解释器命令行提示功能,
尊重namespace到其中的numpy是import由模块-ed(221下面线)
顺利
|>>> QuantFX.np.ndarray( ( 2, 3 ), dtype = QuantFX.np.float32, order = 'F' )
array([[ 1.22325565e+10, -2.81789980e-05, 1.22325565e+10],
[ 1.91215026e+00, -1.81719875e+00, 1.91215026e+00]], dtype=float32)
+0:00:13.282000
22:20:54
|
|>>> QuantFX.np.__version__
'1.10.4'
+0:00:24.438000
22:21:18
|
).
第14630行的Traceback细节和细节:
第221行import numpy as np+ State-Full重新加载的概念:
虽然QuantFX模块编码可能会引起关于PEP*等的反对意见,但语法构造函数的确定是为了通过import/ 允许上下文完整的操作方式/ reload()以及在简约RAM/CPU中快速且安全的无上下文轻量级操作通过选择/复制/粘贴远程终端只需一个组件代码用于诊断或控制CLI目的(如a aRemoteKEYBOARD()或aMiniRESPONDER())但不需要import整个QuantFX模块(配置后期导入,包括所述numpy(第14426行)对于这样的QuantFX-Context-less modus operandi).
寻求什么:
关于2.7.9和2.7.11之间的变化的任何解释,可能与观察到的np.ndarray()碰撞有关,就在上周五引入.
对于替代模块语法构造函数的布局的任何建议也具有很高的价值,如果这样的提议既保留了状态完整性reload( QuantFX )又允许上下文完整和无上下文的代码库组件使用.
更新:
与其可能的外观相反,这篇文章之前进行了适当的审查和调试工作,以增加体内自我诊断,以便隔离问题
""" DEBUG: VM02, after about-a-year working here stable... THROWS EXC. HERE: "UnboundLocalError: local variable 'np' referenced before assignment" ????? .EXC on np.ndarray()
>>> import QuantFX
Is QuantFX_FLAG seen in dir() during <module> import: True
SECTION: ____IDENTIFY____ [TRY].OK
SECTION: ____IDENTIFY____ [FIN]
SECTION: import [TRY].OK
SECTION: import [FIN]
SECTION: FX-MARKET CONTEXTs [TRY].OK
SECTION: FX-MARKET CONTEXTs [FIN]
SECTION: GENERAL TOOLS [TRY].OK
SECTION: GENERAL TOOLS [FIN]
SECTION: DATA INPUTs [TRY].OK
SECTION: DATA INPUTs [FIN]
SECTION: v41 PROCESS [TRY].OK
SECTION: v41 PROCESS [FIN]
SECTION: v41 QUANT TOOLS [TRY].OK
SECTION: v41 QUANT TOOLS [FIN]
QuantFX.py:10966: RuntimeWarning: invalid value encountered in divide
ST2 = np.where( ST2 != 0, ( ST[-LLV.shape[0]:] - LLV ) / ST2, ST2 ) # / ST2 ( == HHV - LLV ) # __main__:4162: RuntimeWarning: invalid value encountered in divide
SECTION: v41 MODEL [TRY].OK
SECTION: v41 MODEL [FIN]
SECTION: v41 PREDICTOR [TRY].OK
SECTION: v41 PREDICTOR [FIN]
SECTION: MetaTrader RESPONDER [TRY].OK
SECTION: MetaTrader RESPONDER [FIN]
SECTION: ____TEMPLATE____ [TRY].OK
SECTION: ____TEMPLATE____ [FIN]
SECTION: ____TEMPLATE____ [TRY].OK
SECTION: ____TEMPLATE____ [FIN]
SECTION: ____TEMPLATE____ [TRY].OK
SECTION: ____TEMPLATE____ [FIN]
+0:00:01.219000
18:20:14
|
|>>> QuantFX.aMiniRESPONDER( aTarget2Bind2_URL = "tcp://10.0.0.62:9999", anInstrumentDictOfPARAMs = QuantFX.anFxCTX[ QuantFX.aCtxID ] )
[aMiniRESPONDER]: runs in it's own full QuantFX context
[aMiniRESPONDER]: imports DONE
[aMiniRESPONDER]: variables DONE
[aMiniRESPONDER]: messaging & signalling sockets RTO:[aZmqCONTEXT] <class 'zmq.sugar.context.Context'> setup with 15.2.0 version
[aMiniRESPONDER]: messaging & signalling sockets [aXmitSOCKET]-pre
[aMiniRESPONDER]: messaging & signalling sockets (aXmitSOCKET)-to be instantiated-via a call to aZmqCONTEXT.socket( zmq.PAIR )
[aMiniRESPONDER]: messaging & signalling sockets [aXmitSOCKET]-post
[aMiniRESPONDER]: messaging & signalling sockets [aXmitSOCKET]-bind()
[aMiniRESPONDER]: messaging & signalling sockets RTO:[aXmitSOCKET]
[aMiniRESPONDER]: messaging & signalling sockets RTO:[aCtrlSOCKET]
[aMiniRESPONDER]: messaging & signalling sockets RTO:[aSIGsSOCKET]
[aMiniRESPONDER]: messaging & signalling sockets RTO:<infrastructure-setup>
START:
Sun Apr 17 20:19:33 2016 ...
[aMiniRESPONDER]: final attempt to de-block a remote-processing failed at aXmitSOCKET.send()
SIG->: Sun Apr 17 20:19:33 2016 PUB-lished a <_|SIG_EXIT|_> signal to peers, will sleep( 3 ) to allow for reception
ZMQ:
Sun Apr 17 20:19:36 2016 Graceful Exit Done. RET(0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "QuantFX.py", line 14630, in aMiniRESPONDER
aDSegFLOAT= np.ndarray( ( 80, 7 ),
UnboundLocalError: local variable 'np' referenced before assignment
体内numpy-nd.array-symbol-checker
def debug_check_np_ndarray( aCallerSideInspectFrameINFO ): # DEBUG-CHECK
#ebug_check_np_ndarray( aCallerSideInspectFrameINFO = getframeinfo( currentframe() ) )
#rom inspect import currentframe, getframeinfo # EXTERNAL RESPONSIBILITY TO import + setup aCallerSideInspectFrameINFO .OnCall
#ef np_check(): # DEF:
try: # TRY: np.*
np.ndarray
print 70*" ", "* NP: np.ndarray call [OK], FILE: ", aCallerSideInspectFrameINFO.filename, " LINE: ", aCallerSideInspectFrameINFO.lineno
except: # EXC: found the issue
print 70*" ", "* NP: np.ndarray call [**], FILE: ", aCallerSideInspectFrameINFO.filename, " LINE: ", aCallerSideInspectFrameINFO.lineno
exc_type, exc_value, exc_traceback = sys.exc_info() # prepare a traceback detail
traceback.print_tb( exc_traceback, limit = 5, file = sys.stdout )
raise
else: # ELSE: ok, passed
#rint("* NP check ok") # optionally add current function name via a traceback module
return
在发布时提供详细的逐行确认,在Line 14644崩溃之前没有np.ndarray符号屏蔽aDSegFLOAT = np.ndarray( ... ):
|>>> QuantFX.aMiniRESPONDER()
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14387
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14391
[aMiniRESPONDER]: runs in it's own full QuantFX context
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14395
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14404
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14410
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14412
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14419
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14423
[aMiniRESPONDER]: runs in a full QuantFX mode, few imports DONE
[aMiniRESPONDER]: imports DONE
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14441
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14443
[aMiniRESPONDER]: variables DONE
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14454
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14527
[aMiniRESPONDER]: messaging & signalling sockets RTO:[aZmqCONTEXT] <class 'zmq.sugar.context.Context'> setup with 15.2.0 version
[aMiniRESPONDER]: messaging & signalling sockets RTO:<infrastructure-setup>
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14626
START:
Tue Apr 19 16:02:12 2016 ...
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14632
* NP: np.ndarray call [OK], FILE: QuantFX.py LINE: 14643
[aMiniRESPONDER].FINALLY: will start attempts to de-block a remote-processing ( after a prior intentional SIG_EXIT or an unhandled EXC )
[aMiniRESPONDER]: final attempt to de-block a remote-processing failed at aXmitSOCKET.send()
[aMiniRESPONDER]: final attempt to propagate a signal to peers failed at aSIGsSOCKET.send()
EXC. ZmqError(ZMQError('No such file or directory')) on aSIGsSOCKET .setsockopt( zmq.LINGER, 0 ) / .close( aSIGsSOCKET )
EXC. ZmqError(ZMQError('No such file or directory')) on aCtrlSOCKET .setsockopt( zmq.LINGER, 0 ) / .close( aCtrlSOCKET )
EXC. ZmqError(ZMQError('No such file or directory')) on aXmitSOCKET .setsockopt( zmq.LINGER, 0 ) / .close( aXmitSOCKET )
ZMQ:
Tue Apr 19 16:02:15 2016 Graceful Exit Done. RET(0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "QuantFX.py", line 14644, in aMiniRESPONDER
aDSegFLOAT = np.ndarray( ( 80, 7 ), # # DSegFLOAT[][] ### ToDo.MS: size-independent ( 80+ ) np.*() :::::::::::::::
UnboundLocalError: local variable 'np' referenced before assignment
+0:00:10.734000
16:02:15
|
从您显示的代码中,python更新不太可能/不可能改变截屏代码中的任何内容.
可能会改变的是环境,或其他未设置的地方的代码aMiniRESPONDER_in_full_QuantFX_CONTEXT.如果设置了变量,import numpy as np则会进行注释,您将获得此处显示的异常.
一些调试思路:要么在调试器中运行,要么print在你认为应该导入numpy的地方使用-debug和break/print - 可能你会发现你只是不进入那些分支.
另一个:一次改变一件事.如果我正确地理解了这个问题,你就会一次性改变:python,包管理器和部署环境,发现这些东西不再以你不理解的方式工作了.回到原始环境并重新测试.然后升级python而不是别的.重新测试.然后升级包管理器(但不是包!).重新测试....直到您确切地发现哪个更改会破坏您的应用.
另一个问题:当某些东西出现故障并且您无法从中恢复时(例如导入失败),请不要继续.在第二个屏幕截图中,您import在try块中有很多语句,它们将忽略错误并继续执行.(L 328-332).如果你继续,稍后会爆炸.如果代码的其余部分看起来像那样 - 是的,你最终会出现无声的失败而不知道它们来自何处.
另一个检查@ hpaulj的建议.非常粗略的方法:创建一个如下函数:
def np_check():
try:
np.ndarray
except:
# found the issue
raise
else:
print("* NP check ok") # optionally add current function name via traceback module
将代码洒在代码周围,找出np名称"消失" 时的第一个位置.在import您希望工作之后立即进行第一次调用,以便在错误发生之前至少看到一个"*NP check ok".然后,您只需要在上次成功的位置和第一次失败之间添加更多检查,直到您隔离导致问题的行.
- @ user3666197添加了另一个想法,仔细检查hpaulj的建议.您可以通过几种方式摆脱该名称 - 我不会仅依赖于扫描源代码. (2认同)
我可以用一个简单的脚本重现您的错误 - 我导入一个模块并使用它一次就好。但是当我第二次使用它时，全局分配被本地分配所掩盖。即使在尝试使用之后发生本地分配，也会发生这种屏蔽。
import collections
def foo1():
dd=collections.defaultdict(list)
return dd
def foo2():
dd=collections.defaultdict(int)
collections = []
return dd
print foo1()
print foo2()
产生
2112:~/mypy$ python stack1.py
defaultdict(<type 'list'>, {})
Traceback (most recent call last):
File "stack1.py", line 13, in <module>
print foo2()
File "stack1.py", line 8, in foo2
dd=collections.defaultdict(int)
UnboundLocalError: local variable 'collections' referenced before assignment
在您的代码或可能您调用的代码中搜索np=...类似的语句。首先查看QuantFX.aMiniRESPONDER，因为这是错误的直接上下文。
将我的测试函数更改为
def foo2():
dd = foo1()
collections = []
return dd
不会产生错误。所以屏蔽的范围相当有限。