不同的语法错误在输出中隐藏行

Question

我有一个调用的脚本compile。

try:
    code = compile('3 = 3', 'test', 'exec')
except Exception as e:
    sys.stderr.write(''.join(traceback.format_exception_only(type(e), e)))

3 = 3 结果是：

File "test", line 1
SyntaxError: can't assign to literal

而3 = 3a实际上打印行

File "test", line 1
    3 = 3a
        ^
SyntaxError: invalid syntax

知道为什么吗？

Answer 1

Python SyntaxError在两个地方产生异常：

1. 解析时，由Python语法驱动
2. 从解析结果创建抽象语法树（AST）时；AST驱动编译器。

这是因为Python语法有一些特殊情况，在这种情况下，保持语法简单是容易的，但在解析完成后再进行其他检查，并在完成其他语法检查的地方构建AST。

分配是其中之一，因为=符号左侧允许的规则与右侧允许的规则不同，但仍然紧密相关。左侧是目标端，目标的结构可以像列表或元组（拆包分配）一样，您可以分配给属性或索引操作（listobj[1] = ...，等等）。但是要让解析器检测到目标实际上是文字而不是变量名或属性等，将需要非常不同的解析器结构，因此将其留给AST。

因此，您的3 = 3错误通过了解析阶段，但随后在以后的AST“分配目标”检查阶段失败了，而3 = 3a在解析器阶段陷入了困境（在此处3a很容易发现错误）。

To give you a good syntax error, exceptions raised by the parser contain the source code line in the exception:

>>> try:
...     code = compile('3 = 3a', 'test', 'exec')
... except Exception as e:
...     print(repr(e))
...
SyntaxError('invalid syntax', ('test', 1, 6, '3 = 3a\n'))

Note the ('test', 1, 6, '3 = 3a\n') tuple in the exception; these are available via the SyntaxError attributes filename, lineno (the line number), offset (the column offset) and text for the source code line itself. For the parser, this is easily provided, as it has access to the source code.

But the AST doesn't have the source code. It only has the filename, line number, column, and parse tree objects. It does not have the original source text. It would normally try to read that from the filename, but test is not actually a file. So the line is empty:

>>> try:
...     code = compile('3 = 3', 'test', 'exec')
... except Exception as e:
...     print(repr(e))
...
SyntaxError('cannot assign to literal', ('test', 1, 1, ''))

You can test for this and fix it by replacing the SyntaxError exception with a new one with the empty string replaced with your source text:

>>> source = '3 = 3'
>>> try:
...     code = compile(source, 'test', 'exec')
... except Exception as e:
...     if isinstance(e, SyntaxError) and not e.text:
...         sline = source.splitlines(True)[e.lineno - 1]
...         e = SyntaxError(e.msg, (e.filename, e.lineno, e.offset, sline))
...     sys.stderr.write(''.join(traceback.format_exception_only(type(e), e)))
...
  File "test", line 1
    3 = 3
    ^
SyntaxError: cannot assign to literal

Note that for a multi-line source string you’d want to split that source into lines and use the .lineno attribute to select the indicated source line.

The alternative is to write the source code to a temporary filename, and pass that filename to compile() so that when a SyntaxError exception is found when building the AST, Python can then open that temporary file and find the corresponding text line.

Note that when you use the special filename '<string>', that no attempt is made to find the source code for a line and e.text is set to None:

>>> try:
...     code = compile('3 = 3', '<string>', 'exec')
... except Exception as e:
...     print(repr(e))
...
SyntaxError('cannot assign to literal', ('<string>', 1, 1, None))

and when the .text attribute is set to None, the traceback module forgoes printing the line-and-marker section.

If you are interested in exactly why the Python grammar parser won't detect literals in the assignment target, you might be interested in the work Guido van Rossum is doing in writing a different parser for Python, which includes exposition on why the current parser works the way it does and how an alternative parser model can avoid these issues.