Haskell 解析器是否应该允许数字文字中的 Unicode 数字？

Question

作为练习，我正在从头开始为 Haskell 编写解析器。在制作词法分析器时，我注意到Haskell 2010 Report 中的以下规则：

digit ? ascDigit | uniDigit
ascDigit ? 0 | 1 | … | 9
uniDigit ? any Unicode decimal digit
octit ? 0 | 1 | … | 7
hexit ? digit | A | … | F | a | … | f

decimal ? digit{digit}
octal ? octit{octit}
hexadecimal ? hexit{hexit}

integer ? decimal | 0o octal | 0O octal | 0x hexadecimal | 0X hexadecimal
float ? decimal . decimal [exponent] | decimal exponent
exponent ? (e | E) [+ | -] decimal

Decimal and hexadecimal literals, along with float literals, are all based on digit, which admits any Unicode decimal digit, instead of ascDigit, which admits only the basic digits 0-9 from ASCII. Strangely, octal is based on octit, which instead only admits the ASCII digits 0-7. I would guess that these "Unicode decimal digit"s are any Unicode codepoints with the "Nd" General Category. However, this includes characters such as the Full-Width digits ?-? and the Devanagari numerals ?-?. I can see why it might be desirable to allow these in identifiers, but I can see no benefit whatsoever for allowing one to write ?? for the literal 90.

GHC seems to agree with me. When I try to compile this file,

module DigitTest where
x1 = ?

它吐出这个错误。

digitTest1.hs:2:6: error: lexical error at character '\65297'
  |
2 | x1 = ?
  |      ^

然而，这个文件

module DigitTest where
x? = 1

编译就好了。我是否错误地阅读了语言规范？GHC 的（明智的）行为实际上是正确的，还是在技术上违反了报告中的规范？我在任何地方都找不到提及此事。

Answer 1

在GHC源代码文件中compiler/parser/Lexer.x，可以找到如下代码：

ascdigit  = 0-9
$unidigit  = \x03 -- Trick Alex into handling Unicode. See [Unicode in Alex].
$decdigit  = $ascdigit -- for now, should really be $digit (ToDo)
$digit     = [$ascdigit $unidigit]
...
$binit     = 0-1
$octit     = 0-7
$hexit     = [$decdigit A-F a-f]
...
@numspc       = _*                   -- numeric spacer (#14473)
@decimal      = $decdigit(@numspc $decdigit)*
@binary       = $binit(@numspc $binit)*
@octal        = $octit(@numspc $octit)*
@hexadecimal  = $hexit(@numspc $hexit)*
@exponent     = @numspc [eE] [\-\+]? @decimal
@bin_exponent = @numspc [pP] [\-\+]? @decimal

此处，$decdigit用于解析十进制和十六进制文字（及其浮点变体），而$digit用于字母数字标识符的“数字”部分。“ToDo”注释明确指出这是 GHC 与语言标准的公认偏差。

因此，您正确阅读了规范，而 GHC 是半故意违反规范的。有一张公开票建议至少记录偏差，但我认为没有人表示有兴趣修复它。