Add NewLazyLexer and MustNewLazyLexer which accept a function that
returns the rules for the lexer. This allows us to defer the rules
definitions until they're needed.
Lexers in a, g, s, and x packages have been updated to use the new lazy
lexer.
Using these examples:
~~~
0x21
1_000
1e3
~~~
Several languages failed that support these syntax. Here are ones I found:
Lexer | 0x21 | 1_000 | 1e3
-----------|------|-------|-----
C# | fail | fail | fail
Go | | fail |
JavaScript | | fail | fail
PHP | | fail |
Python | | fail |
Ruby | | | fail
I fixed these issues, and added tests.
This was done to speed up incremental compilation when working on
lexers. That is, modifying a single lexer will no longer require
recompiling all lexers.
This is a (slightly) backwards breaking change in that lexers are no
longer exported directly in the lexers package. The registry API is
"aliased" at the old location.
