6 Commits

Author	SHA1	Message	Date
Steven Penny	7a68f3e25d	LiteralNumberHex: underscore support ... Most languages allow for underscore in number literals. Fix support for a few languages. References: - https://docs.python.org/reference/lexical_analysis.html#integer-literals - https://golang.org/ref/spec#Integer_literals - https://php.net/language.types.integer	2021-03-03 12:57:43 +11:00
Cameron Moore	59126c5b32	Add NewLazyLexer to defer rules definitions and reduce init costs (#449) ... Add NewLazyLexer and MustNewLazyLexer which accept a function that returns the rules for the lexer. This allows us to defer the rules definitions until they're needed. Lexers in a, g, s, and x packages have been updated to use the new lazy lexer.	2021-02-08 12:16:49 +11:00
Steven Penny	881d54f096	Improve number literals for several languages ... Using these examples: ~~~ 0x21 1_000 1e3 ~~~ Several languages failed that support these syntax. Here are ones I found: Lexer | 0x21 | 1_000 | 1e3 -----------|------|-------|----- C# | fail | fail | fail Go | | fail | JavaScript | | fail | fail PHP | | fail | Python | | fail | Ruby | | | fail I fixed these issues, and added tests.	2020-11-22 08:30:29 +11:00
Alec Thomas	2b9ea60d89	Split PHP into two lexers - PHP and PHTML. ... The former is pure PHP code while the latter is PHP code in <? ?> tags, within HTML. Fixes #210.	2020-06-30 21:00:09 +10:00
Alec Thomas	5ef2d37c8c	Analysre for PHP.	2018-03-19 00:47:15 +11:00
Alec Thomas	db6920e68f	Implemented delegating lexer. ... This is a lexer that is useful for templating languages, where the surrounding text may be of a different syntax. eg. PHP+HTML The PHP lexer has been changed accordingly. Fixes #80	2018-03-18 20:38:29 +11:00