Version 2 of Chroma

This cleans up the API in general, removing a bunch of deprecated stuff,
cleaning up circular imports, etc.

But the biggest change is switching to an optional XML format for the
regex lexer.

Having lexers defined only in Go is not ideal for a couple of reasons.
Firstly, it impedes a significant portion of contributors who use Chroma
in Hugo, but don't know Go. Secondly, it bloats the binary size of any
project that imports Chroma.

Why XML? YAML is an abomination and JSON is not human editable. XML
also compresses very well (eg. Go template lexer XML compresses from
3239 bytes to 718).

Why a new syntax format? All major existing formats rely on the
Oniguruma regex engine, which is extremely complex and for which there
is no Go port.

Why not earlier? Prior to the existence of fs.FS this was not a viable
option.

Benchmarks:

    $ hyperfine --warmup 3 \
        './chroma.master --version' \
        './chroma.xml-pre-opt --version' \
        './chroma.xml --version'
    Benchmark 1: ./chroma.master --version
      Time (mean ± σ):       5.3 ms ±   0.5 ms    [User: 3.6 ms, System: 1.4 ms]
      Range (min … max):     4.2 ms …   6.6 ms    233 runs

    Benchmark 2: ./chroma.xml-pre-opt --version
      Time (mean ± σ):      50.6 ms ±   0.5 ms    [User: 52.4 ms, System: 3.6 ms]
      Range (min … max):    49.2 ms …  51.5 ms    51 runs

    Benchmark 3: ./chroma.xml --version
      Time (mean ± σ):       6.9 ms ±   1.1 ms    [User: 5.1 ms, System: 1.5 ms]
      Range (min … max):     5.7 ms …  19.9 ms    196 runs

    Summary
      './chroma.master --version' ran
        1.30 ± 0.23 times faster than './chroma.xml --version'
        9.56 ± 0.83 times faster than './chroma.xml-pre-opt --version'

A slight increase in init time, but I think this is okay given the
increase in flexibility.

And binary size difference:

    $ du -h lexers.test*
    $ du -sh chroma*                                                                                                                                                                                                                                                                                                                                                                                                                                                             951371ms
    8.8M	chroma.master
    7.8M	chroma.xml
    7.8M	chroma.xml-pre-opt

Benchmarks:

    $ hyperfine --warmup 3 \
        './chroma.master --version' \
        './chroma.xml-pre-opt --version' \
        './chroma.xml --version'
    Benchmark 1: ./chroma.master --version
      Time (mean ± σ):       5.3 ms ±   0.5 ms    [User: 3.6 ms, System: 1.4 ms]
      Range (min … max):     4.2 ms …   6.6 ms    233 runs

    Benchmark 2: ./chroma.xml-pre-opt --version
      Time (mean ± σ):      50.6 ms ±   0.5 ms    [User: 52.4 ms, System: 3.6 ms]
      Range (min … max):    49.2 ms …  51.5 ms    51 runs

    Benchmark 3: ./chroma.xml --version
      Time (mean ± σ):       6.9 ms ±   1.1 ms    [User: 5.1 ms, System: 1.5 ms]
      Range (min … max):     5.7 ms …  19.9 ms    196 runs

    Summary
      './chroma.master --version' ran
        1.30 ± 0.23 times faster than './chroma.xml --version'
        9.56 ± 0.83 times faster than './chroma.xml-pre-opt --version'

Incompatible changes:

- (*RegexLexer).SetAnalyser: changed from func(func(text string) float32) *RegexLexer to func(func(text string) float32) Lexer
- (*TokenType).UnmarshalJSON: removed
- Lexer.AnalyseText: added
- Lexer.SetAnalyser: added
- Lexer.SetRegistry: added
- MustNewLazyLexer: removed
- MustNewLexer: changed from func(*Config, Rules) *RegexLexer to func(*Config, func() Rules) *RegexLexer
- Mutators: changed from func(...Mutator) MutatorFunc to func(...Mutator) Mutator
- NewLazyLexer: removed
- NewLexer: changed from func(*Config, Rules) (*RegexLexer, error) to func(*Config, func() Rules) (*RegexLexer, error)
- Pop: changed from func(int) MutatorFunc to func(int) Mutator
- Push: changed from func(...string) MutatorFunc to func(...string) Mutator
- TokenType.MarshalJSON: removed
- Using: changed from func(Lexer) Emitter to func(string) Emitter
- UsingByGroup: changed from func(func(string) Lexer, int, int, ...Emitter) Emitter to func(int, int, ...Emitter) Emitter

.github/workflows/release.yml

@@ -12,6 +12,7 @@ jobs:
       with:
         fetch-depth: 0
     - run: ./bin/hermit env --raw >> $GITHUB_ENV
+    - run: find ./lexers -name '*.xml' | xargs gzip -9
     - run: goreleaser release --rm-dist
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.golangci.yml

@@ -36,6 +36,7 @@ linters:
     - ifshort
     - wrapcheck
     - stylecheck
+    - thelper
 
 linters-settings:
   govet:
@@ -48,8 +49,8 @@ linters-settings:
     min-len: 8
     min-occurrences: 3
   forbidigo:
-    forbid:
+    #forbid:
-      - (Must)?NewLexer
+    #  - (Must)?NewLexer$
     exclude_godoc_examples: false
 
 
@@ -74,3 +75,4 @@ issues:
     - 'methods on the same type should have the same receiver name'
     - '_TokenType_name should be _TokenTypeName'
     - '`_TokenType_map` should be `_TokenTypeMap`'
+    - 'rewrite if-else to switch statement'

_tools/css2style/main.go

@@ -10,14 +10,14 @@ import (
 	"github.com/aymerick/douceur/parser"
 	"gopkg.in/alecthomas/kingpin.v3-unstable"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
 )
 
 const (
 	outputTemplate = `package styles
 
 import (
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
 )
 
 // {{.Name}} style.

_tools/exercise/main.go

@@ -5,9 +5,9 @@ import (
 	"io/ioutil"
 	"os"
 
-	"github.com/alecthomas/chroma/formatters"
+	"github.com/alecthomas/chroma/v2/formatters"
-	"github.com/alecthomas/chroma/lexers"
+	"github.com/alecthomas/chroma/v2/lexers"
-	"github.com/alecthomas/chroma/styles"
+	"github.com/alecthomas/chroma/v2/styles"
 	"gopkg.in/alecthomas/kingpin.v3-unstable"
 )
 

_tools/pygments2chroma.py

@@ -15,8 +15,8 @@ TEMPLATE = r'''
 package {{package}}
 
 import (
-	. "github.com/alecthomas/chroma" // nolint
+	. "github.com/alecthomas/chroma/v2" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
+	"github.com/alecthomas/chroma/v2/lexers/internal"
 )
 
 // {{upper_name}} lexer.

_tools/style.py

@@ -10,7 +10,7 @@ TEMPLATE = r'''
 package styles
 
 import (
-    "github.com/alecthomas/chroma"
+    "github.com/alecthomas/chroma/v2"
 )
 
 // {{upper_name}} style.

bin/.hyperfine-1.12.0.pkg

@@ -0,0 +1 @@
+hermit

bin/hyperfine

@@ -0,0 +1 @@
+.hyperfine-1.12.0.pkg

cmd/chroma/go.mod

@@ -1,12 +1,12 @@
-module github.com/alecthomas/chroma/cmd/chroma
+module github.com/alecthomas/chroma/v2/cmd/chroma
 
-go 1.16
+go 1.17
 
-replace github.com/alecthomas/chroma => ../../
+replace github.com/alecthomas/chroma/v2 => ../../
 
 require (
-	github.com/alecthomas/chroma v0.0.0-00010101000000-000000000000
+	github.com/alecthomas/chroma/v2 v2.0.0-00010101000000-000000000000
 	github.com/alecthomas/kong v0.2.17
-	github.com/mattn/go-colorable v0.1.10
+	github.com/mattn/go-colorable v0.1.12
 	github.com/mattn/go-isatty v0.0.14
 )

cmd/chroma/go.sum

@@ -1,15 +1,14 @@
 github.com/alecthomas/kong v0.2.17 h1:URDISCI96MIgcIlQyoCAlhOmrSw6pZScBNkctg8r0W0=
 github.com/alecthomas/kong v0.2.17/go.mod h1:ka3VZ8GZNPXv9Ov+j4YNLkI8mTuhXyr/0ktSlqIydQQ=
-github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964 h1:y5HC9v93H5EPKqaS1UYVg1uYah5Xf51mBfIoWehClUQ=
+github.com/alecthomas/repr v0.0.0-20220113201626-b1b626ac65ae h1:zzGwJfFlFGD94CyyYwCJeSuD32Gj9GTaSi5y9hoVzdY=
-github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964/go.mod h1:Xd9hchkHSWYkEqJwUGisez3G1QY8Ryz0sdWrLPMGjLk=
+github.com/alecthomas/repr v0.0.0-20220113201626-b1b626ac65ae/go.mod h1:2kn6fqh/zIyPLmm3ugklbEi5hg5wS435eygvNfaDQL8=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/dlclark/regexp2 v1.4.0 h1:F1rxgk7p4uKjwIQxBs9oAXe5CqrXlCduYEJvrF4u93E=
 github.com/dlclark/regexp2 v1.4.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
-github.com/mattn/go-colorable v0.1.10 h1:KWqbp83oZ6YOEgIbNW3BM1Jbe2tz4jgmWA9FOuAF8bw=
+github.com/mattn/go-colorable v0.1.12 h1:jF+Du6AlPIjs2BiUiQlKOX0rt3SujHxPnksPKZbaA40=
-github.com/mattn/go-colorable v0.1.10/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
+github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
-github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
 github.com/mattn/go-isatty v0.0.14 h1:yVuAays6BHfxijgZPzw+3Zlu5yQgKGP2/hcQbHb7S9Y=
 github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
 github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
@@ -19,10 +18,9 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
-golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c h1:F1jZWGFhYfh0Ci55sIpILtKKK8p3i2/krTr0H1rg74I=
 golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6 h1:foEbQz/B0Oz6YIqu/69kfXPYeFQAuuMYFkjaqXzl5Wo=
+golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=

cmd/chroma/list.before

@@ -0,0 +1,810 @@
+lexers:
+  ABAP
+    aliases: abap
+    filenames: *.abap *.ABAP
+    mimetypes: text/x-abap
+  ABNF
+    aliases: abnf
+    filenames: *.abnf
+    mimetypes: text/x-abnf
+  ActionScript
+    aliases: as actionscript
+    filenames: *.as
+    mimetypes: application/x-actionscript text/x-actionscript text/actionscript
+  ActionScript 3
+    aliases: as3 actionscript3
+    filenames: *.as
+    mimetypes: application/x-actionscript3 text/x-actionscript3 text/actionscript3
+  Ada
+    aliases: ada ada95 ada2005
+    filenames: *.adb *.ads *.ada
+    mimetypes: text/x-ada
+  AL
+    aliases: al
+    filenames: *.al *.dal
+    mimetypes: text/x-al
+  Angular2
+    aliases: ng2
+  ANTLR
+    aliases: antlr
+  ApacheConf
+    aliases: apacheconf aconf apache
+    filenames: .htaccess apache.conf apache2.conf
+    mimetypes: text/x-apacheconf
+  APL
+    aliases: apl
+    filenames: *.apl
+  AppleScript
+    aliases: applescript
+    filenames: *.applescript
+  Arduino
+    aliases: arduino
+    filenames: *.ino
+    mimetypes: text/x-arduino
+  ArmAsm
+    aliases: armasm
+    filenames: *.s *.S
+    mimetypes: text/x-armasm text/x-asm
+  Awk
+    aliases: awk gawk mawk nawk
+    filenames: *.awk
+    mimetypes: application/x-awk
+  Ballerina
+    aliases: ballerina
+    filenames: *.bal
+    mimetypes: text/x-ballerina
+  Base Makefile
+    aliases: make makefile mf bsdmake
+    filenames: *.mak *.mk Makefile makefile Makefile.* GNUmakefile
+    mimetypes: text/x-makefile
+  Bash
+    aliases: bash sh ksh zsh shell
+    filenames: *.sh *.ksh *.bash *.ebuild *.eclass .env *.env *.exheres-0 *.exlib *.zsh *.zshrc .bashrc bashrc .bash_* bash_* zshrc .zshrc PKGBUILD
+    mimetypes: application/x-sh application/x-shellscript
+  BashSession
+    aliases: bash-session console shell-session
+    filenames: .sh-session
+    mimetypes: text/x-sh
+  Batchfile
+    aliases: bat batch dosbatch winbatch
+    filenames: *.bat *.cmd
+    mimetypes: application/x-dos-batch
+  BibTeX
+    aliases: bib bibtex
+    filenames: *.bib
+    mimetypes: text/x-bibtex
+  Bicep
+    aliases: bicep
+    filenames: *.bicep
+  BlitzBasic
+    aliases: blitzbasic b3d bplus
+    filenames: *.bb *.decls
+    mimetypes: text/x-bb
+  BNF
+    aliases: bnf
+    filenames: *.bnf
+    mimetypes: text/x-bnf
+  Brainfuck
+    aliases: brainfuck bf
+    filenames: *.bf *.b
+    mimetypes: application/x-brainfuck
+  C
+    aliases: c
+    filenames: *.c *.h *.idc *.x[bp]m
+    mimetypes: text/x-chdr text/x-csrc image/x-xbitmap image/x-xpixmap
+  C#
+    aliases: csharp c#
+    filenames: *.cs
+    mimetypes: text/x-csharp
+  C++
+    aliases: cpp c++
+    filenames: *.cpp *.hpp *.c++ *.h++ *.cc *.hh *.cxx *.hxx *.C *.H *.cp *.CPP
+    mimetypes: text/x-c++hdr text/x-c++src
+  Caddyfile
+    aliases: caddyfile caddy
+    filenames: Caddyfile*
+  Caddyfile Directives
+    aliases: caddyfile-directives caddyfile-d caddy-d
+  Cap'n Proto
+    aliases: capnp
+    filenames: *.capnp
+  Cassandra CQL
+    aliases: cassandra cql
+    filenames: *.cql
+    mimetypes: text/x-cql
+  Ceylon
+    aliases: ceylon
+    filenames: *.ceylon
+    mimetypes: text/x-ceylon
+  CFEngine3
+    aliases: cfengine3 cf3
+    filenames: *.cf
+  cfstatement
+    aliases: cfs
+  ChaiScript
+    aliases: chai chaiscript
+    filenames: *.chai
+    mimetypes: text/x-chaiscript application/x-chaiscript
+  Cheetah
+    aliases: cheetah spitfire
+    filenames: *.tmpl *.spt
+    mimetypes: application/x-cheetah application/x-spitfire
+  Clojure
+    aliases: clojure clj
+    filenames: *.clj
+    mimetypes: text/x-clojure application/x-clojure
+  CMake
+    aliases: cmake
+    filenames: *.cmake CMakeLists.txt
+    mimetypes: text/x-cmake
+  COBOL
+    aliases: cobol
+    filenames: *.cob *.COB *.cpy *.CPY
+    mimetypes: text/x-cobol
+  CoffeeScript
+    aliases: coffee-script coffeescript coffee
+    filenames: *.coffee
+    mimetypes: text/coffeescript
+  Common Lisp
+    aliases: common-lisp cl lisp
+    filenames: *.cl *.lisp
+    mimetypes: text/x-common-lisp
+  Common Lisp
+    aliases: common-lisp cl lisp
+    filenames: *.cl *.lisp
+    mimetypes: text/x-common-lisp
+  Coq
+    aliases: coq
+    filenames: *.v
+    mimetypes: text/x-coq
+  Crystal
+    aliases: cr crystal
+    filenames: *.cr
+    mimetypes: text/x-crystal
+  CSS
+    aliases: css
+    filenames: *.css
+    mimetypes: text/css
+  Cython
+    aliases: cython pyx pyrex
+    filenames: *.pyx *.pxd *.pxi
+    mimetypes: text/x-cython application/x-cython
+  D
+    aliases: d
+    filenames: *.d *.di
+    mimetypes: text/x-d
+  Dart
+    aliases: dart
+    filenames: *.dart
+    mimetypes: text/x-dart
+  Diff
+    aliases: diff udiff
+    filenames: *.diff *.patch
+    mimetypes: text/x-diff text/x-patch
+  Django/Jinja
+    aliases: django jinja
+    mimetypes: application/x-django-templating application/x-jinja
+  Docker
+    aliases: docker dockerfile
+    filenames: Dockerfile *.docker
+    mimetypes: text/x-dockerfile-config
+  DTD
+    aliases: dtd
+    filenames: *.dtd
+    mimetypes: application/xml-dtd
+  Dylan
+    aliases: dylan
+    filenames: *.dylan *.dyl *.intr
+    mimetypes: text/x-dylan
+  EBNF
+    aliases: ebnf
+    filenames: *.ebnf
+    mimetypes: text/x-ebnf
+  Elixir
+    aliases: elixir ex exs
+    filenames: *.ex *.exs
+    mimetypes: text/x-elixir
+  Elm
+    aliases: elm
+    filenames: *.elm
+    mimetypes: text/x-elm
+  EmacsLisp
+    aliases: emacs elisp emacs-lisp
+    filenames: *.el
+    mimetypes: text/x-elisp application/x-elisp
+  EmacsLisp
+    aliases: emacs elisp emacs-lisp
+    filenames: *.el
+    mimetypes: text/x-elisp application/x-elisp
+  Erlang
+    aliases: erlang
+    filenames: *.erl *.hrl *.es *.escript
+    mimetypes: text/x-erlang
+  Factor
+    aliases: factor
+    filenames: *.factor
+    mimetypes: text/x-factor
+  Fennel
+    aliases: fennel fnl
+    filenames: *.fennel
+    mimetypes: text/x-fennel application/x-fennel
+  Fish
+    aliases: fish fishshell
+    filenames: *.fish *.load
+    mimetypes: application/x-fish
+  Forth
+    aliases: forth
+    filenames: *.frt *.fth *.fs
+    mimetypes: application/x-forth
+  Fortran
+    aliases: fortran
+    filenames: *.f03 *.f90 *.F03 *.F90
+    mimetypes: text/x-fortran
+  FortranFixed
+    aliases: fortranfixed
+    filenames: *.f *.F
+    mimetypes: text/x-fortran
+  FSharp
+    aliases: fsharp
+    filenames: *.fs *.fsi
+    mimetypes: text/x-fsharp
+  GAS
+    aliases: gas asm
+    filenames: *.s *.S
+    mimetypes: text/x-gas
+  GDScript
+    aliases: gdscript gd
+    filenames: *.gd
+    mimetypes: text/x-gdscript application/x-gdscript
+  Genshi
+    aliases: genshi kid xml+genshi xml+kid
+    filenames: *.kid
+    mimetypes: application/x-genshi application/x-kid
+  Genshi HTML
+    aliases: html+genshi html+kid
+    mimetypes: text/html+genshi
+  Genshi Text
+    aliases: genshitext
+    mimetypes: application/x-genshi-text text/x-genshi
+  Gherkin
+    aliases: cucumber Cucumber gherkin Gherkin
+    filenames: *.feature *.FEATURE
+    mimetypes: text/x-gherkin
+  GLSL
+    aliases: glsl
+    filenames: *.vert *.frag *.geo
+    mimetypes: text/x-glslsrc
+  Gnuplot
+    aliases: gnuplot
+    filenames: *.plot *.plt
+    mimetypes: text/x-gnuplot
+  Go
+    aliases: go golang
+    filenames: *.go
+    mimetypes: text/x-gosrc
+  Go HTML Template
+    aliases: go-html-template
+  Go HTML Template
+    aliases: go-html-template
+  Go Text Template
+    aliases: go-text-template
+  GraphQL
+    aliases: graphql graphqls gql
+    filenames: *.graphql *.graphqls
+  Groff
+    aliases: groff nroff man
+    filenames: *.[1-9] *.1p *.3pm *.man
+    mimetypes: application/x-troff text/troff
+  Groovy
+    aliases: groovy
+    filenames: *.groovy *.gradle
+    mimetypes: text/x-groovy
+  Handlebars
+    aliases: handlebars hbs
+    filenames: *.handlebars *.hbs
+  Haskell
+    aliases: haskell hs
+    filenames: *.hs
+    mimetypes: text/x-haskell
+  Haxe
+    aliases: hx haxe hxsl
+    filenames: *.hx *.hxsl
+    mimetypes: text/haxe text/x-haxe text/x-hx
+  HCL
+    aliases: hcl
+    filenames: *.hcl
+    mimetypes: application/x-hcl
+  Hexdump
+    aliases: hexdump
+  HLB
+    aliases: hlb
+    filenames: *.hlb
+  HTML
+    aliases: html
+    filenames: *.html *.htm *.xhtml *.xslt
+    mimetypes: text/html application/xhtml+xml
+  HTTP
+    aliases: http
+  Hy
+    aliases: hylang
+    filenames: *.hy
+    mimetypes: text/x-hy application/x-hy
+  Idris
+    aliases: idris idr
+    filenames: *.idr
+    mimetypes: text/x-idris
+  Igor
+    aliases: igor igorpro
+    filenames: *.ipf
+    mimetypes: text/ipf
+  INI
+    aliases: ini cfg dosini
+    filenames: *.ini *.cfg *.inf .gitconfig .editorconfig
+    mimetypes: text/x-ini text/inf
+  Io
+    aliases: io
+    filenames: *.io
+    mimetypes: text/x-iosrc
+  J
+    aliases: j
+    filenames: *.ijs
+    mimetypes: text/x-j
+  Java
+    aliases: java
+    filenames: *.java
+    mimetypes: text/x-java
+  JavaScript
+    aliases: js javascript
+    filenames: *.js *.jsm *.mjs
+    mimetypes: application/javascript application/x-javascript text/x-javascript text/javascript
+  JSON
+    aliases: json
+    filenames: *.json
+    mimetypes: application/json
+  Julia
+    aliases: julia jl
+    filenames: *.jl
+    mimetypes: text/x-julia application/x-julia
+  Jungle
+    aliases: jungle
+    filenames: *.jungle
+    mimetypes: text/x-jungle
+  Kotlin
+    aliases: kotlin
+    filenames: *.kt
+    mimetypes: text/x-kotlin
+  Lighttpd configuration file
+    aliases: lighty lighttpd
+    mimetypes: text/x-lighttpd-conf
+  LLVM
+    aliases: llvm
+    filenames: *.ll
+    mimetypes: text/x-llvm
+  Lua
+    aliases: lua
+    filenames: *.lua *.wlua
+    mimetypes: text/x-lua application/x-lua
+  Mako
+    aliases: mako
+    filenames: *.mao
+    mimetypes: application/x-mako
+  markdown
+    aliases: md mkd
+    filenames: *.md *.mkd *.markdown
+    mimetypes: text/x-markdown
+  Mason
+    aliases: mason
+    filenames: *.m *.mhtml *.mc *.mi autohandler dhandler
+    mimetypes: application/x-mason
+  Mathematica
+    aliases: mathematica mma nb
+    filenames: *.nb *.cdf *.nbp *.ma
+    mimetypes: application/mathematica application/vnd.wolfram.mathematica application/vnd.wolfram.mathematica.package application/vnd.wolfram.cdf
+  Matlab
+    aliases: matlab
+    filenames: *.m
+    mimetypes: text/matlab
+  mcfunction
+    aliases: mcfunction
+    filenames: *.mcfunction
+  Meson
+    aliases: meson meson.build
+    filenames: meson.build meson_options.txt
+    mimetypes: text/x-meson
+  Metal
+    aliases: metal
+    filenames: *.metal
+    mimetypes: text/x-metal
+  MiniZinc
+    aliases: minizinc MZN mzn
+    filenames: *.mzn *.dzn *.fzn
+    mimetypes: text/minizinc
+  MLIR
+    aliases: mlir
+    filenames: *.mlir
+    mimetypes: text/x-mlir
+  Modula-2
+    aliases: modula2 m2
+    filenames: *.def *.mod
+    mimetypes: text/x-modula2
+  MonkeyC
+    aliases: monkeyc
+    filenames: *.mc
+    mimetypes: text/x-monkeyc
+  MorrowindScript
+    aliases: morrowind mwscript
+  Myghty
+    aliases: myghty
+    filenames: *.myt autodelegate
+    mimetypes: application/x-myghty
+  MySQL
+    aliases: mysql mariadb
+    filenames: *.sql
+    mimetypes: text/x-mysql text/x-mariadb
+  NASM
+    aliases: nasm
+    filenames: *.asm *.ASM
+    mimetypes: text/x-nasm
+  Newspeak
+    aliases: newspeak
+    filenames: *.ns2
+    mimetypes: text/x-newspeak
+  Nginx configuration file
+    aliases: nginx
+    filenames: nginx.conf
+    mimetypes: text/x-nginx-conf
+  Nim
+    aliases: nim nimrod
+    filenames: *.nim *.nimrod
+    mimetypes: text/x-nim
+  Nix
+    aliases: nixos nix
+    filenames: *.nix
+    mimetypes: text/x-nix
+  Objective-C
+    aliases: objective-c objectivec obj-c objc
+    filenames: *.m *.h
+    mimetypes: text/x-objective-c
+  OCaml
+    aliases: ocaml
+    filenames: *.ml *.mli *.mll *.mly
+    mimetypes: text/x-ocaml
+  Octave
+    aliases: octave
+    filenames: *.m
+    mimetypes: text/octave
+  OnesEnterprise
+    aliases: ones onesenterprise 1S 1S:Enterprise
+    filenames: *.EPF *.epf *.ERF *.erf
+    mimetypes: application/octet-stream
+  OpenEdge ABL
+    aliases: openedge abl progress openedgeabl
+    filenames: *.p *.cls *.w *.i
+    mimetypes: text/x-openedge application/x-openedge
+  OpenSCAD
+    aliases: openscad
+    filenames: *.scad
+    mimetypes: text/x-scad
+  Org Mode
+    aliases: org orgmode
+    filenames: *.org
+    mimetypes: text/org
+  PacmanConf
+    aliases: pacmanconf
+    filenames: pacman.conf
+  Perl
+    aliases: perl pl
+    filenames: *.pl *.pm *.t
+    mimetypes: text/x-perl application/x-perl
+  PHP
+    aliases: php php3 php4 php5
+    filenames: *.php *.php[345] *.inc
+    mimetypes: text/x-php
+  PHTML
+    aliases: phtml
+    filenames: *.phtml *.php *.php[345] *.inc
+    mimetypes: application/x-php application/x-httpd-php application/x-httpd-php3 application/x-httpd-php4 application/x-httpd-php5 text/x-php
+  Pig
+    aliases: pig
+    filenames: *.pig
+    mimetypes: text/x-pig
+  PkgConfig
+    aliases: pkgconfig
+    filenames: *.pc
+  PL/pgSQL
+    aliases: plpgsql
+    mimetypes: text/x-plpgsql
+  plaintext
+    aliases: text plain no-highlight
+    filenames: *.txt
+    mimetypes: text/plain
+  Plutus Core
+    aliases: plutus-core plc
+    filenames: *.plc
+    mimetypes: text/x-plutus-core application/x-plutus-core
+  Pony
+    aliases: pony
+    filenames: *.pony
+  PostgreSQL SQL dialect
+    aliases: postgresql postgres
+    mimetypes: text/x-postgresql
+  PostScript
+    aliases: postscript postscr
+    filenames: *.ps *.eps
+    mimetypes: application/postscript
+  POVRay
+    aliases: pov
+    filenames: *.pov *.inc
+    mimetypes: text/x-povray
+  PowerQuery
+    aliases: powerquery pq
+    filenames: *.pq
+    mimetypes: text/x-powerquery
+  PowerShell
+    aliases: powershell posh ps1 psm1 psd1
+    filenames: *.ps1 *.psm1 *.psd1
+    mimetypes: text/x-powershell
+  Prolog
+    aliases: prolog
+    filenames: *.ecl *.prolog *.pro *.pl
+    mimetypes: text/x-prolog
+  PromQL
+    aliases: promql
+    filenames: *.promql
+  Protocol Buffer
+    aliases: protobuf proto
+    filenames: *.proto
+  Puppet
+    aliases: puppet
+    filenames: *.pp
+  Python
+    aliases: python py sage python3 py3
+    filenames: *.py *.pyi *.pyw *.jy *.sage *.sc SConstruct SConscript *.bzl BUCK BUILD BUILD.bazel WORKSPACE *.tac
+    mimetypes: text/x-python application/x-python text/x-python3 application/x-python3
+  Python 2
+    aliases: python2 py2
+    mimetypes: text/x-python2 application/x-python2
+  QBasic
+    aliases: qbasic basic
+    filenames: *.BAS *.bas
+    mimetypes: text/basic
+  QML
+    aliases: qml qbs
+    filenames: *.qml *.qbs
+    mimetypes: application/x-qml application/x-qt.qbs+qml
+  R
+    aliases: splus s r
+    filenames: *.S *.R *.r .Rhistory .Rprofile .Renviron
+    mimetypes: text/S-plus text/S text/x-r-source text/x-r text/x-R text/x-r-history text/x-r-profile
+  Racket
+    aliases: racket rkt
+    filenames: *.rkt *.rktd *.rktl
+    mimetypes: text/x-racket application/x-racket
+  Ragel
+    aliases: ragel
+  Raku
+    aliases: perl6 pl6 raku
+    filenames: *.pl *.pm *.nqp *.p6 *.6pl *.p6l *.pl6 *.6pm *.p6m *.pm6 *.t *.raku *.rakumod *.rakutest *.rakudoc
+    mimetypes: text/x-perl6 application/x-perl6 text/x-raku application/x-raku
+  react
+    aliases: jsx react
+    filenames: *.jsx *.react
+    mimetypes: text/jsx text/typescript-jsx
+  ReasonML
+    aliases: reason reasonml
+    filenames: *.re *.rei
+    mimetypes: text/x-reasonml
+  reg
+    aliases: registry
+    filenames: *.reg
+    mimetypes: text/x-windows-registry
+  reStructuredText
+    aliases: rst rest restructuredtext
+    filenames: *.rst *.rest
+    mimetypes: text/x-rst text/prs.fallenstein.rst
+  Rexx
+    aliases: rexx arexx
+    filenames: *.rexx *.rex *.rx *.arexx
+    mimetypes: text/x-rexx
+  Ruby
+    aliases: rb ruby duby
+    filenames: *.rb *.rbw Rakefile *.rake *.gemspec *.rbx *.duby Gemfile
+    mimetypes: text/x-ruby application/x-ruby
+  Rust
+    aliases: rust rs
+    filenames: *.rs *.rs.in
+    mimetypes: text/rust text/x-rust
+  SAS
+    aliases: sas
+    filenames: *.SAS *.sas
+    mimetypes: text/x-sas text/sas application/x-sas
+  Sass
+    aliases: sass
+    filenames: *.sass
+    mimetypes: text/x-sass
+  Scala
+    aliases: scala
+    filenames: *.scala
+    mimetypes: text/x-scala
+  Scheme
+    aliases: scheme scm
+    filenames: *.scm *.ss
+    mimetypes: text/x-scheme application/x-scheme
+  Scilab
+    aliases: scilab
+    filenames: *.sci *.sce *.tst
+    mimetypes: text/scilab
+  SCSS
+    aliases: scss
+    filenames: *.scss
+    mimetypes: text/x-scss
+  Sieve
+    aliases: sieve
+    filenames: *.siv *.sieve
+  Smalltalk
+    aliases: smalltalk squeak st
+    filenames: *.st
+    mimetypes: text/x-smalltalk
+  Smarty
+    aliases: smarty
+    filenames: *.tpl
+    mimetypes: application/x-smarty
+  Snobol
+    aliases: snobol
+    filenames: *.snobol
+    mimetypes: text/x-snobol
+  Solidity
+    aliases: sol solidity
+    filenames: *.sol
+  SPARQL
+    aliases: sparql
+    filenames: *.rq *.sparql
+    mimetypes: application/sparql-query
+  SQL
+    aliases: sql
+    filenames: *.sql
+    mimetypes: text/x-sql
+  SquidConf
+    aliases: squidconf squid.conf squid
+    filenames: squid.conf
+    mimetypes: text/x-squidconf
+  Standard ML
+    aliases: sml
+    filenames: *.sml *.sig *.fun
+    mimetypes: text/x-standardml application/x-standardml
+  Stylus
+    aliases: stylus
+    filenames: *.styl
+    mimetypes: text/x-styl
+  Svelte
+    aliases: svelte
+    filenames: *.svelte
+    mimetypes: application/x-svelte
+  Swift
+    aliases: swift
+    filenames: *.swift
+    mimetypes: text/x-swift
+  SYSTEMD
+    aliases: systemd
+    filenames: *.automount *.device *.dnssd *.link *.mount *.netdev *.network *.path *.scope *.service *.slice *.socket *.swap *.target *.timer
+    mimetypes: text/plain
+  systemverilog
+    aliases: systemverilog sv
+    filenames: *.sv *.svh
+    mimetypes: text/x-systemverilog
+  TableGen
+    aliases: tablegen
+    filenames: *.td
+    mimetypes: text/x-tablegen
+  TASM
+    aliases: tasm
+    filenames: *.asm *.ASM *.tasm
+    mimetypes: text/x-tasm
+  Tcl
+    aliases: tcl
+    filenames: *.tcl *.rvt
+    mimetypes: text/x-tcl text/x-script.tcl application/x-tcl
+  Tcsh
+    aliases: tcsh csh
+    filenames: *.tcsh *.csh
+    mimetypes: application/x-csh
+  Termcap
+    aliases: termcap
+    filenames: termcap termcap.src
+  Terminfo
+    aliases: terminfo
+    filenames: terminfo terminfo.src
+  Terraform
+    aliases: terraform tf
+    filenames: *.tf
+    mimetypes: application/x-tf application/x-terraform
+  TeX
+    aliases: tex latex
+    filenames: *.tex *.aux *.toc
+    mimetypes: text/x-tex text/x-latex
+  Thrift
+    aliases: thrift
+    filenames: *.thrift
+    mimetypes: application/x-thrift
+  TOML
+    aliases: toml
+    filenames: *.toml
+    mimetypes: text/x-toml
+  TradingView
+    aliases: tradingview tv
+    filenames: *.tv
+    mimetypes: text/x-tradingview
+  Transact-SQL
+    aliases: tsql t-sql
+    mimetypes: text/x-tsql
+  Turing
+    aliases: turing
+    filenames: *.turing *.tu
+    mimetypes: text/x-turing
+  Turtle
+    aliases: turtle
+    filenames: *.ttl
+    mimetypes: text/turtle application/x-turtle
+  Twig
+    aliases: twig
+    mimetypes: application/x-twig
+  TypeScript
+    aliases: ts tsx typescript
+    filenames: *.ts *.tsx
+    mimetypes: text/x-typescript
+  TypoScript
+    aliases: typoscript
+    filenames: *.ts
+    mimetypes: text/x-typoscript
+  TypoScriptCssData
+    aliases: typoscriptcssdata
+  TypoScriptHtmlData
+    aliases: typoscripthtmldata
+  VB.net
+    aliases: vb.net vbnet
+    filenames: *.vb *.bas
+    mimetypes: text/x-vbnet text/x-vba
+  verilog
+    aliases: verilog v
+    filenames: *.v
+    mimetypes: text/x-verilog
+  VHDL
+    aliases: vhdl
+    filenames: *.vhdl *.vhd
+    mimetypes: text/x-vhdl
+  VimL
+    aliases: vim
+    filenames: *.vim .vimrc .exrc .gvimrc _vimrc _exrc _gvimrc vimrc gvimrc
+    mimetypes: text/x-vim
+  vue
+    aliases: vue vuejs
+    filenames: *.vue
+    mimetypes: text/x-vue application/x-vue
+  WDTE
+    filenames: *.wdte
+  XML
+    aliases: xml
+    filenames: *.xml *.xsl *.rss *.xslt *.xsd *.wsdl *.wsf *.svg *.csproj *.vcxproj *.fsproj
+    mimetypes: text/xml application/xml image/svg+xml application/rss+xml application/atom+xml
+  Xorg
+    aliases: xorg.conf
+    filenames: xorg.conf
+  YAML
+    aliases: yaml
+    filenames: *.yaml *.yml
+    mimetypes: text/x-yaml
+  YANG
+    aliases: yang
+    filenames: *.yang
+    mimetypes: application/yang
+  Zed
+    aliases: zed
+    filenames: *.zed
+    mimetypes: text/zed
+  Zig
+    aliases: zig
+    filenames: *.zig
+    mimetypes: text/zig
+
+styles: abap algol algol_nu arduino autumn base16-snazzy borland bw colorful doom-one doom-one2 dracula emacs friendly fruity github hr_high_contrast hrdark igor lovelace manni monokai monokailight murphy native nord onesenterprise paraiso-dark paraiso-light pastie perldoc pygments rainbow_dash rrt solarized-dark solarized-dark256 solarized-light swapoff tango trac vim vs vulcan witchhazel xcode xcode-dark
+formatters: html json noop svg terminal terminal16 terminal16m terminal256 terminal8 tokens

cmd/chroma/main.go

@@ -9,6 +9,8 @@ import (
 	"os"
 	"os/signal"
 	"path"
+	"path/filepath"
+	"regexp"
 	"runtime"
 	"runtime/pprof"
 	"sort"
@@ -16,14 +18,14 @@ import (
 	"strings"
 
 	"github.com/alecthomas/kong"
-	"github.com/mattn/go-colorable"
+	colorable "github.com/mattn/go-colorable"
-	"github.com/mattn/go-isatty"
+	isatty "github.com/mattn/go-isatty"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
-	"github.com/alecthomas/chroma/formatters"
+	"github.com/alecthomas/chroma/v2/formatters"
-	"github.com/alecthomas/chroma/formatters/html"
+	"github.com/alecthomas/chroma/v2/formatters/html"
-	"github.com/alecthomas/chroma/lexers"
+	"github.com/alecthomas/chroma/v2/lexers"
-	"github.com/alecthomas/chroma/styles"
+	"github.com/alecthomas/chroma/v2/styles"
 )
 
 var (
@@ -53,6 +55,8 @@ command, for Go.
 
 		JSON bool `help:"Output JSON representation of tokens."`
 
+		XML string `help:"Generate XML lexer definitions." type:"existingdir" placeholder:"DIR"`
+
 		HTML                      bool   `help:"Enable HTML mode (equivalent to '--formatter html')."`
 		HTMLPrefix                string `help:"HTML CSS class prefix." placeholder:"PREFIX"`
 		HTMLStyles                bool   `help:"Output HTML CSS styles."`
@@ -138,6 +142,11 @@ func main() {
 		"styles":     strings.Join(styles.Names(), ","),
 		"formatters": strings.Join(formatters.Names(), ","),
 	})
+	if cli.XML != "" {
+		err := dumpXMLLexerDefinitions(cli.XML)
+		ctx.FatalIfErrorf(err)
+		return
+	}
 	if cli.List {
 		listAll()
 		return
@@ -279,8 +288,8 @@ func configureHTMLFormatter(ctx *kong.Context) {
 
 func listAll() {
 	fmt.Println("lexers:")
-	sort.Sort(lexers.Registry.Lexers)
+	sort.Sort(lexers.GlobalLexerRegistry.Lexers)
-	for _, l := range lexers.Registry.Lexers {
+	for _, l := range lexers.GlobalLexerRegistry.Lexers {
 		config := l.Config()
 		fmt.Printf("  %s\n", config.Name)
 		filenames := []string{}
@@ -352,3 +361,33 @@ func check(filename string, it chroma.Iterator) {
 		}
 	}
 }
+
+var nameCleanRe = regexp.MustCompile(`[^A-Za-z0-9_#+-]`)
+
+func dumpXMLLexerDefinitions(dir string) error {
+	for _, name := range lexers.Names(false) {
+		lex := lexers.Get(name)
+		if rlex, ok := lex.(*chroma.RegexLexer); ok {
+			data, err := chroma.Marshal(rlex)
+			if err != nil {
+				if errors.Is(err, chroma.ErrNotSerialisable) {
+					fmt.Fprintf(os.Stderr, "warning: %q: %s\n", name, err)
+					continue
+				}
+				return err
+			}
+			name := strings.ToLower(nameCleanRe.ReplaceAllString(lex.Config().Name, "_"))
+			filename := filepath.Join(dir, name) + ".xml"
+			// fmt.Println(name)
+			_, err = os.Stat(filename)
+			if err == nil {
+				return fmt.Errorf("%s already exists", filename)
+			}
+			err = ioutil.WriteFile(filename, data, 0600)
+			if err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}

cmd/chromad/go.mod

@@ -1,9 +1,9 @@
-module github.com/alecthomas/chroma/cmd/chromad
+module github.com/alecthomas/chroma/v2/cmd/chromad
 
-go 1.16
+go 1.17
 
 require (
-	github.com/alecthomas/chroma v0.0.0-00010101000000-000000000000
+	github.com/alecthomas/chroma/v2 v2.0.0-00010101000000-000000000000
 	github.com/alecthomas/kong v0.2.4
 	github.com/alecthomas/kong-hcl v0.2.0
 	github.com/gorilla/csrf v1.6.2
@@ -11,4 +11,4 @@ require (
 	github.com/gorilla/mux v1.7.3
 )
 
-replace github.com/alecthomas/chroma => ../../
+replace github.com/alecthomas/chroma/v2 => ../../

cmd/chromad/go.sum

@@ -3,6 +3,8 @@ github.com/alecthomas/kong v0.2.4 h1:Y0ZBCHAvHhTHw7FFJ2FzCAAG4pkbTgA45nc7BpMhDNk
 github.com/alecthomas/kong v0.2.4/go.mod h1:kQOmtJgV+Lb4aj+I2LEn40cbtawdWJ9Y8QLq+lElKxE=
 github.com/alecthomas/kong-hcl v0.2.0 h1:l1+pkGJm2BtRJF9dCq9hw6KZWEanteY4Ar9suW3Qm0g=
 github.com/alecthomas/kong-hcl v0.2.0/go.mod h1:S5D46RHGG8Ubdxk8TuXBT9wndShsA8/JYSxxiI9y01Y=
+github.com/alecthomas/repr v0.0.0-20220113201626-b1b626ac65ae h1:zzGwJfFlFGD94CyyYwCJeSuD32Gj9GTaSi5y9hoVzdY=
+github.com/alecthomas/repr v0.0.0-20220113201626-b1b626ac65ae/go.mod h1:2kn6fqh/zIyPLmm3ugklbEi5hg5wS435eygvNfaDQL8=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=

cmd/chromad/main.go

@@ -15,10 +15,10 @@ import (
 	"github.com/gorilla/handlers"
 	"github.com/gorilla/mux"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
-	"github.com/alecthomas/chroma/formatters/html"
+	"github.com/alecthomas/chroma/v2/formatters/html"
-	"github.com/alecthomas/chroma/lexers"
+	"github.com/alecthomas/chroma/v2/lexers"
-	"github.com/alecthomas/chroma/styles"
+	"github.com/alecthomas/chroma/v2/styles"
 )
 
 var (
@@ -137,7 +137,7 @@ func newContext(r *http.Request) context {
 	if ctx.SelectedStyle == "" {
 		ctx.SelectedStyle = "monokailight"
 	}
-	for _, lexer := range lexers.Registry.Lexers {
+	for _, lexer := range lexers.GlobalLexerRegistry.Lexers {
 		ctx.Languages = append(ctx.Languages, lexer.Config().Name)
 	}
 	sort.Strings(ctx.Languages)

coalesce_test.go

@@ -7,7 +7,7 @@ import (
 )
 
 func TestCoalesce(t *testing.T) {
-	lexer := Coalesce(MustNewLexer(nil, Rules{ // nolint: forbidigo
+	lexer := Coalesce(mustNewLexer(t, nil, Rules{ // nolint: forbidigo
 		"root": []Rule{
 			{`[!@#$%^&*()]`, Punctuation, nil},
 		},

delegate.go

@@ -24,6 +24,21 @@ func DelegatingLexer(root Lexer, language Lexer) Lexer {
 	}
 }
 
+func (d *delegatingLexer) AnalyseText(text string) float32 {
+	return d.root.AnalyseText(text)
+}
+
+func (d *delegatingLexer) SetAnalyser(analyser func(text string) float32) Lexer {
+	d.root.SetAnalyser(analyser)
+	return d
+}
+
+func (d *delegatingLexer) SetRegistry(r *LexerRegistry) Lexer {
+	d.root.SetRegistry(r)
+	d.language.SetRegistry(r)
+	return d
+}
+
 func (d *delegatingLexer) Config() *Config {
 	return d.language.Config()
 }

delegate_test.go

@@ -6,8 +6,8 @@ import (
 	"github.com/stretchr/testify/assert"
 )
 
-func makeDelegationTestLexers() (lang Lexer, root Lexer) {
+func makeDelegationTestLexers(t *testing.T) (lang Lexer, root Lexer) {
-	return MustNewLexer(nil, Rules{ // nolint: forbidigo
+	return mustNewLexer(t, nil, Rules{ // nolint: forbidigo
 			"root": {
 				{`\<\?`, CommentPreproc, Push("inside")},
 				{`.`, Other, nil},
@@ -18,7 +18,7 @@ func makeDelegationTestLexers() (lang Lexer, root Lexer) {
 				{`\s+`, Whitespace, nil},
 			},
 		}),
-		MustNewLexer(nil, Rules{ // nolint: forbidigo
+		mustNewLexer(t, nil, Rules{ // nolint: forbidigo
 			"root": {
 				{`\bhello\b`, Keyword, nil},
 				{`\b(world|there)\b`, Name, nil},
@@ -97,7 +97,7 @@ func TestDelegate(t *testing.T) {
 			{Keyword, "hello"},
 		}},
 	}
-	lang, root := makeDelegationTestLexers()
+	lang, root := makeDelegationTestLexers(t)
 	delegate := DelegatingLexer(root, lang)
 	for _, test := range testdata {
 		// nolint: scopelint

emitters.go

@@ -0,0 +1,221 @@
+package chroma
+
+import (
+	"fmt"
+)
+
+// An Emitter takes group matches and returns tokens.
+type Emitter interface {
+	// Emit tokens for the given regex groups.
+	Emit(groups []string, state *LexerState) Iterator
+}
+
+// SerialisableEmitter is an Emitter that can be serialised and deserialised to/from JSON.
+type SerialisableEmitter interface {
+	Emitter
+	EmitterKind() string
+}
+
+// EmitterFunc is a function that is an Emitter.
+type EmitterFunc func(groups []string, state *LexerState) Iterator
+
+// Emit tokens for groups.
+func (e EmitterFunc) Emit(groups []string, state *LexerState) Iterator {
+	return e(groups, state)
+}
+
+type Emitters []Emitter
+
+type byGroupsEmitter struct {
+	Emitters
+}
+
+// ByGroups emits a token for each matching group in the rule's regex.
+func ByGroups(emitters ...Emitter) Emitter {
+	return &byGroupsEmitter{Emitters: emitters}
+}
+
+func (b *byGroupsEmitter) EmitterKind() string { return "bygroups" }
+
+func (b *byGroupsEmitter) Emit(groups []string, state *LexerState) Iterator {
+	iterators := make([]Iterator, 0, len(groups)-1)
+	if len(b.Emitters) != len(groups)-1 {
+		iterators = append(iterators, Error.Emit(groups, state))
+		// panic(errors.Errorf("number of groups %q does not match number of emitters %v", groups, emitters))
+	} else {
+		for i, group := range groups[1:] {
+			if b.Emitters[i] != nil {
+				iterators = append(iterators, b.Emitters[i].Emit([]string{group}, state))
+			}
+		}
+	}
+	return Concaterator(iterators...)
+}
+
+// ByGroupNames emits a token for each named matching group in the rule's regex.
+func ByGroupNames(emitters map[string]Emitter) Emitter {
+	return EmitterFunc(func(groups []string, state *LexerState) Iterator {
+		iterators := make([]Iterator, 0, len(state.NamedGroups)-1)
+		if len(state.NamedGroups)-1 == 0 {
+			if emitter, ok := emitters[`0`]; ok {
+				iterators = append(iterators, emitter.Emit(groups, state))
+			} else {
+				iterators = append(iterators, Error.Emit(groups, state))
+			}
+		} else {
+			ruleRegex := state.Rules[state.State][state.Rule].Regexp
+			for i := 1; i < len(state.NamedGroups); i++ {
+				groupName := ruleRegex.GroupNameFromNumber(i)
+				group := state.NamedGroups[groupName]
+				if emitter, ok := emitters[groupName]; ok {
+					if emitter != nil {
+						iterators = append(iterators, emitter.Emit([]string{group}, state))
+					}
+				} else {
+					iterators = append(iterators, Error.Emit([]string{group}, state))
+				}
+			}
+		}
+		return Concaterator(iterators...)
+	})
+}
+
+// UsingByGroup emits tokens for the matched groups in the regex using a
+// "sublexer". Used when lexing code blocks where the name of a sublexer is
+// contained within the block, for example on a Markdown text block or SQL
+// language block.
+//
+// The sublexer will be retrieved using sublexerGetFunc (typically
+// internal.Get), using the captured value from the matched sublexerNameGroup.
+//
+// If sublexerGetFunc returns a non-nil lexer for the captured sublexerNameGroup,
+// then tokens for the matched codeGroup will be emitted using the retrieved
+// lexer. Otherwise, if the sublexer is nil, then tokens will be emitted from
+// the passed emitter.
+//
+// Example:
+//
+// 	var Markdown = internal.Register(MustNewLexer(
+// 		&Config{
+// 			Name:      "markdown",
+// 			Aliases:   []string{"md", "mkd"},
+// 			Filenames: []string{"*.md", "*.mkd", "*.markdown"},
+// 			MimeTypes: []string{"text/x-markdown"},
+// 		},
+// 		Rules{
+// 			"root": {
+// 				{"^(```)(\\w+)(\\n)([\\w\\W]*?)(^```$)",
+// 					UsingByGroup(
+// 						internal.Get,
+// 						2, 4,
+// 						String, String, String, Text, String,
+// 					),
+// 					nil,
+// 				},
+// 			},
+// 		},
+// 	))
+//
+// See the lexers/m/markdown.go for the complete example.
+//
+// Note: panic's if the number of emitters does not equal the number of matched
+// groups in the regex.
+func UsingByGroup(sublexerNameGroup, codeGroup int, emitters ...Emitter) Emitter {
+	return &usingByGroup{
+		SublexerNameGroup: sublexerNameGroup,
+		CodeGroup:         codeGroup,
+		Emitters:          emitters,
+	}
+}
+
+type usingByGroup struct {
+	SublexerNameGroup int      `xml:"sublexer_name_group"`
+	CodeGroup         int      `xml:"code_group"`
+	Emitters          Emitters `xml:"emitters"`
+}
+
+func (u *usingByGroup) EmitterKind() string { return "usingbygroup" }
+func (u *usingByGroup) Emit(groups []string, state *LexerState) Iterator {
+	// bounds check
+	if len(u.Emitters) != len(groups)-1 {
+		panic("UsingByGroup expects number of emitters to be the same as len(groups)-1")
+	}
+
+	// grab sublexer
+	sublexer := state.Registry.Get(groups[u.SublexerNameGroup])
+
+	// build iterators
+	iterators := make([]Iterator, len(groups)-1)
+	for i, group := range groups[1:] {
+		if i == u.CodeGroup-1 && sublexer != nil {
+			var err error
+			iterators[i], err = sublexer.Tokenise(nil, groups[u.CodeGroup])
+			if err != nil {
+				panic(err)
+			}
+		} else if u.Emitters[i] != nil {
+			iterators[i] = u.Emitters[i].Emit([]string{group}, state)
+		}
+	}
+	return Concaterator(iterators...)
+}
+
+// UsingLexer returns an Emitter that uses a given Lexer for parsing and emitting.
+//
+// This Emitter is not serialisable.
+func UsingLexer(lexer Lexer) Emitter {
+	return EmitterFunc(func(groups []string, _ *LexerState) Iterator {
+		it, err := lexer.Tokenise(&TokeniseOptions{State: "root", Nested: true}, groups[0])
+		if err != nil {
+			panic(err)
+		}
+		return it
+	})
+}
+
+type usingEmitter struct {
+	Lexer string `xml:"lexer,attr"`
+}
+
+func (u *usingEmitter) EmitterKind() string { return "using" }
+
+func (u *usingEmitter) Emit(groups []string, state *LexerState) Iterator {
+	if state.Registry == nil {
+		panic(fmt.Sprintf("no LexerRegistry available for Using(%q)", u.Lexer))
+	}
+	lexer := state.Registry.Get(u.Lexer)
+	if lexer == nil {
+		panic(fmt.Sprintf("no such lexer %q", u.Lexer))
+	}
+	it, err := lexer.Tokenise(&TokeniseOptions{State: "root", Nested: true}, groups[0])
+	if err != nil {
+		panic(err)
+	}
+	return it
+}
+
+// Using returns an Emitter that uses a given Lexer reference for parsing and emitting.
+//
+// The referenced lexer must be stored in the same LexerRegistry.
+func Using(lexer string) Emitter {
+	return &usingEmitter{Lexer: lexer}
+}
+
+type usingSelfEmitter struct {
+	State string `xml:"state,attr"`
+}
+
+func (u *usingSelfEmitter) EmitterKind() string { return "usingself" }
+
+func (u *usingSelfEmitter) Emit(groups []string, state *LexerState) Iterator {
+	it, err := state.Lexer.Tokenise(&TokeniseOptions{State: u.State, Nested: true}, groups[0])
+	if err != nil {
+		panic(err)
+	}
+	return it
+}
+
+// UsingSelf is like Using, but uses the current Lexer.
+func UsingSelf(stateName string) Emitter {
+	return &usingSelfEmitter{stateName}
+}

formatters/api.go

@@ -4,9 +4,9 @@ import (
 	"io"
 	"sort"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
-	"github.com/alecthomas/chroma/formatters/html"
+	"github.com/alecthomas/chroma/v2/formatters/html"
-	"github.com/alecthomas/chroma/formatters/svg"
+	"github.com/alecthomas/chroma/v2/formatters/svg"
 )
 
 var (

formatters/html/html.go

@@ -7,7 +7,7 @@ import (
 	"sort"
 	"strings"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
 )
 
 // Option sets an option of the HTML formatter.

formatters/html/html_test.go

@@ -9,9 +9,9 @@ import (
 
 	"github.com/stretchr/testify/assert"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
-	"github.com/alecthomas/chroma/lexers"
+	"github.com/alecthomas/chroma/v2/lexers"
-	"github.com/alecthomas/chroma/styles"
+	"github.com/alecthomas/chroma/v2/styles"
 )
 
 func TestCompressStyle(t *testing.T) {

formatters/json.go

@@ -5,7 +5,7 @@ import (
 	"fmt"
 	"io"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
 )
 
 // JSON formatter outputs the raw token structures as JSON.

formatters/svg/svg.go

@@ -10,7 +10,7 @@ import (
 	"path"
 	"strings"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
 )
 
 // Option sets an option of the SVG formatter.

formatters/tokens.go

@@ -4,7 +4,7 @@ import (
 	"fmt"
 	"io"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
 )
 
 // Tokens formatter outputs the raw token structures.

formatters/tty_indexed.go

@@ -5,7 +5,7 @@ import (
 	"io"
 	"math"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
 )
 
 type ttyTable struct {

formatters/tty_indexed_test.go

@@ -3,7 +3,7 @@ package formatters
 import (
 	"testing"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
 	"github.com/stretchr/testify/assert"
 )
 

formatters/tty_truecolour.go

@@ -4,7 +4,7 @@ import (
 	"fmt"
 	"io"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
 )
 
 // TTY16m is a true-colour terminal formatter.

go.mod

@@ -1,8 +1,9 @@
-module github.com/alecthomas/chroma
+module github.com/alecthomas/chroma/v2
 
-go 1.13
+go 1.17
 
 require (
+	github.com/alecthomas/repr v0.0.0-20220113201626-b1b626ac65ae // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/dlclark/regexp2 v1.4.0
 	github.com/stretchr/testify v1.7.0

go.sum

@@ -1,3 +1,5 @@
+github.com/alecthomas/repr v0.0.0-20220113201626-b1b626ac65ae h1:zzGwJfFlFGD94CyyYwCJeSuD32Gj9GTaSi5y9hoVzdY=
+github.com/alecthomas/repr v0.0.0-20220113201626-b1b626ac65ae/go.mod h1:2kn6fqh/zIyPLmm3ugklbEi5hg5wS435eygvNfaDQL8=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=

lexer.go

@@ -15,30 +15,30 @@ var (
 // Config for a lexer.
 type Config struct {
 	// Name of the lexer.
-	Name string
+	Name string `xml:"name,omitempty"`
 
 	// Shortcuts for the lexer
-	Aliases []string
+	Aliases []string `xml:"alias,omitempty"`
 
 	// File name globs
-	Filenames []string
+	Filenames []string `xml:"filename,omitempty"`
 
 	// Secondary file name globs
-	AliasFilenames []string
+	AliasFilenames []string `xml:"alias_filename,omitempty"`
 
 	// MIME types
-	MimeTypes []string
+	MimeTypes []string `xml:"mime_type,omitempty"`
 
 	// Regex matching is case-insensitive.
-	CaseInsensitive bool
+	CaseInsensitive bool `xml:"case_insensitive,omitempty"`
 
 	// Regex matches all characters.
-	DotAll bool
+	DotAll bool `xml:"dot_all,omitempty"`
 
 	// Regex does not match across lines ($ matches EOL).
 	//
 	// Defaults to multiline.
-	NotMultiline bool
+	NotMultiline bool `xml:"not_multiline,omitempty"`
 
 	// Don't strip leading and trailing newlines from the input.
 	// DontStripNL bool
@@ -48,7 +48,7 @@ type Config struct {
 
 	// Make sure that the input ends with a newline. This
 	// is required for some lexers that consume input linewise.
-	EnsureNL bool
+	EnsureNL bool `xml:"ensure_nl,omitempty"`
 
 	// If given and greater than 0, expand tabs in the input.
 	// TabSize int
@@ -56,7 +56,7 @@ type Config struct {
 	// Priority of lexer.
 	//
 	// If this is 0 it will be treated as a default of 1.
-	Priority float32
+	Priority float32 `xml:"priority,omitempty"`
 }
 
 // Token output to formatter.
@@ -94,6 +94,20 @@ type Lexer interface {
 	Config() *Config
 	// Tokenise returns an Iterator over tokens in text.
 	Tokenise(options *TokeniseOptions, text string) (Iterator, error)
+	// SetRegistry sets the registry this Lexer is associated with.
+	//
+	// The registry should be used by the Lexer if it needs to look up other
+	// lexers.
+	SetRegistry(registry *LexerRegistry) Lexer
+	// SetAnalyser sets a function the Lexer should use for scoring how
+	// likely a fragment of text is to match this lexer, between 0.0 and 1.0.
+	// A value of 1 indicates high confidence.
+	//
+	// Lexers may ignore this if they implement their own analysers.
+	SetAnalyser(analyser func(text string) float32) Lexer
+	// AnalyseText scores how likely a fragment of text is to match
+	// this lexer, between 0.0 and 1.0. A value of 1 indicates high confidence.
+	AnalyseText(text string) float32
 }
 
 // Lexers is a slice of lexers sortable by name.

lexer_test.go

@@ -3,38 +3,34 @@ package chroma
 import (
 	"testing"
 
-	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
 )
 
 func TestTokenTypeClassifiers(t *testing.T) {
-	assert.True(t, GenericDeleted.InCategory(Generic))
+	require.True(t, GenericDeleted.InCategory(Generic))
-	assert.True(t, LiteralStringBacktick.InSubCategory(String))
+	require.True(t, LiteralStringBacktick.InSubCategory(String))
-	assert.Equal(t, LiteralStringBacktick.String(), "LiteralStringBacktick")
+	require.Equal(t, LiteralStringBacktick.String(), "LiteralStringBacktick")
 }
 
 func TestSimpleLexer(t *testing.T) {
-	lexer, err := NewLexer( // nolint: forbidigo
+	lexer := mustNewLexer(t, &Config{
-		&Config{
 		Name:      "INI",
 		Aliases:   []string{"ini", "cfg"},
 		Filenames: []string{"*.ini", "*.cfg"},
-		},
+	}, map[string][]Rule{
-		map[string][]Rule{
 		"root": {
 			{`\s+`, Whitespace, nil},
 			{`;.*?$`, Comment, nil},
 			{`\[.*?\]$`, Keyword, nil},
 			{`(.*?)(\s*)(=)(\s*)(.*?)$`, ByGroups(Name, Whitespace, Operator, Whitespace, String), nil},
 		},
-		},
+	})
-	)
-	assert.NoError(t, err)
 	actual, err := Tokenise(lexer, nil, `
 	; this is a comment
 	[section]
 	a = 10
 `)
-	assert.NoError(t, err)
+	require.NoError(t, err)
 	expected := []Token{
 		{Whitespace, "\n\t"},
 		{Comment, "; this is a comment"},
@@ -48,5 +44,5 @@ func TestSimpleLexer(t *testing.T) {
 		{LiteralString, "10"},
 		{Whitespace, "\n"},
 	}
-	assert.Equal(t, expected, actual)
+	require.Equal(t, expected, actual)
 }

lexers/a/abap.go

@@ -1,60 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// ABAP lexer.
-var Abap = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "ABAP",
-		Aliases:         []string{"abap"},
-		Filenames:       []string{"*.abap", "*.ABAP"},
-		MimeTypes:       []string{"text/x-abap"},
-		CaseInsensitive: true,
-	},
-	abapRules,
-))
-
-func abapRules() Rules {
-	return Rules{
-		"common": {
-			{`\s+`, Text, nil},
-			{`^\*.*$`, CommentSingle, nil},
-			{`\".*?\n`, CommentSingle, nil},
-			{`##\w+`, CommentSpecial, nil},
-		},
-		"variable-names": {
-			{`<\S+>`, NameVariable, nil},
-			{`\w[\w~]*(?:(\[\])|->\*)?`, NameVariable, nil},
-		},
-		"root": {
-			Include("common"),
-			{`CALL\s+(?:BADI|CUSTOMER-FUNCTION|FUNCTION)`, Keyword, nil},
-			{`(CALL\s+(?:DIALOG|SCREEN|SUBSCREEN|SELECTION-SCREEN|TRANSACTION|TRANSFORMATION))\b`, Keyword, nil},
-			{`(FORM|PERFORM)(\s+)(\w+)`, ByGroups(Keyword, Text, NameFunction), nil},
-			{`(PERFORM)(\s+)(\()(\w+)(\))`, ByGroups(Keyword, Text, Punctuation, NameVariable, Punctuation), nil},
-			{`(MODULE)(\s+)(\S+)(\s+)(INPUT|OUTPUT)`, ByGroups(Keyword, Text, NameFunction, Text, Keyword), nil},
-			{`(METHOD)(\s+)([\w~]+)`, ByGroups(Keyword, Text, NameFunction), nil},
-			{`(\s+)([\w\-]+)([=\-]>)([\w\-~]+)`, ByGroups(Text, NameVariable, Operator, NameFunction), nil},
-			{`(?<=(=|-)>)([\w\-~]+)(?=\()`, NameFunction, nil},
-			{`(TEXT)(-)(\d{3})`, ByGroups(Keyword, Punctuation, LiteralNumberInteger), nil},
-			{`(TEXT)(-)(\w{3})`, ByGroups(Keyword, Punctuation, NameVariable), nil},
-			{`(ADD-CORRESPONDING|AUTHORITY-CHECK|CLASS-DATA|CLASS-EVENTS|CLASS-METHODS|CLASS-POOL|DELETE-ADJACENT|DIVIDE-CORRESPONDING|EDITOR-CALL|ENHANCEMENT-POINT|ENHANCEMENT-SECTION|EXIT-COMMAND|FIELD-GROUPS|FIELD-SYMBOLS|FUNCTION-POOL|INTERFACE-POOL|INVERTED-DATE|LOAD-OF-PROGRAM|LOG-POINT|MESSAGE-ID|MOVE-CORRESPONDING|MULTIPLY-CORRESPONDING|NEW-LINE|NEW-PAGE|NEW-SECTION|NO-EXTENSION|OUTPUT-LENGTH|PRINT-CONTROL|SELECT-OPTIONS|START-OF-SELECTION|SUBTRACT-CORRESPONDING|SYNTAX-CHECK|SYSTEM-EXCEPTIONS|TYPE-POOL|TYPE-POOLS|NO-DISPLAY)\b`, Keyword, nil},
-			{`(?<![-\>])(CREATE\s+(PUBLIC|PRIVATE|DATA|OBJECT)|(PUBLIC|PRIVATE|PROTECTED)\s+SECTION|(TYPE|LIKE)\s+((LINE\s+OF|REF\s+TO|(SORTED|STANDARD|HASHED)\s+TABLE\s+OF))?|FROM\s+(DATABASE|MEMORY)|CALL\s+METHOD|(GROUP|ORDER) BY|HAVING|SEPARATED BY|GET\s+(BADI|BIT|CURSOR|DATASET|LOCALE|PARAMETER|PF-STATUS|(PROPERTY|REFERENCE)\s+OF|RUN\s+TIME|TIME\s+(STAMP)?)?|SET\s+(BIT|BLANK\s+LINES|COUNTRY|CURSOR|DATASET|EXTENDED\s+CHECK|HANDLER|HOLD\s+DATA|LANGUAGE|LEFT\s+SCROLL-BOUNDARY|LOCALE|MARGIN|PARAMETER|PF-STATUS|PROPERTY\s+OF|RUN\s+TIME\s+(ANALYZER|CLOCK\s+RESOLUTION)|SCREEN|TITLEBAR|UPADTE\s+TASK\s+LOCAL|USER-COMMAND)|CONVERT\s+((INVERTED-)?DATE|TIME|TIME\s+STAMP|TEXT)|(CLOSE|OPEN)\s+(DATASET|CURSOR)|(TO|FROM)\s+(DATA BUFFER|INTERNAL TABLE|MEMORY ID|DATABASE|SHARED\s+(MEMORY|BUFFER))|DESCRIBE\s+(DISTANCE\s+BETWEEN|FIELD|LIST|TABLE)|FREE\s(MEMORY|OBJECT)?|PROCESS\s+(BEFORE\s+OUTPUT|AFTER\s+INPUT|ON\s+(VALUE-REQUEST|HELP-REQUEST))|AT\s+(LINE-SELECTION|USER-COMMAND|END\s+OF|NEW)|AT\s+SELECTION-SCREEN(\s+(ON(\s+(BLOCK|(HELP|VALUE)-REQUEST\s+FOR|END\s+OF|RADIOBUTTON\s+GROUP))?|OUTPUT))?|SELECTION-SCREEN:?\s+((BEGIN|END)\s+OF\s+((TABBED\s+)?BLOCK|LINE|SCREEN)|COMMENT|FUNCTION\s+KEY|INCLUDE\s+BLOCKS|POSITION|PUSHBUTTON|SKIP|ULINE)|LEAVE\s+(LIST-PROCESSING|PROGRAM|SCREEN|TO LIST-PROCESSING|TO TRANSACTION)(ENDING|STARTING)\s+AT|FORMAT\s+(COLOR|INTENSIFIED|INVERSE|HOTSPOT|INPUT|FRAMES|RESET)|AS\s+(CHECKBOX|SUBSCREEN|WINDOW)|WITH\s+(((NON-)?UNIQUE)?\s+KEY|FRAME)|(BEGIN|END)\s+OF|DELETE(\s+ADJACENT\s+DUPLICATES\sFROM)?|COMPARING(\s+ALL\s+FIELDS)?|(INSERT|APPEND)(\s+INITIAL\s+LINE\s+(IN)?TO|\s+LINES\s+OF)?|IN\s+((BYTE|CHARACTER)\s+MODE|PROGRAM)|END-OF-(DEFINITION|PAGE|SELECTION)|WITH\s+FRAME(\s+TITLE)|(REPLACE|FIND)\s+((FIRST|ALL)\s+OCCURRENCES?\s+OF\s+)?(SUBSTRING|REGEX)?|MATCH\s+(LENGTH|COUNT|LINE|OFFSET)|(RESPECTING|IGNORING)\s+CASE|IN\s+UPDATE\s+TASK|(SOURCE|RESULT)\s+(XML)?|REFERENCE\s+INTO|AND\s+(MARK|RETURN)|CLIENT\s+SPECIFIED|CORRESPONDING\s+FIELDS\s+OF|IF\s+FOUND|FOR\s+EVENT|INHERITING\s+FROM|LEAVE\s+TO\s+SCREEN|LOOP\s+AT\s+(SCREEN)?|LOWER\s+CASE|MATCHCODE\s+OBJECT|MODIF\s+ID|MODIFY\s+SCREEN|NESTING\s+LEVEL|NO\s+INTERVALS|OF\s+STRUCTURE|RADIOBUTTON\s+GROUP|RANGE\s+OF|REF\s+TO|SUPPRESS DIALOG|TABLE\s+OF|UPPER\s+CASE|TRANSPORTING\s+NO\s+FIELDS|VALUE\s+CHECK|VISIBLE\s+LENGTH|HEADER\s+LINE|COMMON\s+PART)\b`, Keyword, nil},
-			{`(^|(?<=(\s|\.)))(ABBREVIATED|ABSTRACT|ADD|ALIASES|ALIGN|ALPHA|ASSERT|AS|ASSIGN(ING)?|AT(\s+FIRST)?|BACK|BLOCK|BREAK-POINT|CASE|CATCH|CHANGING|CHECK|CLASS|CLEAR|COLLECT|COLOR|COMMIT|CREATE|COMMUNICATION|COMPONENTS?|COMPUTE|CONCATENATE|CONDENSE|CONSTANTS|CONTEXTS|CONTINUE|CONTROLS|COUNTRY|CURRENCY|DATA|DATE|DECIMALS|DEFAULT|DEFINE|DEFINITION|DEFERRED|DEMAND|DETAIL|DIRECTORY|DIVIDE|DO|DUMMY|ELSE(IF)?|ENDAT|ENDCASE|ENDCATCH|ENDCLASS|ENDDO|ENDFORM|ENDFUNCTION|ENDIF|ENDINTERFACE|ENDLOOP|ENDMETHOD|ENDMODULE|ENDSELECT|ENDTRY|ENDWHILE|ENHANCEMENT|EVENTS|EXACT|EXCEPTIONS?|EXIT|EXPONENT|EXPORT|EXPORTING|EXTRACT|FETCH|FIELDS?|FOR|FORM|FORMAT|FREE|FROM|FUNCTION|HIDE|ID|IF|IMPORT|IMPLEMENTATION|IMPORTING|IN|INCLUDE|INCLUDING|INDEX|INFOTYPES|INITIALIZATION|INTERFACE|INTERFACES|INTO|LANGUAGE|LEAVE|LENGTH|LINES|LOAD|LOCAL|JOIN|KEY|NEXT|MAXIMUM|MESSAGE|METHOD[S]?|MINIMUM|MODULE|MODIFIER|MODIFY|MOVE|MULTIPLY|NODES|NUMBER|OBLIGATORY|OBJECT|OF|OFF|ON|OTHERS|OVERLAY|PACK|PAD|PARAMETERS|PERCENTAGE|POSITION|PROGRAM|PROVIDE|PUBLIC|PUT|PF\d\d|RAISE|RAISING|RANGES?|READ|RECEIVE|REDEFINITION|REFRESH|REJECT|REPORT|RESERVE|RESUME|RETRY|RETURN|RETURNING|RIGHT|ROLLBACK|REPLACE|SCROLL|SEARCH|SELECT|SHIFT|SIGN|SINGLE|SIZE|SKIP|SORT|SPLIT|STATICS|STOP|STYLE|SUBMATCHES|SUBMIT|SUBTRACT|SUM(?!\()|SUMMARY|SUMMING|SUPPLY|TABLE|TABLES|TIMESTAMP|TIMES?|TIMEZONE|TITLE|\??TO|TOP-OF-PAGE|TRANSFER|TRANSLATE|TRY|TYPES|ULINE|UNDER|UNPACK|UPDATE|USING|VALUE|VALUES|VIA|VARYING|VARY|WAIT|WHEN|WHERE|WIDTH|WHILE|WITH|WINDOW|WRITE|XSD|ZERO)\b`, Keyword, nil},
-			{`(abs|acos|asin|atan|boolc|boolx|bit_set|char_off|charlen|ceil|cmax|cmin|condense|contains|contains_any_of|contains_any_not_of|concat_lines_of|cos|cosh|count|count_any_of|count_any_not_of|dbmaxlen|distance|escape|exp|find|find_end|find_any_of|find_any_not_of|floor|frac|from_mixed|insert|lines|log|log10|match|matches|nmax|nmin|numofchar|repeat|replace|rescale|reverse|round|segment|shift_left|shift_right|sign|sin|sinh|sqrt|strlen|substring|substring_after|substring_from|substring_before|substring_to|tan|tanh|to_upper|to_lower|to_mixed|translate|trunc|xstrlen)(\()\b`, ByGroups(NameBuiltin, Punctuation), nil},
-			{`&[0-9]`, Name, nil},
-			{`[0-9]+`, LiteralNumberInteger, nil},
-			{`(?<=(\s|.))(AND|OR|EQ|NE|GT|LT|GE|LE|CO|CN|CA|NA|CS|NOT|NS|CP|NP|BYTE-CO|BYTE-CN|BYTE-CA|BYTE-NA|BYTE-CS|BYTE-NS|IS\s+(NOT\s+)?(INITIAL|ASSIGNED|REQUESTED|BOUND))\b`, OperatorWord, nil},
-			Include("variable-names"),
-			{`[?*<>=\-+&]`, Operator, nil},
-			{`'(''|[^'])*'`, LiteralStringSingle, nil},
-			{"`([^`])*`", LiteralStringSingle, nil},
-			{`([|}])([^{}|]*?)([|{])`, ByGroups(Punctuation, LiteralStringSingle, Punctuation), nil},
-			{`[/;:()\[\],.]`, Punctuation, nil},
-			{`(!)(\w+)`, ByGroups(Operator, Name), nil},
-		},
-	}
-}

lexers/a/abnf.go

@@ -1,42 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Abnf lexer.
-var Abnf = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "ABNF",
-		Aliases:   []string{"abnf"},
-		Filenames: []string{"*.abnf"},
-		MimeTypes: []string{"text/x-abnf"},
-	},
-	abnfRules,
-))
-
-func abnfRules() Rules {
-	return Rules{
-		"root": {
-			{`;.*$`, CommentSingle, nil},
-			{`(%[si])?"[^"]*"`, Literal, nil},
-			{`%b[01]+\-[01]+\b`, Literal, nil},
-			{`%b[01]+(\.[01]+)*\b`, Literal, nil},
-			{`%d[0-9]+\-[0-9]+\b`, Literal, nil},
-			{`%d[0-9]+(\.[0-9]+)*\b`, Literal, nil},
-			{`%x[0-9a-fA-F]+\-[0-9a-fA-F]+\b`, Literal, nil},
-			{`%x[0-9a-fA-F]+(\.[0-9a-fA-F]+)*\b`, Literal, nil},
-			{`\b[0-9]+\*[0-9]+`, Operator, nil},
-			{`\b[0-9]+\*`, Operator, nil},
-			{`\b[0-9]+`, Operator, nil},
-			{`\*`, Operator, nil},
-			{Words(``, `\b`, `ALPHA`, `BIT`, `CHAR`, `CR`, `CRLF`, `CTL`, `DIGIT`, `DQUOTE`, `HEXDIG`, `HTAB`, `LF`, `LWSP`, `OCTET`, `SP`, `VCHAR`, `WSP`), Keyword, nil},
-			{`[a-zA-Z][a-zA-Z0-9-]+\b`, NameClass, nil},
-			{`(=/|=|/)`, Operator, nil},
-			{`[\[\]()]`, Punctuation, nil},
-			{`\s+`, Text, nil},
-			{`.`, Text, nil},
-		},
-	}
-}

lexers/a/actionscript.go

@@ -1,43 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Actionscript lexer.
-var Actionscript = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:         "ActionScript",
-		Aliases:      []string{"as", "actionscript"},
-		Filenames:    []string{"*.as"},
-		MimeTypes:    []string{"application/x-actionscript", "text/x-actionscript", "text/actionscript"},
-		NotMultiline: true,
-		DotAll:       true,
-	},
-	actionscriptRules,
-))
-
-func actionscriptRules() Rules {
-	return Rules{
-		"root": {
-			{`\s+`, Text, nil},
-			{`//.*?\n`, CommentSingle, nil},
-			{`/\*.*?\*/`, CommentMultiline, nil},
-			{`/(\\\\|\\/|[^/\n])*/[gim]*`, LiteralStringRegex, nil},
-			{`[~^*!%&<>|+=:;,/?\\-]+`, Operator, nil},
-			{`[{}\[\]();.]+`, Punctuation, nil},
-			{Words(``, `\b`, `case`, `default`, `for`, `each`, `in`, `while`, `do`, `break`, `return`, `continue`, `if`, `else`, `throw`, `try`, `catch`, `var`, `with`, `new`, `typeof`, `arguments`, `instanceof`, `this`, `switch`), Keyword, nil},
-			{Words(``, `\b`, `class`, `public`, `final`, `internal`, `native`, `override`, `private`, `protected`, `static`, `import`, `extends`, `implements`, `interface`, `intrinsic`, `return`, `super`, `dynamic`, `function`, `const`, `get`, `namespace`, `package`, `set`), KeywordDeclaration, nil},
-			{`(true|false|null|NaN|Infinity|-Infinity|undefined|Void)\b`, KeywordConstant, nil},
-			{Words(``, `\b`, `Accessibility`, `AccessibilityProperties`, `ActionScriptVersion`, `ActivityEvent`, `AntiAliasType`, `ApplicationDomain`, `AsBroadcaster`, `Array`, `AsyncErrorEvent`, `AVM1Movie`, `BevelFilter`, `Bitmap`, `BitmapData`, `BitmapDataChannel`, `BitmapFilter`, `BitmapFilterQuality`, `BitmapFilterType`, `BlendMode`, `BlurFilter`, `Boolean`, `ByteArray`, `Camera`, `Capabilities`, `CapsStyle`, `Class`, `Color`, `ColorMatrixFilter`, `ColorTransform`, `ContextMenu`, `ContextMenuBuiltInItems`, `ContextMenuEvent`, `ContextMenuItem`, `ConvultionFilter`, `CSMSettings`, `DataEvent`, `Date`, `DefinitionError`, `DeleteObjectSample`, `Dictionary`, `DisplacmentMapFilter`, `DisplayObject`, `DisplacmentMapFilterMode`, `DisplayObjectContainer`, `DropShadowFilter`, `Endian`, `EOFError`, `Error`, `ErrorEvent`, `EvalError`, `Event`, `EventDispatcher`, `EventPhase`, `ExternalInterface`, `FileFilter`, `FileReference`, `FileReferenceList`, `FocusDirection`, `FocusEvent`, `Font`, `FontStyle`, `FontType`, `FrameLabel`, `FullScreenEvent`, `Function`, `GlowFilter`, `GradientBevelFilter`, `GradientGlowFilter`, `GradientType`, `Graphics`, `GridFitType`, `HTTPStatusEvent`, `IBitmapDrawable`, `ID3Info`, `IDataInput`, `IDataOutput`, `IDynamicPropertyOutputIDynamicPropertyWriter`, `IEventDispatcher`, `IExternalizable`, `IllegalOperationError`, `IME`, `IMEConversionMode`, `IMEEvent`, `int`, `InteractiveObject`, `InterpolationMethod`, `InvalidSWFError`, `InvokeEvent`, `IOError`, `IOErrorEvent`, `JointStyle`, `Key`, `Keyboard`, `KeyboardEvent`, `KeyLocation`, `LineScaleMode`, `Loader`, `LoaderContext`, `LoaderInfo`, `LoadVars`, `LocalConnection`, `Locale`, `Math`, `Matrix`, `MemoryError`, `Microphone`, `MorphShape`, `Mouse`, `MouseEvent`, `MovieClip`, `MovieClipLoader`, `Namespace`, `NetConnection`, `NetStatusEvent`, `NetStream`, `NewObjectSample`, `Number`, `Object`, `ObjectEncoding`, `PixelSnapping`, `Point`, `PrintJob`, `PrintJobOptions`, `PrintJobOrientation`, `ProgressEvent`, `Proxy`, `QName`, `RangeError`, `Rectangle`, `ReferenceError`, `RegExp`, `Responder`, `Sample`, `Scene`, `ScriptTimeoutError`, `Security`, `SecurityDomain`, `SecurityError`, `SecurityErrorEvent`, `SecurityPanel`, `Selection`, `Shape`, `SharedObject`, `SharedObjectFlushStatus`, `SimpleButton`, `Socket`, `Sound`, `SoundChannel`, `SoundLoaderContext`, `SoundMixer`, `SoundTransform`, `SpreadMethod`, `Sprite`, `StackFrame`, `StackOverflowError`, `Stage`, `StageAlign`, `StageDisplayState`, `StageQuality`, `StageScaleMode`, `StaticText`, `StatusEvent`, `String`, `StyleSheet`, `SWFVersion`, `SyncEvent`, `SyntaxError`, `System`, `TextColorType`, `TextField`, `TextFieldAutoSize`, `TextFieldType`, `TextFormat`, `TextFormatAlign`, `TextLineMetrics`, `TextRenderer`, `TextSnapshot`, `Timer`, `TimerEvent`, `Transform`, `TypeError`, `uint`, `URIError`, `URLLoader`, `URLLoaderDataFormat`, `URLRequest`, `URLRequestHeader`, `URLRequestMethod`, `URLStream`, `URLVariabeles`, `VerifyError`, `Video`, `XML`, `XMLDocument`, `XMLList`, `XMLNode`, `XMLNodeType`, `XMLSocket`, `XMLUI`), NameBuiltin, nil},
-			{Words(``, `\b`, `decodeURI`, `decodeURIComponent`, `encodeURI`, `escape`, `eval`, `isFinite`, `isNaN`, `isXMLName`, `clearInterval`, `fscommand`, `getTimer`, `getURL`, `getVersion`, `parseFloat`, `parseInt`, `setInterval`, `trace`, `updateAfterEvent`, `unescape`), NameFunction, nil},
-			{`[$a-zA-Z_]\w*`, NameOther, nil},
-			{`[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?`, LiteralNumberFloat, nil},
-			{`0x[0-9a-f]+`, LiteralNumberHex, nil},
-			{`[0-9]+`, LiteralNumberInteger, nil},
-			{`"(\\\\|\\"|[^"])*"`, LiteralStringDouble, nil},
-			{`'(\\\\|\\'|[^'])*'`, LiteralStringSingle, nil},
-		},
-	}
-}

lexers/a/actionscript3.go

@@ -1,60 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Actionscript 3 lexer.
-var Actionscript3 = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "ActionScript 3",
-		Aliases:   []string{"as3", "actionscript3"},
-		Filenames: []string{"*.as"},
-		MimeTypes: []string{"application/x-actionscript3", "text/x-actionscript3", "text/actionscript3"},
-		DotAll:    true,
-	},
-	actionscript3Rules,
-))
-
-func actionscript3Rules() Rules {
-	return Rules{
-		"root": {
-			{`\s+`, Text, nil},
-			{`(function\s+)([$a-zA-Z_]\w*)(\s*)(\()`, ByGroups(KeywordDeclaration, NameFunction, Text, Operator), Push("funcparams")},
-			{`(var|const)(\s+)([$a-zA-Z_]\w*)(\s*)(:)(\s*)([$a-zA-Z_]\w*(?:\.<\w+>)?)`, ByGroups(KeywordDeclaration, Text, Name, Text, Punctuation, Text, KeywordType), nil},
-			{`(import|package)(\s+)((?:[$a-zA-Z_]\w*|\.)+)(\s*)`, ByGroups(Keyword, Text, NameNamespace, Text), nil},
-			{`(new)(\s+)([$a-zA-Z_]\w*(?:\.<\w+>)?)(\s*)(\()`, ByGroups(Keyword, Text, KeywordType, Text, Operator), nil},
-			{`//.*?\n`, CommentSingle, nil},
-			{`/\*.*?\*/`, CommentMultiline, nil},
-			{`/(\\\\|\\/|[^\n])*/[gisx]*`, LiteralStringRegex, nil},
-			{`(\.)([$a-zA-Z_]\w*)`, ByGroups(Operator, NameAttribute), nil},
-			{`(case|default|for|each|in|while|do|break|return|continue|if|else|throw|try|catch|with|new|typeof|arguments|instanceof|this|switch|import|include|as|is)\b`, Keyword, nil},
-			{`(class|public|final|internal|native|override|private|protected|static|import|extends|implements|interface|intrinsic|return|super|dynamic|function|const|get|namespace|package|set)\b`, KeywordDeclaration, nil},
-			{`(true|false|null|NaN|Infinity|-Infinity|undefined|void)\b`, KeywordConstant, nil},
-			{`(decodeURI|decodeURIComponent|encodeURI|escape|eval|isFinite|isNaN|isXMLName|clearInterval|fscommand|getTimer|getURL|getVersion|isFinite|parseFloat|parseInt|setInterval|trace|updateAfterEvent|unescape)\b`, NameFunction, nil},
-			{`[$a-zA-Z_]\w*`, Name, nil},
-			{`[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?`, LiteralNumberFloat, nil},
-			{`0x[0-9a-f]+`, LiteralNumberHex, nil},
-			{`[0-9]+`, LiteralNumberInteger, nil},
-			{`"(\\\\|\\"|[^"])*"`, LiteralStringDouble, nil},
-			{`'(\\\\|\\'|[^'])*'`, LiteralStringSingle, nil},
-			{`[~^*!%&<>|+=:;,/?\\{}\[\]().-]+`, Operator, nil},
-		},
-		"funcparams": {
-			{`\s+`, Text, nil},
-			{`(\s*)(\.\.\.)?([$a-zA-Z_]\w*)(\s*)(:)(\s*)([$a-zA-Z_]\w*(?:\.<\w+>)?|\*)(\s*)`, ByGroups(Text, Punctuation, Name, Text, Operator, Text, KeywordType, Text), Push("defval")},
-			{`\)`, Operator, Push("type")},
-		},
-		"type": {
-			{`(\s*)(:)(\s*)([$a-zA-Z_]\w*(?:\.<\w+>)?|\*)`, ByGroups(Text, Operator, Text, KeywordType), Pop(2)},
-			{`\s+`, Text, Pop(2)},
-			Default(Pop(2)),
-		},
-		"defval": {
-			{`(=)(\s*)([^(),]+)(\s*)(,?)`, ByGroups(Operator, Text, UsingSelf("root"), Text, Operator), Pop(1)},
-			{`,`, Operator, Pop(1)},
-			Default(Pop(1)),
-		},
-	}
-}

lexers/a/ada.go

@@ -1,118 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Ada lexer.
-var Ada = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "Ada",
-		Aliases:         []string{"ada", "ada95", "ada2005"},
-		Filenames:       []string{"*.adb", "*.ads", "*.ada"},
-		MimeTypes:       []string{"text/x-ada"},
-		CaseInsensitive: true,
-	},
-	adaRules,
-))
-
-func adaRules() Rules {
-	return Rules{
-		"root": {
-			{`[^\S\n]+`, Text, nil},
-			{`--.*?\n`, CommentSingle, nil},
-			{`[^\S\n]+`, Text, nil},
-			{`function|procedure|entry`, KeywordDeclaration, Push("subprogram")},
-			{`(subtype|type)(\s+)(\w+)`, ByGroups(KeywordDeclaration, Text, KeywordType), Push("type_def")},
-			{`task|protected`, KeywordDeclaration, nil},
-			{`(subtype)(\s+)`, ByGroups(KeywordDeclaration, Text), nil},
-			{`(end)(\s+)`, ByGroups(KeywordReserved, Text), Push("end")},
-			{`(pragma)(\s+)(\w+)`, ByGroups(KeywordReserved, Text, CommentPreproc), nil},
-			{`(true|false|null)\b`, KeywordConstant, nil},
-			{Words(``, `\b`, `Address`, `Byte`, `Boolean`, `Character`, `Controlled`, `Count`, `Cursor`, `Duration`, `File_Mode`, `File_Type`, `Float`, `Generator`, `Integer`, `Long_Float`, `Long_Integer`, `Long_Long_Float`, `Long_Long_Integer`, `Natural`, `Positive`, `Reference_Type`, `Short_Float`, `Short_Integer`, `Short_Short_Float`, `Short_Short_Integer`, `String`, `Wide_Character`, `Wide_String`), KeywordType, nil},
-			{`(and(\s+then)?|in|mod|not|or(\s+else)|rem)\b`, OperatorWord, nil},
-			{`generic|private`, KeywordDeclaration, nil},
-			{`package`, KeywordDeclaration, Push("package")},
-			{`array\b`, KeywordReserved, Push("array_def")},
-			{`(with|use)(\s+)`, ByGroups(KeywordNamespace, Text), Push("import")},
-			{`(\w+)(\s*)(:)(\s*)(constant)`, ByGroups(NameConstant, Text, Punctuation, Text, KeywordReserved), nil},
-			{`<<\w+>>`, NameLabel, nil},
-			{`(\w+)(\s*)(:)(\s*)(declare|begin|loop|for|while)`, ByGroups(NameLabel, Text, Punctuation, Text, KeywordReserved), nil},
-			{Words(`\b`, `\b`, `abort`, `abs`, `abstract`, `accept`, `access`, `aliased`, `all`, `array`, `at`, `begin`, `body`, `case`, `constant`, `declare`, `delay`, `delta`, `digits`, `do`, `else`, `elsif`, `end`, `entry`, `exception`, `exit`, `interface`, `for`, `goto`, `if`, `is`, `limited`, `loop`, `new`, `null`, `of`, `or`, `others`, `out`, `overriding`, `pragma`, `protected`, `raise`, `range`, `record`, `renames`, `requeue`, `return`, `reverse`, `select`, `separate`, `subtype`, `synchronized`, `task`, `tagged`, `terminate`, `then`, `type`, `until`, `when`, `while`, `xor`), KeywordReserved, nil},
-			{`"[^"]*"`, LiteralString, nil},
-			Include("attribute"),
-			Include("numbers"),
-			{`'[^']'`, LiteralStringChar, nil},
-			{`(\w+)(\s*|[(,])`, ByGroups(Name, UsingSelf("root")), nil},
-			{`(<>|=>|:=|[()|:;,.'])`, Punctuation, nil},
-			{`[*<>+=/&-]`, Operator, nil},
-			{`\n+`, Text, nil},
-		},
-		"numbers": {
-			{`[0-9_]+#[0-9a-f]+#`, LiteralNumberHex, nil},
-			{`[0-9_]+\.[0-9_]*`, LiteralNumberFloat, nil},
-			{`[0-9_]+`, LiteralNumberInteger, nil},
-		},
-		"attribute": {
-			{`(')(\w+)`, ByGroups(Punctuation, NameAttribute), nil},
-		},
-		"subprogram": {
-			{`\(`, Punctuation, Push("#pop", "formal_part")},
-			{`;`, Punctuation, Pop(1)},
-			{`is\b`, KeywordReserved, Pop(1)},
-			{`"[^"]+"|\w+`, NameFunction, nil},
-			Include("root"),
-		},
-		"end": {
-			{`(if|case|record|loop|select)`, KeywordReserved, nil},
-			{`"[^"]+"|[\w.]+`, NameFunction, nil},
-			{`\s+`, Text, nil},
-			{`;`, Punctuation, Pop(1)},
-		},
-		"type_def": {
-			{`;`, Punctuation, Pop(1)},
-			{`\(`, Punctuation, Push("formal_part")},
-			{`with|and|use`, KeywordReserved, nil},
-			{`array\b`, KeywordReserved, Push("#pop", "array_def")},
-			{`record\b`, KeywordReserved, Push("record_def")},
-			{`(null record)(;)`, ByGroups(KeywordReserved, Punctuation), Pop(1)},
-			Include("root"),
-		},
-		"array_def": {
-			{`;`, Punctuation, Pop(1)},
-			{`(\w+)(\s+)(range)`, ByGroups(KeywordType, Text, KeywordReserved), nil},
-			Include("root"),
-		},
-		"record_def": {
-			{`end record`, KeywordReserved, Pop(1)},
-			Include("root"),
-		},
-		"import": {
-			{`[\w.]+`, NameNamespace, Pop(1)},
-			Default(Pop(1)),
-		},
-		"formal_part": {
-			{`\)`, Punctuation, Pop(1)},
-			{`\w+`, NameVariable, nil},
-			{`,|:[^=]`, Punctuation, nil},
-			{`(in|not|null|out|access)\b`, KeywordReserved, nil},
-			Include("root"),
-		},
-		"package": {
-			{`body`, KeywordDeclaration, nil},
-			{`is\s+new|renames`, KeywordReserved, nil},
-			{`is`, KeywordReserved, Pop(1)},
-			{`;`, Punctuation, Pop(1)},
-			{`\(`, Punctuation, Push("package_instantiation")},
-			{`([\w.]+)`, NameClass, nil},
-			Include("root"),
-		},
-		"package_instantiation": {
-			{`("[^"]+"|\w+)(\s+)(=>)`, ByGroups(NameVariable, Text, Punctuation), nil},
-			{`[\w.\'"]`, Text, nil},
-			{`\)`, Punctuation, Pop(1)},
-			Include("root"),
-		},
-	}
-}

lexers/a/al.go

@@ -1,47 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Al lexer.
-var Al = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "AL",
-		Aliases:         []string{"al"},
-		Filenames:       []string{"*.al", "*.dal"},
-		MimeTypes:       []string{"text/x-al"},
-		DotAll:          true,
-		CaseInsensitive: true,
-	},
-	alRules,
-))
-
-// https://github.com/microsoft/AL/blob/master/grammar/alsyntax.tmlanguage
-func alRules() Rules {
-	return Rules{
-		"root": {
-			{`\s+`, TextWhitespace, nil},
-			{`(?s)\/\*.*?\\*\*\/`, CommentMultiline, nil},
-			{`(?s)//.*?\n`, CommentSingle, nil},
-			{`\"([^\"])*\"`, Text, nil},
-			{`'([^'])*'`, LiteralString, nil},
-			{`\b(?i:(ARRAY|ASSERTERROR|BEGIN|BREAK|CASE|DO|DOWNTO|ELSE|END|EVENT|EXIT|FOR|FOREACH|FUNCTION|IF|IMPLEMENTS|IN|INDATASET|INTERFACE|INTERNAL|LOCAL|OF|PROCEDURE|PROGRAM|PROTECTED|REPEAT|RUNONCLIENT|SECURITYFILTERING|SUPPRESSDISPOSE|TEMPORARY|THEN|TO|TRIGGER|UNTIL|VAR|WHILE|WITH|WITHEVENTS))\b`, Keyword, nil},
-			{`\b(?i:(AND|DIV|MOD|NOT|OR|XOR))\b`, OperatorWord, nil},
-			{`\b(?i:(AVERAGE|CONST|COUNT|EXIST|FIELD|FILTER|LOOKUP|MAX|MIN|ORDER|SORTING|SUM|TABLEDATA|UPPERLIMIT|WHERE|ASCENDING|DESCENDING))\b`, Keyword, nil},
-			{`\b(?i:(CODEUNIT|PAGE|PAGEEXTENSION|PAGECUSTOMIZATION|DOTNET|ENUM|ENUMEXTENSION|VALUE|QUERY|REPORT|TABLE|TABLEEXTENSION|XMLPORT|PROFILE|CONTROLADDIN|REPORTEXTENSION|INTERFACE|PERMISSIONSET|PERMISSIONSETEXTENSION|ENTITLEMENT))\b`, Keyword, nil},
-			{`\b(?i:(Action|Array|Automation|BigInteger|BigText|Blob|Boolean|Byte|Char|ClientType|Code|Codeunit|CompletionTriggerErrorLevel|ConnectionType|Database|DataClassification|DataScope|Date|DateFormula|DateTime|Decimal|DefaultLayout|Dialog|Dictionary|DotNet|DotNetAssembly|DotNetTypeDeclaration|Duration|Enum|ErrorInfo|ErrorType|ExecutionContext|ExecutionMode|FieldClass|FieldRef|FieldType|File|FilterPageBuilder|Guid|InStream|Integer|Joker|KeyRef|List|ModuleDependencyInfo|ModuleInfo|None|Notification|NotificationScope|ObjectType|Option|OutStream|Page|PageResult|Query|Record|RecordId|RecordRef|Report|ReportFormat|SecurityFilter|SecurityFiltering|Table|TableConnectionType|TableFilter|TestAction|TestField|TestFilterField|TestPage|TestPermissions|TestRequestPage|Text|TextBuilder|TextConst|TextEncoding|Time|TransactionModel|TransactionType|Variant|Verbosity|Version|XmlPort|HttpContent|HttpHeaders|HttpClient|HttpRequestMessage|HttpResponseMessage|JsonToken|JsonValue|JsonArray|JsonObject|View|Views|XmlAttribute|XmlAttributeCollection|XmlComment|XmlCData|XmlDeclaration|XmlDocument|XmlDocumentType|XmlElement|XmlNamespaceManager|XmlNameTable|XmlNode|XmlNodeList|XmlProcessingInstruction|XmlReadOptions|XmlText|XmlWriteOptions|WebServiceActionContext|WebServiceActionResultCode|SessionSettings))\b`, Keyword, nil},
-			{`\b([<>]=|<>|<|>)\b?`, Operator, nil},
-			{`\b(\-|\+|\/|\*)\b`, Operator, nil},
-			{`\s*(\:=|\+=|-=|\/=|\*=)\s*?`, Operator, nil},
-			{`\b(?i:(ADD|ADDFIRST|ADDLAST|ADDAFTER|ADDBEFORE|ACTION|ACTIONS|AREA|ASSEMBLY|CHARTPART|CUEGROUP|CUSTOMIZES|COLUMN|DATAITEM|DATASET|ELEMENTS|EXTENDS|FIELD|FIELDGROUP|FIELDATTRIBUTE|FIELDELEMENT|FIELDGROUPS|FIELDS|FILTER|FIXED|GRID|GROUP|MOVEAFTER|MOVEBEFORE|KEY|KEYS|LABEL|LABELS|LAYOUT|MODIFY|MOVEFIRST|MOVELAST|MOVEBEFORE|MOVEAFTER|PART|REPEATER|USERCONTROL|REQUESTPAGE|SCHEMA|SEPARATOR|SYSTEMPART|TABLEELEMENT|TEXTATTRIBUTE|TEXTELEMENT|TYPE))\b`, Keyword, nil},
-			{`\s*[(\.\.)&\|]\s*`, Operator, nil},
-			{`\b((0(x|X)[0-9a-fA-F]*)|(([0-9]+\.?[0-9]*)|(\.[0-9]+))((e|E)(\+|-)?[0-9]+)?)(L|l|UL|ul|u|U|F|f|ll|LL|ull|ULL)?\b`, LiteralNumber, nil},
-			{`[;:,]`, Punctuation, nil},
-			{`#[ \t]*(if|else|elif|endif|define|undef|region|endregion|pragma)\b.*?\n`, CommentPreproc, nil},
-			{`\w+`, Text, nil},
-			{`.`, Text, nil},
-		},
-	}
-}

lexers/a/angular2.go

@@ -1,46 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Angular2 lexer.
-var Angular2 = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Angular2",
-		Aliases:   []string{"ng2"},
-		Filenames: []string{},
-		MimeTypes: []string{},
-	},
-	angular2Rules,
-))
-
-func angular2Rules() Rules {
-	return Rules{
-		"root": {
-			{`[^{([*#]+`, Other, nil},
-			{`(\{\{)(\s*)`, ByGroups(CommentPreproc, Text), Push("ngExpression")},
-			{`([([]+)([\w:.-]+)([\])]+)(\s*)(=)(\s*)`, ByGroups(Punctuation, NameAttribute, Punctuation, Text, Operator, Text), Push("attr")},
-			{`([([]+)([\w:.-]+)([\])]+)(\s*)`, ByGroups(Punctuation, NameAttribute, Punctuation, Text), nil},
-			{`([*#])([\w:.-]+)(\s*)(=)(\s*)`, ByGroups(Punctuation, NameAttribute, Punctuation, Operator), Push("attr")},
-			{`([*#])([\w:.-]+)(\s*)`, ByGroups(Punctuation, NameAttribute, Punctuation), nil},
-		},
-		"ngExpression": {
-			{`\s+(\|\s+)?`, Text, nil},
-			{`\}\}`, CommentPreproc, Pop(1)},
-			{`:?(true|false)`, LiteralStringBoolean, nil},
-			{`:?"(\\\\|\\"|[^"])*"`, LiteralStringDouble, nil},
-			{`:?'(\\\\|\\'|[^'])*'`, LiteralStringSingle, nil},
-			{`[0-9](\.[0-9]*)?(eE[+-][0-9])?[flFLdD]?|0[xX][0-9a-fA-F]+[Ll]?`, LiteralNumber, nil},
-			{`[a-zA-Z][\w-]*(\(.*\))?`, NameVariable, nil},
-			{`\.[\w-]+(\(.*\))?`, NameVariable, nil},
-			{`(\?)(\s*)([^}\s]+)(\s*)(:)(\s*)([^}\s]+)(\s*)`, ByGroups(Operator, Text, LiteralString, Text, Operator, Text, LiteralString, Text), nil},
-		},
-		"attr": {
-			{`".*?"`, LiteralString, Pop(1)},
-			{`'.*?'`, LiteralString, Pop(1)},
-			{`[^\s>]+`, LiteralString, Pop(1)},
-		},
-	}
-}

lexers/a/antlr.go

@@ -1,105 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// ANTLR lexer.
-var ANTLR = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "ANTLR",
-		Aliases:   []string{"antlr"},
-		Filenames: []string{},
-		MimeTypes: []string{},
-	},
-	antlrRules,
-))
-
-func antlrRules() Rules {
-	return Rules{
-		"whitespace": {
-			{`\s+`, TextWhitespace, nil},
-		},
-		"comments": {
-			{`//.*$`, Comment, nil},
-			{`/\*(.|\n)*?\*/`, Comment, nil},
-		},
-		"root": {
-			Include("whitespace"),
-			Include("comments"),
-			{`(lexer|parser|tree)?(\s*)(grammar\b)(\s*)([A-Za-z]\w*)(;)`, ByGroups(Keyword, TextWhitespace, Keyword, TextWhitespace, NameClass, Punctuation), nil},
-			{`options\b`, Keyword, Push("options")},
-			{`tokens\b`, Keyword, Push("tokens")},
-			{`(scope)(\s*)([A-Za-z]\w*)(\s*)(\{)`, ByGroups(Keyword, TextWhitespace, NameVariable, TextWhitespace, Punctuation), Push("action")},
-			{`(catch|finally)\b`, Keyword, Push("exception")},
-			{`(@[A-Za-z]\w*)(\s*)(::)?(\s*)([A-Za-z]\w*)(\s*)(\{)`, ByGroups(NameLabel, TextWhitespace, Punctuation, TextWhitespace, NameLabel, TextWhitespace, Punctuation), Push("action")},
-			{`((?:protected|private|public|fragment)\b)?(\s*)([A-Za-z]\w*)(!)?`, ByGroups(Keyword, TextWhitespace, NameLabel, Punctuation), Push("rule-alts", "rule-prelims")},
-		},
-		"exception": {
-			{`\n`, TextWhitespace, Pop(1)},
-			{`\s`, TextWhitespace, nil},
-			Include("comments"),
-			{`\[`, Punctuation, Push("nested-arg-action")},
-			{`\{`, Punctuation, Push("action")},
-		},
-		"rule-prelims": {
-			Include("whitespace"),
-			Include("comments"),
-			{`returns\b`, Keyword, nil},
-			{`\[`, Punctuation, Push("nested-arg-action")},
-			{`\{`, Punctuation, Push("action")},
-			{`(throws)(\s+)([A-Za-z]\w*)`, ByGroups(Keyword, TextWhitespace, NameLabel), nil},
-			{`(,)(\s*)([A-Za-z]\w*)`, ByGroups(Punctuation, TextWhitespace, NameLabel), nil},
-			{`options\b`, Keyword, Push("options")},
-			{`(scope)(\s+)(\{)`, ByGroups(Keyword, TextWhitespace, Punctuation), Push("action")},
-			{`(scope)(\s+)([A-Za-z]\w*)(\s*)(;)`, ByGroups(Keyword, TextWhitespace, NameLabel, TextWhitespace, Punctuation), nil},
-			{`(@[A-Za-z]\w*)(\s*)(\{)`, ByGroups(NameLabel, TextWhitespace, Punctuation), Push("action")},
-			{`:`, Punctuation, Pop(1)},
-		},
-		"rule-alts": {
-			Include("whitespace"),
-			Include("comments"),
-			{`options\b`, Keyword, Push("options")},
-			{`:`, Punctuation, nil},
-			{`'(\\\\|\\'|[^'])*'`, LiteralString, nil},
-			{`"(\\\\|\\"|[^"])*"`, LiteralString, nil},
-			{`<<([^>]|>[^>])>>`, LiteralString, nil},
-			{`\$?[A-Z_]\w*`, NameConstant, nil},
-			{`\$?[a-z_]\w*`, NameVariable, nil},
-			{`(\+|\||->|=>|=|\(|\)|\.\.|\.|\?|\*|\^|!|\#|~)`, Operator, nil},
-			{`,`, Punctuation, nil},
-			{`\[`, Punctuation, Push("nested-arg-action")},
-			{`\{`, Punctuation, Push("action")},
-			{`;`, Punctuation, Pop(1)},
-		},
-		"tokens": {
-			Include("whitespace"),
-			Include("comments"),
-			{`\{`, Punctuation, nil},
-			{`([A-Z]\w*)(\s*)(=)?(\s*)(\'(?:\\\\|\\\'|[^\']*)\')?(\s*)(;)`, ByGroups(NameLabel, TextWhitespace, Punctuation, TextWhitespace, LiteralString, TextWhitespace, Punctuation), nil},
-			{`\}`, Punctuation, Pop(1)},
-		},
-		"options": {
-			Include("whitespace"),
-			Include("comments"),
-			{`\{`, Punctuation, nil},
-			{`([A-Za-z]\w*)(\s*)(=)(\s*)([A-Za-z]\w*|\'(?:\\\\|\\\'|[^\']*)\'|[0-9]+|\*)(\s*)(;)`, ByGroups(NameVariable, TextWhitespace, Punctuation, TextWhitespace, Text, TextWhitespace, Punctuation), nil},
-			{`\}`, Punctuation, Pop(1)},
-		},
-		"action": {
-			{`([^${}\'"/\\]+|"(\\\\|\\"|[^"])*"|'(\\\\|\\'|[^'])*'|//.*$\n?|/\*(.|\n)*?\*/|/(?!\*)(\\\\|\\/|[^/])*/|\\(?!%)|/)+`, Other, nil},
-			{`(\\)(%)`, ByGroups(Punctuation, Other), nil},
-			{`(\$[a-zA-Z]+)(\.?)(text|value)?`, ByGroups(NameVariable, Punctuation, NameProperty), nil},
-			{`\{`, Punctuation, Push()},
-			{`\}`, Punctuation, Pop(1)},
-		},
-		"nested-arg-action": {
-			{`([^$\[\]\'"/]+|"(\\\\|\\"|[^"])*"|'(\\\\|\\'|[^'])*'|//.*$\n?|/\*(.|\n)*?\*/|/(?!\*)(\\\\|\\/|[^/])*/|/)+`, Other, nil},
-			{`\[`, Punctuation, Push()},
-			{`\]`, Punctuation, Pop(1)},
-			{`(\$[a-zA-Z]+)(\.?)(text|value)?`, ByGroups(NameVariable, Punctuation, NameProperty), nil},
-			{`(\\\\|\\\]|\\\[|[^\[\]])+`, Other, nil},
-		},
-	}
-}

lexers/a/apache.go

@@ -1,42 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Apacheconf lexer.
-var Apacheconf = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "ApacheConf",
-		Aliases:         []string{"apacheconf", "aconf", "apache"},
-		Filenames:       []string{".htaccess", "apache.conf", "apache2.conf"},
-		MimeTypes:       []string{"text/x-apacheconf"},
-		CaseInsensitive: true,
-	},
-	apacheconfRules,
-))
-
-func apacheconfRules() Rules {
-	return Rules{
-		"root": {
-			{`\s+`, Text, nil},
-			{`(#.*?)$`, Comment, nil},
-			{`(<[^\s>]+)(?:(\s+)(.*?))?(>)`, ByGroups(NameTag, Text, LiteralString, NameTag), nil},
-			{`([a-z]\w*)(\s+)`, ByGroups(NameBuiltin, Text), Push("value")},
-			{`\.+`, Text, nil},
-		},
-		"value": {
-			{`\\\n`, Text, nil},
-			{`$`, Text, Pop(1)},
-			{`\\`, Text, nil},
-			{`[^\S\n]+`, Text, nil},
-			{`\d+\.\d+\.\d+\.\d+(?:/\d+)?`, LiteralNumber, nil},
-			{`\d+`, LiteralNumber, nil},
-			{`/([a-z0-9][\w./-]+)`, LiteralStringOther, nil},
-			{`(on|off|none|any|all|double|email|dns|min|minimal|os|productonly|full|emerg|alert|crit|error|warn|notice|info|debug|registry|script|inetd|standalone|user|group)\b`, Keyword, nil},
-			{`"([^"\\]*(?:\\.[^"\\]*)*)"`, LiteralStringDouble, nil},
-			{`[^\s"\\]+`, Text, nil},
-		},
-	}
-}

lexers/a/apl.go

@@ -1,40 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Apl lexer.
-var Apl = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "APL",
-		Aliases:   []string{"apl"},
-		Filenames: []string{"*.apl"},
-		MimeTypes: []string{},
-	},
-	aplRules,
-))
-
-func aplRules() Rules {
-	return Rules{
-		"root": {
-			{`\s+`, Text, nil},
-			{`[⍝#].*$`, CommentSingle, nil},
-			{`\'((\'\')|[^\'])*\'`, LiteralStringSingle, nil},
-			{`"(("")|[^"])*"`, LiteralStringDouble, nil},
-			{`[⋄◇()]`, Punctuation, nil},
-			{`[\[\];]`, LiteralStringRegex, nil},
-			{`⎕[A-Za-zΔ∆⍙][A-Za-zΔ∆⍙_¯0-9]*`, NameFunction, nil},
-			{`[A-Za-zΔ∆⍙_][A-Za-zΔ∆⍙_¯0-9]*`, NameVariable, nil},
-			{`¯?(0[Xx][0-9A-Fa-f]+|[0-9]*\.?[0-9]+([Ee][+¯]?[0-9]+)?|¯|∞)([Jj]¯?(0[Xx][0-9A-Fa-f]+|[0-9]*\.?[0-9]+([Ee][+¯]?[0-9]+)?|¯|∞))?`, LiteralNumber, nil},
-			{`[\.\\/⌿⍀¨⍣⍨⍠⍤∘⍥@⌺⌶⍢]`, NameAttribute, nil},
-			{`[+\-×÷⌈⌊∣|⍳?*⍟○!⌹<≤=>≥≠≡≢∊⍷∪∩~∨∧⍱⍲⍴,⍪⌽⊖⍉↑↓⊂⊃⌷⍋⍒⊤⊥⍕⍎⊣⊢⍁⍂≈⌸⍯↗⊆⍸]`, Operator, nil},
-			{`⍬`, NameConstant, nil},
-			{`[⎕⍞]`, NameVariableGlobal, nil},
-			{`[←→]`, KeywordDeclaration, nil},
-			{`[⍺⍵⍶⍹∇:]`, NameBuiltinPseudo, nil},
-			{`[{}]`, KeywordType, nil},
-		},
-	}
-}

lexers/a/applescript.go

@@ -1,59 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Applescript lexer.
-var Applescript = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "AppleScript",
-		Aliases:   []string{"applescript"},
-		Filenames: []string{"*.applescript"},
-		MimeTypes: []string{},
-		DotAll:    true,
-	},
-	applescriptRules,
-))
-
-func applescriptRules() Rules {
-	return Rules{
-		"root": {
-			{`\s+`, Text, nil},
-			{`¬\n`, LiteralStringEscape, nil},
-			{`'s\s+`, Text, nil},
-			{`(--|#).*?$`, Comment, nil},
-			{`\(\*`, CommentMultiline, Push("comment")},
-			{`[(){}!,.:]`, Punctuation, nil},
-			{`(«)([^»]+)(»)`, ByGroups(Text, NameBuiltin, Text), nil},
-			{`\b((?:considering|ignoring)\s*)(application responses|case|diacriticals|hyphens|numeric strings|punctuation|white space)`, ByGroups(Keyword, NameBuiltin), nil},
-			{`(-|\*|\+|&|≠|>=?|<=?|=|≥|≤|/|÷|\^)`, Operator, nil},
-			{`\b(and|or|is equal|equals|(is )?equal to|is not|isn't|isn't equal( to)?|is not equal( to)?|doesn't equal|does not equal|(is )?greater than|comes after|is not less than or equal( to)?|isn't less than or equal( to)?|(is )?less than|comes before|is not greater than or equal( to)?|isn't greater than or equal( to)?|(is  )?greater than or equal( to)?|is not less than|isn't less than|does not come before|doesn't come before|(is )?less than or equal( to)?|is not greater than|isn't greater than|does not come after|doesn't come after|starts? with|begins? with|ends? with|contains?|does not contain|doesn't contain|is in|is contained by|is not in|is not contained by|isn't contained by|div|mod|not|(a  )?(ref( to)?|reference to)|is|does)\b`, OperatorWord, nil},
-			{`^(\s*(?:on|end)\s+)(zoomed|write to file|will zoom|will show|will select tab view item|will resize( sub views)?|will resign active|will quit|will pop up|will open|will move|will miniaturize|will hide|will finish launching|will display outline cell|will display item cell|will display cell|will display browser cell|will dismiss|will close|will become active|was miniaturized|was hidden|update toolbar item|update parameters|update menu item|shown|should zoom|should selection change|should select tab view item|should select row|should select item|should select column|should quit( after last window closed)?|should open( untitled)?|should expand item|should end editing|should collapse item|should close|should begin editing|selection changing|selection changed|selected tab view item|scroll wheel|rows changed|right mouse up|right mouse dragged|right mouse down|resized( sub views)?|resigned main|resigned key|resigned active|read from file|prepare table drop|prepare table drag|prepare outline drop|prepare outline drag|prepare drop|plugin loaded|parameters updated|panel ended|opened|open untitled|number of rows|number of items|number of browser rows|moved|mouse up|mouse moved|mouse exited|mouse entered|mouse dragged|mouse down|miniaturized|load data representation|launched|keyboard up|keyboard down|items changed|item value changed|item value|item expandable|idle|exposed|end editing|drop|drag( (entered|exited|updated))?|double clicked|document nib name|dialog ended|deminiaturized|data representation|conclude drop|column resized|column moved|column clicked|closed|clicked toolbar item|clicked|choose menu item|child of item|changed|change item value|change cell value|cell value changed|cell value|bounds changed|begin editing|became main|became key|awake from nib|alert ended|activated|action|accept table drop|accept outline drop)`, ByGroups(Keyword, NameFunction), nil},
-			{`^(\s*)(in|on|script|to)(\s+)`, ByGroups(Text, Keyword, Text), nil},
-			{`\b(as )(alias |application |boolean |class |constant |date |file |integer |list |number |POSIX file |real |record |reference |RGB color |script |text |unit types|(?:Unicode )?text|string)\b`, ByGroups(Keyword, NameClass), nil},
-			{`\b(AppleScript|current application|false|linefeed|missing value|pi|quote|result|return|space|tab|text item delimiters|true|version)\b`, NameConstant, nil},
-			{`\b(ASCII (character|number)|activate|beep|choose URL|choose application|choose color|choose file( name)?|choose folder|choose from list|choose remote application|clipboard info|close( access)?|copy|count|current date|delay|delete|display (alert|dialog)|do shell script|duplicate|exists|get eof|get volume settings|info for|launch|list (disks|folder)|load script|log|make|mount volume|new|offset|open( (for access|location))?|path to|print|quit|random number|read|round|run( script)?|say|scripting components|set (eof|the clipboard to|volume)|store script|summarize|system attribute|system info|the clipboard|time to GMT|write|quoted form)\b`, NameBuiltin, nil},
-			{`\b(considering|else|error|exit|from|if|ignoring|in|repeat|tell|then|times|to|try|until|using terms from|while|with|with timeout( of)?|with transaction|by|continue|end|its?|me|my|return|of|as)\b`, Keyword, nil},
-			{`\b(global|local|prop(erty)?|set|get)\b`, Keyword, nil},
-			{`\b(but|put|returning|the)\b`, NameBuiltin, nil},
-			{`\b(attachment|attribute run|character|day|month|paragraph|word|year)s?\b`, NameBuiltin, nil},
-			{`\b(about|above|against|apart from|around|aside from|at|below|beneath|beside|between|for|given|instead of|on|onto|out of|over|since)\b`, NameBuiltin, nil},
-			{`\b(accepts arrow key|action method|active|alignment|allowed identifiers|allows branch selection|allows column reordering|allows column resizing|allows column selection|allows customization|allows editing text attributes|allows empty selection|allows mixed state|allows multiple selection|allows reordering|allows undo|alpha( value)?|alternate image|alternate increment value|alternate title|animation delay|associated file name|associated object|auto completes|auto display|auto enables items|auto repeat|auto resizes( outline column)?|auto save expanded items|auto save name|auto save table columns|auto saves configuration|auto scroll|auto sizes all columns to fit|auto sizes cells|background color|bezel state|bezel style|bezeled|border rect|border type|bordered|bounds( rotation)?|box type|button returned|button type|can choose directories|can choose files|can draw|can hide|cell( (background color|size|type))?|characters|class|click count|clicked( data)? column|clicked data item|clicked( data)? row|closeable|collating|color( (mode|panel))|command key down|configuration|content(s| (size|view( margins)?))?|context|continuous|control key down|control size|control tint|control view|controller visible|coordinate system|copies( on scroll)?|corner view|current cell|current column|current( field)?  editor|current( menu)? item|current row|current tab view item|data source|default identifiers|delta (x|y|z)|destination window|directory|display mode|displayed cell|document( (edited|rect|view))?|double value|dragged column|dragged distance|dragged items|draws( cell)? background|draws grid|dynamically scrolls|echos bullets|edge|editable|edited( data)? column|edited data item|edited( data)? row|enabled|enclosing scroll view|ending page|error handling|event number|event type|excluded from windows menu|executable path|expanded|fax number|field editor|file kind|file name|file type|first responder|first visible column|flipped|floating|font( panel)?|formatter|frameworks path|frontmost|gave up|grid color|has data items|has horizontal ruler|has horizontal scroller|has parent data item|has resize indicator|has shadow|has sub menu|has vertical ruler|has vertical scroller|header cell|header view|hidden|hides when deactivated|highlights by|horizontal line scroll|horizontal page scroll|horizontal ruler view|horizontally resizable|icon image|id|identifier|ignores multiple clicks|image( (alignment|dims when disabled|frame style|scaling))?|imports graphics|increment value|indentation per level|indeterminate|index|integer value|intercell spacing|item height|key( (code|equivalent( modifier)?|window))?|knob thickness|label|last( visible)? column|leading offset|leaf|level|line scroll|loaded|localized sort|location|loop mode|main( (bunde|menu|window))?|marker follows cell|matrix mode|maximum( content)? size|maximum visible columns|menu( form representation)?|miniaturizable|miniaturized|minimized image|minimized title|minimum column width|minimum( content)? size|modal|modified|mouse down state|movie( (controller|file|rect))?|muted|name|needs display|next state|next text|number of tick marks|only tick mark values|opaque|open panel|option key down|outline table column|page scroll|pages across|pages down|palette label|pane splitter|parent data item|parent window|pasteboard|path( (names|separator))?|playing|plays every frame|plays selection only|position|preferred edge|preferred type|pressure|previous text|prompt|properties|prototype cell|pulls down|rate|released when closed|repeated|requested print time|required file type|resizable|resized column|resource path|returns records|reuses columns|rich text|roll over|row height|rulers visible|save panel|scripts path|scrollable|selectable( identifiers)?|selected cell|selected( data)? columns?|selected data items?|selected( data)? rows?|selected item identifier|selection by rect|send action on arrow key|sends action when done editing|separates columns|separator item|sequence number|services menu|shared frameworks path|shared support path|sheet|shift key down|shows alpha|shows state by|size( mode)?|smart insert delete enabled|sort case sensitivity|sort column|sort order|sort type|sorted( data rows)?|sound|source( mask)?|spell checking enabled|starting page|state|string value|sub menu|super menu|super view|tab key traverses cells|tab state|tab type|tab view|table view|tag|target( printer)?|text color|text container insert|text container origin|text returned|tick mark position|time stamp|title(d| (cell|font|height|position|rect))?|tool tip|toolbar|trailing offset|transparent|treat packages as directories|truncated labels|types|unmodified characters|update views|use sort indicator|user defaults|uses data source|uses ruler|uses threaded animation|uses title from previous column|value wraps|version|vertical( (line scroll|page scroll|ruler view))?|vertically resizable|view|visible( document rect)?|volume|width|window|windows menu|wraps|zoomable|zoomed)\b`, NameAttribute, nil},
-			{`\b(action cell|alert reply|application|box|browser( cell)?|bundle|button( cell)?|cell|clip view|color well|color-panel|combo box( item)?|control|data( (cell|column|item|row|source))?|default entry|dialog reply|document|drag info|drawer|event|font(-panel)?|formatter|image( (cell|view))?|matrix|menu( item)?|item|movie( view)?|open-panel|outline view|panel|pasteboard|plugin|popup button|progress indicator|responder|save-panel|scroll view|secure text field( cell)?|slider|sound|split view|stepper|tab view( item)?|table( (column|header cell|header view|view))|text( (field( cell)?|view))?|toolbar( item)?|user-defaults|view|window)s?\b`, NameBuiltin, nil},
-			{`\b(animate|append|call method|center|close drawer|close panel|display|display alert|display dialog|display panel|go|hide|highlight|increment|item for|load image|load movie|load nib|load panel|load sound|localized string|lock focus|log|open drawer|path for|pause|perform action|play|register|resume|scroll|select( all)?|show|size to fit|start|step back|step forward|stop|synchronize|unlock focus|update)\b`, NameBuiltin, nil},
-			{`\b((in )?back of|(in )?front of|[0-9]+(st|nd|rd|th)|first|second|third|fourth|fifth|sixth|seventh|eighth|ninth|tenth|after|back|before|behind|every|front|index|last|middle|some|that|through|thru|where|whose)\b`, NameBuiltin, nil},
-			{`"(\\\\|\\"|[^"])*"`, LiteralStringDouble, nil},
-			{`\b([a-zA-Z]\w*)\b`, NameVariable, nil},
-			{`[-+]?(\d+\.\d*|\d*\.\d+)(E[-+][0-9]+)?`, LiteralNumberFloat, nil},
-			{`[-+]?\d+`, LiteralNumberInteger, nil},
-		},
-		"comment": {
-			{`\(\*`, CommentMultiline, Push()},
-			{`\*\)`, CommentMultiline, Pop(1)},
-			{`[^*(]+`, CommentMultiline, nil},
-			{`[*(]`, CommentMultiline, nil},
-		},
-	}
-}

lexers/a/arduino.go

@@ -1,114 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Arduino lexer.
-var Arduino = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Arduino",
-		Aliases:   []string{"arduino"},
-		Filenames: []string{"*.ino"},
-		MimeTypes: []string{"text/x-arduino"},
-		EnsureNL:  true,
-	},
-	arduinoRules,
-))
-
-func arduinoRules() Rules {
-	return Rules{
-		"statements": {
-			{Words(``, `\b`, `catch`, `const_cast`, `delete`, `dynamic_cast`, `explicit`, `export`, `friend`, `mutable`, `namespace`, `new`, `operator`, `private`, `protected`, `public`, `reinterpret_cast`, `restrict`, `static_cast`, `template`, `this`, `throw`, `throws`, `try`, `typeid`, `typename`, `using`, `virtual`, `constexpr`, `nullptr`, `decltype`, `thread_local`, `alignas`, `alignof`, `static_assert`, `noexcept`, `override`, `final`), Keyword, nil},
-			{`char(16_t|32_t)\b`, KeywordType, nil},
-			{`(class)\b`, ByGroups(Keyword, Text), Push("classname")},
-			{`(R)(")([^\\()\s]{,16})(\()((?:.|\n)*?)(\)\3)(")`, ByGroups(LiteralStringAffix, LiteralString, LiteralStringDelimiter, LiteralStringDelimiter, LiteralString, LiteralStringDelimiter, LiteralString), nil},
-			{`(u8|u|U)(")`, ByGroups(LiteralStringAffix, LiteralString), Push("string")},
-			{`(L?)(")`, ByGroups(LiteralStringAffix, LiteralString), Push("string")},
-			{`(L?)(')(\\.|\\[0-7]{1,3}|\\x[a-fA-F0-9]{1,2}|[^\\\'\n])(')`, ByGroups(LiteralStringAffix, LiteralStringChar, LiteralStringChar, LiteralStringChar), nil},
-			{`(\d+\.\d*|\.\d+|\d+)[eE][+-]?\d+[LlUu]*`, LiteralNumberFloat, nil},
-			{`(\d+\.\d*|\.\d+|\d+[fF])[fF]?`, LiteralNumberFloat, nil},
-			{`0x[0-9a-fA-F]+[LlUu]*`, LiteralNumberHex, nil},
-			{`0[0-7]+[LlUu]*`, LiteralNumberOct, nil},
-			{`\d+[LlUu]*`, LiteralNumberInteger, nil},
-			{`\*/`, Error, nil},
-			{`[~!%^&*+=|?:<>/-]`, Operator, nil},
-			{`[()\[\],.]`, Punctuation, nil},
-			{Words(``, `\b`, `asm`, `auto`, `break`, `case`, `const`, `continue`, `default`, `do`, `else`, `enum`, `extern`, `for`, `goto`, `if`, `register`, `restricted`, `return`, `sizeof`, `static`, `struct`, `switch`, `typedef`, `union`, `volatile`, `while`), Keyword, nil},
-			{`(_Bool|_Complex|_Imaginary|array|atomic_bool|atomic_char|atomic_int|atomic_llong|atomic_long|atomic_schar|atomic_short|atomic_uchar|atomic_uint|atomic_ullong|atomic_ulong|atomic_ushort|auto|bool|boolean|BooleanVariables|Byte|byte|Char|char|char16_t|char32_t|class|complex|Const|const|const_cast|delete|double|dynamic_cast|enum|explicit|extern|Float|float|friend|inline|Int|int|int16_t|int32_t|int64_t|int8_t|Long|long|new|NULL|null|operator|private|PROGMEM|protected|public|register|reinterpret_cast|short|signed|sizeof|Static|static|static_cast|String|struct|typedef|uint16_t|uint32_t|uint64_t|uint8_t|union|unsigned|virtual|Void|void|Volatile|volatile|word)\b`, KeywordType, nil},
-			// Start of: Arduino-specific syntax
-			{`(and|final|If|Loop|loop|not|or|override|setup|Setup|throw|try|xor)\b`, Keyword, nil}, // Addition to keywords already defined by C++
-			{`(ANALOG_MESSAGE|BIN|CHANGE|DEC|DEFAULT|DIGITAL_MESSAGE|EXTERNAL|FALLING|FIRMATA_STRING|HALF_PI|HEX|HIGH|INPUT|INPUT_PULLUP|INTERNAL|INTERNAL1V1|INTERNAL1V1|INTERNAL2V56|INTERNAL2V56|LED_BUILTIN|LED_BUILTIN_RX|LED_BUILTIN_TX|LOW|LSBFIRST|MSBFIRST|OCT|OUTPUT|PI|REPORT_ANALOG|REPORT_DIGITAL|RISING|SET_PIN_MODE|SYSEX_START|SYSTEM_RESET|TWO_PI)\b`, KeywordConstant, nil},
-			{`(boolean|const|byte|word|string|String|array)\b`, NameVariable, nil},
-			{`(Keyboard|KeyboardController|MouseController|SoftwareSerial|EthernetServer|EthernetClient|LiquidCrystal|RobotControl|GSMVoiceCall|EthernetUDP|EsploraTFT|HttpClient|RobotMotor|WiFiClient|GSMScanner|FileSystem|Scheduler|GSMServer|YunClient|YunServer|IPAddress|GSMClient|GSMModem|Keyboard|Ethernet|Console|GSMBand|Esplora|Stepper|Process|WiFiUDP|GSM_SMS|Mailbox|USBHost|Firmata|PImage|Client|Server|GSMPIN|FileIO|Bridge|Serial|EEPROM|Stream|Mouse|Audio|Servo|File|Task|GPRS|WiFi|Wire|TFT|GSM|SPI|SD)\b`, NameClass, nil},
-			{`(abs|Abs|accept|ACos|acos|acosf|addParameter|analogRead|AnalogRead|analogReadResolution|AnalogReadResolution|analogReference|AnalogReference|analogWrite|AnalogWrite|analogWriteResolution|AnalogWriteResolution|answerCall|asin|ASin|asinf|atan|ATan|atan2|ATan2|atan2f|atanf|attach|attached|attachGPRS|attachInterrupt|AttachInterrupt|autoscroll|available|availableForWrite|background|beep|begin|beginPacket|beginSD|beginSMS|beginSpeaker|beginTFT|beginTransmission|beginWrite|bit|Bit|BitClear|bitClear|bitRead|BitRead|bitSet|BitSet|BitWrite|bitWrite|blink|blinkVersion|BSSID|buffer|byte|cbrt|cbrtf|Ceil|ceil|ceilf|changePIN|char|charAt|checkPIN|checkPUK|checkReg|circle|cityNameRead|cityNameWrite|clear|clearScreen|click|close|compareTo|compassRead|concat|config|connect|connected|constrain|Constrain|copysign|copysignf|cos|Cos|cosf|cosh|coshf|countryNameRead|countryNameWrite|createChar|cursor|debugPrint|degrees|Delay|delay|DelayMicroseconds|delayMicroseconds|detach|DetachInterrupt|detachInterrupt|DigitalPinToInterrupt|digitalPinToInterrupt|DigitalRead|digitalRead|DigitalWrite|digitalWrite|disconnect|display|displayLogos|drawBMP|drawCompass|encryptionType|end|endPacket|endSMS|endsWith|endTransmission|endWrite|equals|equalsIgnoreCase|exists|exitValue|Exp|exp|expf|fabs|fabsf|fdim|fdimf|fill|find|findUntil|float|floor|Floor|floorf|flush|fma|fmaf|fmax|fmaxf|fmin|fminf|fmod|fmodf|gatewayIP|get|getAsynchronously|getBand|getButton|getBytes|getCurrentCarrier|getIMEI|getKey|getModifiers|getOemKey|getPINUsed|getResult|getSignalStrength|getSocket|getVoiceCallStatus|getXChange|getYChange|hangCall|height|highByte|HighByte|home|hypot|hypotf|image|indexOf|int|interrupts|IPAddress|IRread|isActionDone|isAlpha|isAlphaNumeric|isAscii|isControl|isDigit|isDirectory|isfinite|isGraph|isHexadecimalDigit|isinf|isListening|isLowerCase|isnan|isPIN|isPressed|isPrintable|isPunct|isSpace|isUpperCase|isValid|isWhitespace|keyboardRead|keyPressed|keyReleased|knobRead|lastIndexOf|ldexp|ldexpf|leftToRight|length|line|lineFollowConfig|listen|listenOnLocalhost|loadImage|localIP|log|Log|log10|log10f|logf|long|lowByte|LowByte|lrint|lrintf|lround|lroundf|macAddress|maintain|map|Map|Max|max|messageAvailable|Micros|micros|millis|Millis|Min|min|mkdir|motorsStop|motorsWrite|mouseDragged|mouseMoved|mousePressed|mouseReleased|move|noAutoscroll|noBlink|noBuffer|noCursor|noDisplay|noFill|noInterrupts|NoInterrupts|noListenOnLocalhost|noStroke|noTone|NoTone|onReceive|onRequest|open|openNextFile|overflow|parseCommand|parseFloat|parseInt|parsePacket|pauseMode|peek|PinMode|pinMode|playFile|playMelody|point|pointTo|position|Pow|pow|powf|prepare|press|print|printFirmwareVersion|println|printVersion|process|processInput|PulseIn|pulseIn|pulseInLong|PulseInLong|put|radians|random|Random|randomSeed|RandomSeed|read|readAccelerometer|readBlue|readButton|readBytes|readBytesUntil|readGreen|readJoystickButton|readJoystickSwitch|readJoystickX|readJoystickY|readLightSensor|readMessage|readMicrophone|readNetworks|readRed|readSlider|readString|readStringUntil|readTemperature|ready|rect|release|releaseAll|remoteIP|remoteNumber|remotePort|remove|replace|requestFrom|retrieveCallingNumber|rewindDirectory|rightToLeft|rmdir|robotNameRead|robotNameWrite|round|roundf|RSSI|run|runAsynchronously|running|runShellCommand|runShellCommandAsynchronously|scanNetworks|scrollDisplayLeft|scrollDisplayRight|seek|sendAnalog|sendDigitalPortPair|sendDigitalPorts|sendString|sendSysex|Serial_Available|Serial_Begin|Serial_End|Serial_Flush|Serial_Peek|Serial_Print|Serial_Println|Serial_Read|serialEvent|setBand|setBitOrder|setCharAt|setClockDivider|setCursor|setDataMode|setDNS|setFirmwareVersion|setMode|setPINUsed|setSpeed|setTextSize|setTimeout|ShiftIn|shiftIn|ShiftOut|shiftOut|shutdown|signbit|sin|Sin|sinf|sinh|sinhf|size|sizeof|Sq|sq|Sqrt|sqrt|sqrtf|SSID|startLoop|startsWith|step|stop|stroke|subnetMask|substring|switchPIN|tan|Tan|tanf|tanh|tanhf|tempoWrite|text|toCharArray|toInt|toLowerCase|tone|Tone|toUpperCase|transfer|trim|trunc|truncf|tuneWrite|turn|updateIR|userNameRead|userNameWrite|voiceCall|waitContinue|width|WiFiServer|word|write|writeBlue|writeGreen|writeJSON|writeMessage|writeMicroseconds|writeRed|writeRGB|yield|Yield)\b`, NameFunction, nil},
-			// End of: Arduino-specific syntax
-			{Words(``, `\b`, `inline`, `_inline`, `__inline`, `naked`, `restrict`, `thread`, `typename`), KeywordReserved, nil},
-			{`(__m(128i|128d|128|64))\b`, KeywordReserved, nil},
-			{Words(`__`, `\b`, `asm`, `int8`, `based`, `except`, `int16`, `stdcall`, `cdecl`, `fastcall`, `int32`, `declspec`, `finally`, `int64`, `try`, `leave`, `wchar_t`, `w64`, `unaligned`, `raise`, `noop`, `identifier`, `forceinline`, `assume`), KeywordReserved, nil},
-			{`(true|false|NULL)\b`, NameBuiltin, nil},
-			{`([a-zA-Z_]\w*)(\s*)(:)(?!:)`, ByGroups(NameLabel, Text, Punctuation), nil},
-			{`[a-zA-Z_]\w*`, Name, nil},
-		},
-		"root": {
-			Include("whitespace"),
-			{`((?:[\w*\s])+?(?:\s|[*]))([a-zA-Z_]\w*)(\s*\([^;]*?\))([^;{]*)(\{)`, ByGroups(UsingSelf("root"), NameFunction, UsingSelf("root"), UsingSelf("root"), Punctuation), Push("function")},
-			{`((?:[\w*\s])+?(?:\s|[*]))([a-zA-Z_]\w*)(\s*\([^;]*?\))([^;]*)(;)`, ByGroups(UsingSelf("root"), NameFunction, UsingSelf("root"), UsingSelf("root"), Punctuation), nil},
-			Default(Push("statement")),
-			{Words(`__`, `\b`, `virtual_inheritance`, `uuidof`, `super`, `single_inheritance`, `multiple_inheritance`, `interface`, `event`), KeywordReserved, nil},
-			{`__(offload|blockingoffload|outer)\b`, KeywordPseudo, nil},
-		},
-		"classname": {
-			{`[a-zA-Z_]\w*`, NameClass, Pop(1)},
-			{`\s*(?=>)`, Text, Pop(1)},
-		},
-		"whitespace": {
-			{`^#if\s+0`, CommentPreproc, Push("if0")},
-			{`^#`, CommentPreproc, Push("macro")},
-			{`^(\s*(?:/[*].*?[*]/\s*)?)(#if\s+0)`, ByGroups(UsingSelf("root"), CommentPreproc), Push("if0")},
-			{`^(\s*(?:/[*].*?[*]/\s*)?)(#)`, ByGroups(UsingSelf("root"), CommentPreproc), Push("macro")},
-			{`\n`, Text, nil},
-			{`\s+`, Text, nil},
-			{`\\\n`, Text, nil},
-			{`//(\n|[\w\W]*?[^\\]\n)`, CommentSingle, nil},
-			{`/(\\\n)?[*][\w\W]*?[*](\\\n)?/`, CommentMultiline, nil},
-			{`/(\\\n)?[*][\w\W]*`, CommentMultiline, nil},
-		},
-		"statement": {
-			Include("whitespace"),
-			Include("statements"),
-			{`[{}]`, Punctuation, nil},
-			{`;`, Punctuation, Pop(1)},
-		},
-		"function": {
-			Include("whitespace"),
-			Include("statements"),
-			{`;`, Punctuation, nil},
-			{`\{`, Punctuation, Push()},
-			{`\}`, Punctuation, Pop(1)},
-		},
-		"string": {
-			{`"`, LiteralString, Pop(1)},
-			{`\\([\\abfnrtv"\']|x[a-fA-F0-9]{2,4}|u[a-fA-F0-9]{4}|U[a-fA-F0-9]{8}|[0-7]{1,3})`, LiteralStringEscape, nil},
-			{`[^\\"\n]+`, LiteralString, nil},
-			{`\\\n`, LiteralString, nil},
-			{`\\`, LiteralString, nil},
-		},
-		"macro": {
-			{`(include)(\s*(?:/[*].*?[*]/\s*)?)([^\n]+)`, ByGroups(CommentPreproc, Text, CommentPreprocFile), nil},
-			{`[^/\n]+`, CommentPreproc, nil},
-			{`/[*](.|\n)*?[*]/`, CommentMultiline, nil},
-			{`//.*?\n`, CommentSingle, Pop(1)},
-			{`/`, CommentPreproc, nil},
-			{`(?<=\\)\n`, CommentPreproc, nil},
-			{`\n`, CommentPreproc, Pop(1)},
-		},
-		"if0": {
-			{`^\s*#if.*?(?<!\\)\n`, CommentPreproc, Push()},
-			{`^\s*#el(?:se|if).*\n`, CommentPreproc, Pop(1)},
-			{`^\s*#endif.*?(?<!\\)\n`, CommentPreproc, Pop(1)},
-			{`.*?\n`, Comment, nil},
-		},
-	}
-}

lexers/a/armasm.go

@@ -1,72 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-var ArmAsm = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "ArmAsm",
-		Aliases:   []string{"armasm"},
-		EnsureNL:  true,
-		Filenames: []string{"*.s", "*.S"},
-		MimeTypes: []string{"text/x-armasm", "text/x-asm"},
-	},
-	armasmRules,
-))
-
-func armasmRules() Rules {
-	return Rules{
-		"commentsandwhitespace": {
-			{`\s+`, Text, nil},
-			{`[@;].*?\n`, CommentSingle, nil},
-			{`/\*.*?\*/`, CommentMultiline, nil},
-		},
-		"literal": {
-			// Binary
-			{`0b[01]+`, NumberBin, Pop(1)},
-			// Hex
-			{`0x\w{1,8}`, NumberHex, Pop(1)},
-			// Octal
-			{`0\d+`, NumberOct, Pop(1)},
-			// Float
-			{`\d+?\.\d+?`, NumberFloat, Pop(1)},
-			// Integer
-			{`\d+`, NumberInteger, Pop(1)},
-			// String
-			{`(")(.+)(")`, ByGroups(Punctuation, StringDouble, Punctuation), Pop(1)},
-			// Char
-			{`(')(.{1}|\\.{1})(')`, ByGroups(Punctuation, StringChar, Punctuation), Pop(1)},
-		},
-		"opcode": {
-			// Escape at line end
-			{`\n`, Text, Pop(1)},
-			// Comment
-			{`(@|;).*\n`, CommentSingle, Pop(1)},
-			// Whitespace
-			{`(\s+|,)`, Text, nil},
-			// Register by number
-			{`[rapcfxwbhsdqv]\d{1,2}`, NameClass, nil},
-			// Address by hex
-			{`=0x\w+`, ByGroups(Text, NameLabel), nil},
-			// Pseudo address by label
-			{`(=)(\w+)`, ByGroups(Text, NameLabel), nil},
-			// Immediate
-			{`#`, Text, Push("literal")},
-		},
-		"root": {
-			Include("commentsandwhitespace"),
-			// Directive with optional param
-			{`(\.\w+)([ \t]+\w+\s+?)?`, ByGroups(KeywordNamespace, NameLabel), nil},
-			// Label with data
-			{`(\w+)(:)(\s+\.\w+\s+)`, ByGroups(NameLabel, Punctuation, KeywordNamespace), Push("literal")},
-			// Label
-			{`(\w+)(:)`, ByGroups(NameLabel, Punctuation), nil},
-			// Syscall Op
-			{`svc\s+\w+`, NameNamespace, nil},
-			// Opcode
-			{`[a-zA-Z]+`, Text, Push("opcode")},
-		},
-	}
-}

lexers/a/awk.go

@@ -1,52 +0,0 @@
-package a
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Awk lexer.
-var Awk = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Awk",
-		Aliases:   []string{"awk", "gawk", "mawk", "nawk"},
-		Filenames: []string{"*.awk"},
-		MimeTypes: []string{"application/x-awk"},
-	},
-	awkRules,
-))
-
-func awkRules() Rules {
-	return Rules{
-		"commentsandwhitespace": {
-			{`\s+`, Text, nil},
-			{`#.*$`, CommentSingle, nil},
-		},
-		"slashstartsregex": {
-			Include("commentsandwhitespace"),
-			{`/(\\.|[^[/\\\n]|\[(\\.|[^\]\\\n])*])+/\B`, LiteralStringRegex, Pop(1)},
-			{`(?=/)`, Text, Push("#pop", "badregex")},
-			Default(Pop(1)),
-		},
-		"badregex": {
-			{`\n`, Text, Pop(1)},
-		},
-		"root": {
-			{`^(?=\s|/)`, Text, Push("slashstartsregex")},
-			Include("commentsandwhitespace"),
-			{`\+\+|--|\|\||&&|in\b|\$|!?~|\|&|(\*\*|[-<>+*%\^/!=|])=?`, Operator, Push("slashstartsregex")},
-			{`[{(\[;,]`, Punctuation, Push("slashstartsregex")},
-			{`[})\].]`, Punctuation, nil},
-			{`(break|continue|do|while|exit|for|if|else|return|switch|case|default)\b`, Keyword, Push("slashstartsregex")},
-			{`function\b`, KeywordDeclaration, Push("slashstartsregex")},
-			{`(atan2|cos|exp|int|log|rand|sin|sqrt|srand|gensub|gsub|index|length|match|split|patsplit|sprintf|sub|substr|tolower|toupper|close|fflush|getline|next(file)|print|printf|strftime|systime|mktime|delete|system|strtonum|and|compl|lshift|or|rshift|asorti?|isarray|bindtextdomain|dcn?gettext|@(include|load|namespace))\b`, KeywordReserved, nil},
-			{`(ARGC|ARGIND|ARGV|BEGIN(FILE)?|BINMODE|CONVFMT|ENVIRON|END(FILE)?|ERRNO|FIELDWIDTHS|FILENAME|FNR|FPAT|FS|IGNORECASE|LINT|NF|NR|OFMT|OFS|ORS|PROCINFO|RLENGTH|RS|RSTART|RT|SUBSEP|TEXTDOMAIN)\b`, NameBuiltin, nil},
-			{`[@$a-zA-Z_]\w*`, NameOther, nil},
-			{`[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?`, LiteralNumberFloat, nil},
-			{`0x[0-9a-fA-F]+`, LiteralNumberHex, nil},
-			{`[0-9]+`, LiteralNumberInteger, nil},
-			{`"(\\\\|\\"|[^"])*"`, LiteralStringDouble, nil},
-			{`'(\\\\|\\'|[^'])*'`, LiteralStringSingle, nil},
-		},
-	}
-}

lexers/b/ballerina.go

@@ -1,50 +0,0 @@
-package b
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Ballerina lexer.
-var Ballerina = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Ballerina",
-		Aliases:   []string{"ballerina"},
-		Filenames: []string{"*.bal"},
-		MimeTypes: []string{"text/x-ballerina"},
-		DotAll:    true,
-	},
-	ballerinaRules,
-))
-
-func ballerinaRules() Rules {
-	return Rules{
-		"root": {
-			{`[^\S\n]+`, Text, nil},
-			{`//.*?\n`, CommentSingle, nil},
-			{`/\*.*?\*/`, CommentMultiline, nil},
-			{`(break|catch|continue|done|else|finally|foreach|forever|fork|if|lock|match|return|throw|transaction|try|while)\b`, Keyword, nil},
-			{`((?:(?:[^\W\d]|\$)[\w.\[\]$<>]*\s+)+?)((?:[^\W\d]|\$)[\w$]*)(\s*)(\()`, ByGroups(UsingSelf("root"), NameFunction, Text, Operator), nil},
-			{`@[^\W\d][\w.]*`, NameDecorator, nil},
-			{`(annotation|bind|but|endpoint|error|function|object|private|public|returns|service|type|var|with|worker)\b`, KeywordDeclaration, nil},
-			{`(boolean|byte|decimal|float|int|json|map|nil|record|string|table|xml)\b`, KeywordType, nil},
-			{`(true|false|null)\b`, KeywordConstant, nil},
-			{`(import)(\s+)`, ByGroups(KeywordNamespace, Text), Push("import")},
-			{`"(\\\\|\\"|[^"])*"`, LiteralString, nil},
-			{`'\\.'|'[^\\]'|'\\u[0-9a-fA-F]{4}'`, LiteralStringChar, nil},
-			{`(\.)((?:[^\W\d]|\$)[\w$]*)`, ByGroups(Operator, NameAttribute), nil},
-			{`^\s*([^\W\d]|\$)[\w$]*:`, NameLabel, nil},
-			{`([^\W\d]|\$)[\w$]*`, Name, nil},
-			{`([0-9][0-9_]*\.([0-9][0-9_]*)?|\.[0-9][0-9_]*)([eE][+\-]?[0-9][0-9_]*)?[fFdD]?|[0-9][eE][+\-]?[0-9][0-9_]*[fFdD]?|[0-9]([eE][+\-]?[0-9][0-9_]*)?[fFdD]|0[xX]([0-9a-fA-F][0-9a-fA-F_]*\.?|([0-9a-fA-F][0-9a-fA-F_]*)?\.[0-9a-fA-F][0-9a-fA-F_]*)[pP][+\-]?[0-9][0-9_]*[fFdD]?`, LiteralNumberFloat, nil},
-			{`0[xX][0-9a-fA-F][0-9a-fA-F_]*[lL]?`, LiteralNumberHex, nil},
-			{`0[bB][01][01_]*[lL]?`, LiteralNumberBin, nil},
-			{`0[0-7_]+[lL]?`, LiteralNumberOct, nil},
-			{`0|[1-9][0-9_]*[lL]?`, LiteralNumberInteger, nil},
-			{`[~^*!%&\[\](){}<>|+=:;,./?-]`, Operator, nil},
-			{`\n`, Text, nil},
-		},
-		"import": {
-			{`[\w.]+`, NameNamespace, Pop(1)},
-		},
-	}
-}

lexers/b/bash.go

@@ -1,100 +0,0 @@
-package b
-
-import (
-	"regexp"
-
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// TODO(moorereason): can this be factored away?
-var bashAnalyserRe = regexp.MustCompile(`(?m)^#!.*/bin/(?:env |)(?:bash|zsh|sh|ksh)`)
-
-// Bash lexer.
-var Bash = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Bash",
-		Aliases:   []string{"bash", "sh", "ksh", "zsh", "shell"},
-		Filenames: []string{"*.sh", "*.ksh", "*.bash", "*.ebuild", "*.eclass", ".env", "*.env", "*.exheres-0", "*.exlib", "*.zsh", "*.zshrc", ".bashrc", "bashrc", ".bash_*", "bash_*", "zshrc", ".zshrc", "PKGBUILD"},
-		MimeTypes: []string{"application/x-sh", "application/x-shellscript"},
-	},
-	bashRules,
-).SetAnalyser(func(text string) float32 {
-	if bashAnalyserRe.FindString(text) != "" {
-		return 1.0
-	}
-	return 0.0
-}))
-
-func bashRules() Rules {
-	return Rules{
-		"root": {
-			Include("basic"),
-			{"`", LiteralStringBacktick, Push("backticks")},
-			Include("data"),
-			Include("interp"),
-		},
-		"interp": {
-			{`\$\(\(`, Keyword, Push("math")},
-			{`\$\(`, Keyword, Push("paren")},
-			{`\$\{#?`, LiteralStringInterpol, Push("curly")},
-			{`\$[a-zA-Z_]\w*`, NameVariable, nil},
-			{`\$(?:\d+|[#$?!_*@-])`, NameVariable, nil},
-			{`\$`, Text, nil},
-		},
-		"basic": {
-			{`\b(if|fi|else|while|do|done|for|then|return|function|case|select|continue|until|esac|elif)(\s*)\b`, ByGroups(Keyword, Text), nil},
-			{"\\b(alias|bg|bind|break|builtin|caller|cd|command|compgen|complete|declare|dirs|disown|echo|enable|eval|exec|exit|export|false|fc|fg|getopts|hash|help|history|jobs|kill|let|local|logout|popd|printf|pushd|pwd|read|readonly|set|shift|shopt|source|suspend|test|time|times|trap|true|type|typeset|ulimit|umask|unalias|unset|wait)(?=[\\s)`])", NameBuiltin, nil},
-			{`\A#!.+\n`, CommentPreproc, nil},
-			{`#.*(\S|$)`, CommentSingle, nil},
-			{`\\[\w\W]`, LiteralStringEscape, nil},
-			{`(\b\w+)(\s*)(\+?=)`, ByGroups(NameVariable, Text, Operator), nil},
-			{`[\[\]{}()=]`, Operator, nil},
-			{`<<<`, Operator, nil},
-			{`<<-?\s*(\'?)\\?(\w+)[\w\W]+?\2`, LiteralString, nil},
-			{`&&|\|\|`, Operator, nil},
-		},
-		"data": {
-			{`(?s)\$?"(\\\\|\\[0-7]+|\\.|[^"\\$])*"`, LiteralStringDouble, nil},
-			{`"`, LiteralStringDouble, Push("string")},
-			{`(?s)\$'(\\\\|\\[0-7]+|\\.|[^'\\])*'`, LiteralStringSingle, nil},
-			{`(?s)'.*?'`, LiteralStringSingle, nil},
-			{`;`, Punctuation, nil},
-			{`&`, Punctuation, nil},
-			{`\|`, Punctuation, nil},
-			{`\s+`, Text, nil},
-			{`\d+(?= |$)`, LiteralNumber, nil},
-			{"[^=\\s\\[\\]{}()$\"\\'`\\\\<&|;]+", Text, nil},
-			{`<`, Text, nil},
-		},
-		"string": {
-			{`"`, LiteralStringDouble, Pop(1)},
-			{`(?s)(\\\\|\\[0-7]+|\\.|[^"\\$])+`, LiteralStringDouble, nil},
-			Include("interp"),
-		},
-		"curly": {
-			{`\}`, LiteralStringInterpol, Pop(1)},
-			{`:-`, Keyword, nil},
-			{`\w+`, NameVariable, nil},
-			{"[^}:\"\\'`$\\\\]+", Punctuation, nil},
-			{`:`, Punctuation, nil},
-			Include("root"),
-		},
-		"paren": {
-			{`\)`, Keyword, Pop(1)},
-			Include("root"),
-		},
-		"math": {
-			{`\)\)`, Keyword, Pop(1)},
-			{`[-+*/%^|&]|\*\*|\|\|`, Operator, nil},
-			{`\d+#\d+`, LiteralNumber, nil},
-			{`\d+#(?! )`, LiteralNumber, nil},
-			{`\d+`, LiteralNumber, nil},
-			Include("root"),
-		},
-		"backticks": {
-			{"`", LiteralStringBacktick, Pop(1)},
-			Include("root"),
-		},
-	}
-}

lexers/b/batch.go

@@ -1,198 +0,0 @@
-package b
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Batchfile lexer.
-var Batchfile = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "Batchfile",
-		Aliases:         []string{"bat", "batch", "dosbatch", "winbatch"},
-		Filenames:       []string{"*.bat", "*.cmd"},
-		MimeTypes:       []string{"application/x-dos-batch"},
-		CaseInsensitive: true,
-	},
-	batchfileRules,
-))
-
-func batchfileRules() Rules {
-	return Rules{
-		"root": {
-			{`\)((?=\()|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))(?:(?:[^\n\x1a^]|\^[\n\x1a]?[\w\W])*)`, CommentSingle, nil},
-			{`(?=((?:(?<=^[^:])|^[^:]?)[\t\v\f\r ,;=\xa0]*)(:))`, Text, Push("follow")},
-			{`(?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)`, UsingSelf("text"), nil},
-			Include("redirect"),
-			{`[\n\x1a]+`, Text, nil},
-			{`\(`, Punctuation, Push("root/compound")},
-			{`@+`, Punctuation, nil},
-			{`((?:for|if|rem)(?:(?=(?:\^[\n\x1a]?)?/)|(?:(?!\^)|(?<=m))(?:(?=\()|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0])+)?(?:\^[\n\x1a]?)?/(?:\^[\n\x1a]?)?\?)`, ByGroups(Keyword, UsingSelf("text")), Push("follow")},
-			{`(goto(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(]))((?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|[^"%\n\x1a&<>|])*(?:\^[\n\x1a]?)?/(?:\^[\n\x1a]?)?\?(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|[^"%\n\x1a&<>|])*)`, ByGroups(Keyword, UsingSelf("text")), Push("follow")},
-			{Words(``, `(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(])`, `assoc`, `break`, `cd`, `chdir`, `cls`, `color`, `copy`, `date`, `del`, `dir`, `dpath`, `echo`, `endlocal`, `erase`, `exit`, `ftype`, `keys`, `md`, `mkdir`, `mklink`, `move`, `path`, `pause`, `popd`, `prompt`, `pushd`, `rd`, `ren`, `rename`, `rmdir`, `setlocal`, `shift`, `start`, `time`, `title`, `type`, `ver`, `verify`, `vol`), Keyword, Push("follow")},
-			{`(call)((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)(:)`, ByGroups(Keyword, UsingSelf("text"), Punctuation), Push("call")},
-			{`call(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(])`, Keyword, nil},
-			{`(for(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])(?!\^))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))(/f(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))`, ByGroups(Keyword, UsingSelf("text"), Keyword), Push("for/f", "for")},
-			{`(for(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])(?!\^))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))(/l(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))`, ByGroups(Keyword, UsingSelf("text"), Keyword), Push("for/l", "for")},
-			{`for(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])(?!\^)`, Keyword, Push("for2", "for")},
-			{`(goto(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(]))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)(:?)`, ByGroups(Keyword, UsingSelf("text"), Punctuation), Push("label")},
-			{`(if(?:(?=\()|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))(?!\^))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)((?:/i(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))?)((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)((?:not(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))?)((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)`, ByGroups(Keyword, UsingSelf("text"), Keyword, UsingSelf("text"), Keyword, UsingSelf("text")), Push("(?", "if")},
-			{`rem(((?=\()|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))(?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?(?:[&<>|]+|(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0])+))+)?.*|(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(])(?:(?:[^\n\x1a^]|\^[\n\x1a]?[\w\W])*))`, CommentSingle, Push("follow")},
-			{`(set(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(]))((?:(?:\^[\n\x1a]?)?[^\S\n])*)(/a)`, ByGroups(Keyword, UsingSelf("text"), Keyword), Push("arithmetic")},
-			{`(set(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(]))((?:(?:\^[\n\x1a]?)?[^\S\n])*)((?:/p)?)((?:(?:\^[\n\x1a]?)?[^\S\n])*)((?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|^=]|\^[\n\x1a]?[^"=])+)?)((?:(?:\^[\n\x1a]?)?=)?)`, ByGroups(Keyword, UsingSelf("text"), Keyword, UsingSelf("text"), UsingSelf("variable"), Punctuation), Push("follow")},
-			Default(Push("follow")),
-		},
-		"follow": {
-			{`((?:(?<=^[^:])|^[^:]?)[\t\v\f\r ,;=\xa0]*)(:)([\t\v\f\r ,;=\xa0]*)((?:(?:[^\n\x1a&<>|\t\v\f\r ,;=\xa0+:^]|\^[\n\x1a]?[\w\W])*))(.*)`, ByGroups(Text, Punctuation, Text, NameLabel, CommentSingle), nil},
-			Include("redirect"),
-			{`(?=[\n\x1a])`, Text, Pop(1)},
-			{`\|\|?|&&?`, Punctuation, Pop(1)},
-			Include("text"),
-		},
-		"arithmetic": {
-			{`0[0-7]+`, LiteralNumberOct, nil},
-			{`0x[\da-f]+`, LiteralNumberHex, nil},
-			{`\d+`, LiteralNumberInteger, nil},
-			{`[(),]+`, Punctuation, nil},
-			{`([=+\-*/!~]|%|\^\^)+`, Operator, nil},
-			{`((?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(\^[\n\x1a]?)?[^()=+\-*/!~%^"\n\x1a&<>|\t\v\f\r ,;=\xa0]|\^[\n\x1a\t\v\f\r ,;=\xa0]?[\w\W])+`, UsingSelf("variable"), nil},
-			{`(?=[\x00|&])`, Text, Pop(1)},
-			Include("follow"),
-		},
-		"call": {
-			{`(:?)((?:(?:[^\n\x1a&<>|\t\v\f\r ,;=\xa0+:^]|\^[\n\x1a]?[\w\W])*))`, ByGroups(Punctuation, NameLabel), Pop(1)},
-		},
-		"label": {
-			{`((?:(?:[^\n\x1a&<>|\t\v\f\r ,;=\xa0+:^]|\^[\n\x1a]?[\w\W])*)?)((?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|\^[\n\x1a]?[\w\W]|[^"%^\n\x1a&<>|])*)`, ByGroups(NameLabel, CommentSingle), Pop(1)},
-		},
-		"redirect": {
-			{`((?:(?<=[\n\x1a\t\v\f\r ,;=\xa0])\d)?)(>>?&|<&)([\n\x1a\t\v\f\r ,;=\xa0]*)(\d)`, ByGroups(LiteralNumberInteger, Punctuation, Text, LiteralNumberInteger), nil},
-			{`((?:(?<=[\n\x1a\t\v\f\r ,;=\xa0])(?<!\^[\n\x1a])\d)?)(>>?|<)((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?(?:[&<>|]+|(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0])+))+))`, ByGroups(LiteralNumberInteger, Punctuation, UsingSelf("text")), nil},
-		},
-		"root/compound": {
-			{`\)`, Punctuation, Pop(1)},
-			{`(?=((?:(?<=^[^:])|^[^:]?)[\t\v\f\r ,;=\xa0]*)(:))`, Text, Push("follow/compound")},
-			{`(?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)`, UsingSelf("text"), nil},
-			Include("redirect/compound"),
-			{`[\n\x1a]+`, Text, nil},
-			{`\(`, Punctuation, Push("root/compound")},
-			{`@+`, Punctuation, nil},
-			{`((?:for|if|rem)(?:(?=(?:\^[\n\x1a]?)?/)|(?:(?!\^)|(?<=m))(?:(?=\()|(?:(?=\))|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])))))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0)])+)?(?:\^[\n\x1a]?)?/(?:\^[\n\x1a]?)?\?)`, ByGroups(Keyword, UsingSelf("text")), Push("follow/compound")},
-			{`(goto(?:(?=\))|(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(])))((?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|[^"%\n\x1a&<>|)])*(?:\^[\n\x1a]?)?/(?:\^[\n\x1a]?)?\?(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|[^"%\n\x1a&<>|)])*)`, ByGroups(Keyword, UsingSelf("text")), Push("follow/compound")},
-			{Words(``, `(?:(?=\))|(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(]))`, `assoc`, `break`, `cd`, `chdir`, `cls`, `color`, `copy`, `date`, `del`, `dir`, `dpath`, `echo`, `endlocal`, `erase`, `exit`, `ftype`, `keys`, `md`, `mkdir`, `mklink`, `move`, `path`, `pause`, `popd`, `prompt`, `pushd`, `rd`, `ren`, `rename`, `rmdir`, `setlocal`, `shift`, `start`, `time`, `title`, `type`, `ver`, `verify`, `vol`), Keyword, Push("follow/compound")},
-			{`(call)((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)(:)`, ByGroups(Keyword, UsingSelf("text"), Punctuation), Push("call/compound")},
-			{`call(?:(?=\))|(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(]))`, Keyword, nil},
-			{`(for(?:(?=\))|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))(?!\^))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))(/f(?:(?=\))|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])))`, ByGroups(Keyword, UsingSelf("text"), Keyword), Push("for/f", "for")},
-			{`(for(?:(?=\))|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))(?!\^))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))(/l(?:(?=\))|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])))`, ByGroups(Keyword, UsingSelf("text"), Keyword), Push("for/l", "for")},
-			{`for(?:(?=\))|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))(?!\^)`, Keyword, Push("for2", "for")},
-			{`(goto(?:(?=\))|(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(])))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)(:?)`, ByGroups(Keyword, UsingSelf("text"), Punctuation), Push("label/compound")},
-			{`(if(?:(?=\()|(?:(?=\))|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])))(?!\^))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)((?:/i(?:(?=\))|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])))?)((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)((?:not(?:(?=\))|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])))?)((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)`, ByGroups(Keyword, UsingSelf("text"), Keyword, UsingSelf("text"), Keyword, UsingSelf("text")), Push("(?", "if")},
-			{`rem(((?=\()|(?:(?=\))|(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])))(?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?(?:[&<>|]+|(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0])+))+)?.*|(?:(?=\))|(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(]))(?:(?:[^\n\x1a^)]|\^[\n\x1a]?[^)])*))`, CommentSingle, Push("follow/compound")},
-			{`(set(?:(?=\))|(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(])))((?:(?:\^[\n\x1a]?)?[^\S\n])*)(/a)`, ByGroups(Keyword, UsingSelf("text"), Keyword), Push("arithmetic/compound")},
-			{`(set(?:(?=\))|(?=(?:\^[\n\x1a]?)?[\t\v\f\r ,;=\xa0+./:[\\\]]|[\n\x1a&<>|(])))((?:(?:\^[\n\x1a]?)?[^\S\n])*)((?:/p)?)((?:(?:\^[\n\x1a]?)?[^\S\n])*)((?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|^=)]|\^[\n\x1a]?[^"=])+)?)((?:(?:\^[\n\x1a]?)?=)?)`, ByGroups(Keyword, UsingSelf("text"), Keyword, UsingSelf("text"), UsingSelf("variable"), Punctuation), Push("follow/compound")},
-			Default(Push("follow/compound")),
-		},
-		"follow/compound": {
-			{`(?=\))`, Text, Pop(1)},
-			{`((?:(?<=^[^:])|^[^:]?)[\t\v\f\r ,;=\xa0]*)(:)([\t\v\f\r ,;=\xa0]*)((?:(?:[^\n\x1a&<>|\t\v\f\r ,;=\xa0+:^)]|\^[\n\x1a]?[^)])*))(.*)`, ByGroups(Text, Punctuation, Text, NameLabel, CommentSingle), nil},
-			Include("redirect/compound"),
-			{`(?=[\n\x1a])`, Text, Pop(1)},
-			{`\|\|?|&&?`, Punctuation, Pop(1)},
-			Include("text"),
-		},
-		"arithmetic/compound": {
-			{`(?=\))`, Text, Pop(1)},
-			{`0[0-7]+`, LiteralNumberOct, nil},
-			{`0x[\da-f]+`, LiteralNumberHex, nil},
-			{`\d+`, LiteralNumberInteger, nil},
-			{`[(),]+`, Punctuation, nil},
-			{`([=+\-*/!~]|%|\^\^)+`, Operator, nil},
-			{`((?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(\^[\n\x1a]?)?[^()=+\-*/!~%^"\n\x1a&<>|\t\v\f\r ,;=\xa0]|\^[\n\x1a\t\v\f\r ,;=\xa0]?[^)])+`, UsingSelf("variable"), nil},
-			{`(?=[\x00|&])`, Text, Pop(1)},
-			Include("follow"),
-		},
-		"call/compound": {
-			{`(?=\))`, Text, Pop(1)},
-			{`(:?)((?:(?:[^\n\x1a&<>|\t\v\f\r ,;=\xa0+:^)]|\^[\n\x1a]?[^)])*))`, ByGroups(Punctuation, NameLabel), Pop(1)},
-		},
-		"label/compound": {
-			{`(?=\))`, Text, Pop(1)},
-			{`((?:(?:[^\n\x1a&<>|\t\v\f\r ,;=\xa0+:^)]|\^[\n\x1a]?[^)])*)?)((?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|\^[\n\x1a]?[^)]|[^"%^\n\x1a&<>|)])*)`, ByGroups(NameLabel, CommentSingle), Pop(1)},
-		},
-		"redirect/compound": {
-			{`((?:(?<=[\n\x1a\t\v\f\r ,;=\xa0])\d)?)(>>?&|<&)([\n\x1a\t\v\f\r ,;=\xa0]*)(\d)`, ByGroups(LiteralNumberInteger, Punctuation, Text, LiteralNumberInteger), nil},
-			{`((?:(?<=[\n\x1a\t\v\f\r ,;=\xa0])(?<!\^[\n\x1a])\d)?)(>>?|<)((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?(?:[&<>|]+|(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0)])+))+))`, ByGroups(LiteralNumberInteger, Punctuation, UsingSelf("text")), nil},
-		},
-		"variable-or-escape": {
-			{`(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))`, NameVariable, nil},
-			{`%%|\^[\n\x1a]?(\^!|[\w\W])`, LiteralStringEscape, nil},
-		},
-		"string": {
-			{`"`, LiteralStringDouble, Pop(1)},
-			{`(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))`, NameVariable, nil},
-			{`\^!|%%`, LiteralStringEscape, nil},
-			{`[^"%^\n\x1a]+|[%^]`, LiteralStringDouble, nil},
-			Default(Pop(1)),
-		},
-		"sqstring": {
-			Include("variable-or-escape"),
-			{`[^%]+|%`, LiteralStringSingle, nil},
-		},
-		"bqstring": {
-			Include("variable-or-escape"),
-			{`[^%]+|%`, LiteralStringBacktick, nil},
-		},
-		"text": {
-			{`"`, LiteralStringDouble, Push("string")},
-			Include("variable-or-escape"),
-			{`[^"%^\n\x1a&<>|\t\v\f\r ,;=\xa0\d)]+|.`, Text, nil},
-		},
-		"variable": {
-			{`"`, LiteralStringDouble, Push("string")},
-			Include("variable-or-escape"),
-			{`[^"%^\n\x1a]+|.`, NameVariable, nil},
-		},
-		"for": {
-			{`((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))(in)((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))(\()`, ByGroups(UsingSelf("text"), Keyword, UsingSelf("text"), Punctuation), Pop(1)},
-			Include("follow"),
-		},
-		"for2": {
-			{`\)`, Punctuation, nil},
-			{`((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))(do(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))`, ByGroups(UsingSelf("text"), Keyword), Pop(1)},
-			{`[\n\x1a]+`, Text, nil},
-			Include("follow"),
-		},
-		"for/f": {
-			{`(")((?:(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|[^"])*?")([\n\x1a\t\v\f\r ,;=\xa0]*)(\))`, ByGroups(LiteralStringDouble, UsingSelf("string"), Text, Punctuation), nil},
-			{`"`, LiteralStringDouble, Push("#pop", "for2", "string")},
-			{`('(?:%%|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|[\w\W])*?')([\n\x1a\t\v\f\r ,;=\xa0]*)(\))`, ByGroups(UsingSelf("sqstring"), Text, Punctuation), nil},
-			{"(`(?:%%|(?:(?:%(?:\\*|(?:~[a-z]*(?:\\$[^:]+:)?)?\\d|[^%:\\n\\x1a]+(?::(?:~(?:-?\\d+)?(?:,(?:-?\\d+)?)?|(?:[^%\\n\\x1a^]|\\^[^%\\n\\x1a])[^=\\n\\x1a]*=(?:[^%\\n\\x1a^]|\\^[^%\\n\\x1a])*)?)?%))|(?:\\^?![^!:\\n\\x1a]+(?::(?:~(?:-?\\d+)?(?:,(?:-?\\d+)?)?|(?:[^!\\n\\x1a^]|\\^[^!\\n\\x1a])[^=\\n\\x1a]*=(?:[^!\\n\\x1a^]|\\^[^!\\n\\x1a])*)?)?\\^?!))|[\\w\\W])*?`)([\\n\\x1a\\t\\v\\f\\r ,;=\\xa0]*)(\\))", ByGroups(UsingSelf("bqstring"), Text, Punctuation), nil},
-			Include("for2"),
-		},
-		"for/l": {
-			{`-?\d+`, LiteralNumberInteger, nil},
-			Include("for2"),
-		},
-		"if": {
-			{`((?:cmdextversion|errorlevel)(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))(\d+)`, ByGroups(Keyword, UsingSelf("text"), LiteralNumberInteger), Pop(1)},
-			{`(defined(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))((?:[&<>|]+|(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0])+))+))`, ByGroups(Keyword, UsingSelf("text"), UsingSelf("variable")), Pop(1)},
-			{`(exist(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)(?:[&<>|]+|(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0])+))+))`, ByGroups(Keyword, UsingSelf("text")), Pop(1)},
-			{`((?:-?(?:0[0-7]+|0x[\da-f]+|\d+)(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a]))(?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))((?:equ|geq|gtr|leq|lss|neq))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)(?:-?(?:0[0-7]+|0x[\da-f]+|\d+)(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])))`, ByGroups(UsingSelf("arithmetic"), OperatorWord, UsingSelf("arithmetic")), Pop(1)},
-			{`(?:[&<>|]+|(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0])+))+)`, UsingSelf("text"), Push("#pop", "if2")},
-		},
-		"if2": {
-			{`((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?)(==)((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)?(?:[&<>|]+|(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0])+))+))`, ByGroups(UsingSelf("text"), Operator, UsingSelf("text")), Pop(1)},
-			{`((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+))((?:equ|geq|gtr|leq|lss|neq))((?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)(?:[&<>|]+|(?:(?:"[^\n\x1a"]*(?:"|(?=[\n\x1a])))|(?:(?:%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|[^%:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%\n\x1a^]|\^[^%\n\x1a])[^=\n\x1a]*=(?:[^%\n\x1a^]|\^[^%\n\x1a])*)?)?%))|(?:\^?![^!:\n\x1a]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^!\n\x1a^]|\^[^!\n\x1a])[^=\n\x1a]*=(?:[^!\n\x1a^]|\^[^!\n\x1a])*)?)?\^?!))|(?:(?:(?:\^[\n\x1a]?)?[^"\n\x1a&<>|\t\v\f\r ,;=\xa0])+))+))`, ByGroups(UsingSelf("text"), OperatorWord, UsingSelf("text")), Pop(1)},
-		},
-		"(?": {
-			{`(?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)`, UsingSelf("text"), nil},
-			{`\(`, Punctuation, Push("#pop", "else?", "root/compound")},
-			Default(Pop(1)),
-		},
-		"else?": {
-			{`(?:(?:(?:\^[\n\x1a])?[\t\v\f\r ,;=\xa0])+)`, UsingSelf("text"), nil},
-			{`else(?=\^?[\t\v\f\r ,;=\xa0]|[&<>|\n\x1a])`, Keyword, Pop(1)},
-			Default(Pop(1)),
-		},
-	}
-}

lexers/b/bibtex.go

@@ -1,80 +0,0 @@
-package b
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Bibtex lexer.
-var Bibtex = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "BibTeX",
-		Aliases:         []string{"bib", "bibtex"},
-		Filenames:       []string{"*.bib"},
-		MimeTypes:       []string{"text/x-bibtex"},
-		NotMultiline:    true,
-		CaseInsensitive: true,
-	},
-	bibtexRules,
-))
-
-func bibtexRules() Rules {
-	return Rules{
-		"root": {
-			Include("whitespace"),
-			{`@comment`, Comment, nil},
-			{`@preamble`, NameClass, Push("closing-brace", "value", "opening-brace")},
-			{`@string`, NameClass, Push("closing-brace", "field", "opening-brace")},
-			{"@[a-z_@!$&*+\\-./:;<>?\\[\\\\\\]^`|~][\\w@!$&*+\\-./:;<>?\\[\\\\\\]^`|~]*", NameClass, Push("closing-brace", "command-body", "opening-brace")},
-			{`.+`, Comment, nil},
-		},
-		"opening-brace": {
-			Include("whitespace"),
-			{`[{(]`, Punctuation, Pop(1)},
-		},
-		"closing-brace": {
-			Include("whitespace"),
-			{`[})]`, Punctuation, Pop(1)},
-		},
-		"command-body": {
-			Include("whitespace"),
-			{`[^\s\,\}]+`, NameLabel, Push("#pop", "fields")},
-		},
-		"fields": {
-			Include("whitespace"),
-			{`,`, Punctuation, Push("field")},
-			Default(Pop(1)),
-		},
-		"field": {
-			Include("whitespace"),
-			{"[a-z_@!$&*+\\-./:;<>?\\[\\\\\\]^`|~][\\w@!$&*+\\-./:;<>?\\[\\\\\\]^`|~]*", NameAttribute, Push("value", "=")},
-			Default(Pop(1)),
-		},
-		"=": {
-			Include("whitespace"),
-			{`=`, Punctuation, Pop(1)},
-		},
-		"value": {
-			Include("whitespace"),
-			{"[a-z_@!$&*+\\-./:;<>?\\[\\\\\\]^`|~][\\w@!$&*+\\-./:;<>?\\[\\\\\\]^`|~]*", NameVariable, nil},
-			{`"`, LiteralString, Push("quoted-string")},
-			{`\{`, LiteralString, Push("braced-string")},
-			{`[\d]+`, LiteralNumber, nil},
-			{`#`, Punctuation, nil},
-			Default(Pop(1)),
-		},
-		"quoted-string": {
-			{`\{`, LiteralString, Push("braced-string")},
-			{`"`, LiteralString, Pop(1)},
-			{`[^\{\"]+`, LiteralString, nil},
-		},
-		"braced-string": {
-			{`\{`, LiteralString, Push()},
-			{`\}`, LiteralString, Pop(1)},
-			{`[^\{\}]+`, LiteralString, nil},
-		},
-		"whitespace": {
-			{`\s+`, Text, nil},
-		},
-	}
-}

lexers/b/bicep.go

@@ -1,112 +0,0 @@
-package b
-
-import (
-	"strings"
-
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Bicep lexer.
-var Bicep = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Bicep",
-		Aliases:   []string{"bicep"},
-		Filenames: []string{"*.bicep"},
-	},
-	bicepRules,
-))
-
-func bicepRules() Rules {
-	bicepFunctions := []string{
-		"any",
-		"array",
-		"concat",
-		"contains",
-		"empty",
-		"first",
-		"intersection",
-		"items",
-		"last",
-		"length",
-		"min",
-		"max",
-		"range",
-		"skip",
-		"take",
-		"union",
-		"dateTimeAdd",
-		"utcNow",
-		"deployment",
-		"environment",
-		"loadFileAsBase64",
-		"loadTextContent",
-		"int",
-		"json",
-		"extensionResourceId",
-		"getSecret",
-		"list",
-		"listKeys",
-		"listKeyValue",
-		"listAccountSas",
-		"listSecrets",
-		"pickZones",
-		"reference",
-		"resourceId",
-		"subscriptionResourceId",
-		"tenantResourceId",
-		"managementGroup",
-		"resourceGroup",
-		"subscription",
-		"tenant",
-		"base64",
-		"base64ToJson",
-		"base64ToString",
-		"dataUri",
-		"dataUriToString",
-		"endsWith",
-		"format",
-		"guid",
-		"indexOf",
-		"lastIndexOf",
-		"length",
-		"newGuid",
-		"padLeft",
-		"replace",
-		"split",
-		"startsWith",
-		"string",
-		"substring",
-		"toLower",
-		"toUpper",
-		"trim",
-		"uniqueString",
-		"uri",
-		"uriComponent",
-		"uriComponentToString",
-	}
-
-	return Rules{
-		"root": {
-			{`//[^\n\r]+`, CommentSingle, nil},
-			{`/\*.*?\*/`, CommentMultiline, nil},
-			{`([']?\w+[']?)(:)`, ByGroups(NameProperty, Punctuation), nil},
-			{`\b('(resourceGroup|subscription|managementGroup|tenant)')\b`, KeywordNamespace, nil},
-			{`'[\w\$\{\(\)\}\.]{1,}?'`, LiteralStringInterpol, nil},
-			{`('''|').*?('''|')`, LiteralString, nil},
-			{`\b(allowed|batchSize|description|maxLength|maxValue|metadata|minLength|minValue|secure)\b`, NameDecorator, nil},
-			{`\b(az|sys)\.`, NameNamespace, nil},
-			{`\b(` + strings.Join(bicepFunctions, "|") + `)\b`, NameFunction, nil},
-			// https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-functions-logical
-			{`\b(bool)(\()`, ByGroups(NameFunction, Punctuation), nil},
-			{`\b(for|if|in)\b`, Keyword, nil},
-			{`\b(module|output|param|resource|var)\b`, KeywordDeclaration, nil},
-			{`\b(array|bool|int|object|string)\b`, KeywordType, nil},
-			// https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/operators
-			{`(>=|>|<=|<|==|!=|=~|!~|::|&&|\?\?|!|-|%|\*|\/|\+)`, Operator, nil},
-			{`[\(\)\[\]\.:\?{}@=]`, Punctuation, nil},
-			{`[\w_-]+`, Text, nil},
-			{`\s+`, TextWhitespace, nil},
-		},
-	}
-}

lexers/b/blitz.go

@@ -1,52 +0,0 @@
-package b
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Blitzbasic lexer.
-var Blitzbasic = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "BlitzBasic",
-		Aliases:         []string{"blitzbasic", "b3d", "bplus"},
-		Filenames:       []string{"*.bb", "*.decls"},
-		MimeTypes:       []string{"text/x-bb"},
-		CaseInsensitive: true,
-	},
-	blitzbasicRules,
-))
-
-func blitzbasicRules() Rules {
-	return Rules{
-		"root": {
-			{`[ \t]+`, Text, nil},
-			{`;.*?\n`, CommentSingle, nil},
-			{`"`, LiteralStringDouble, Push("string")},
-			{`[0-9]+\.[0-9]*(?!\.)`, LiteralNumberFloat, nil},
-			{`\.[0-9]+(?!\.)`, LiteralNumberFloat, nil},
-			{`[0-9]+`, LiteralNumberInteger, nil},
-			{`\$[0-9a-f]+`, LiteralNumberHex, nil},
-			{`\%[10]+`, LiteralNumberBin, nil},
-			{Words(`\b`, `\b`, `Shl`, `Shr`, `Sar`, `Mod`, `Or`, `And`, `Not`, `Abs`, `Sgn`, `Handle`, `Int`, `Float`, `Str`, `First`, `Last`, `Before`, `After`), Operator, nil},
-			{`([+\-*/~=<>^])`, Operator, nil},
-			{`[(),:\[\]\\]`, Punctuation, nil},
-			{`\.([ \t]*)([a-z]\w*)`, NameLabel, nil},
-			{`\b(New)\b([ \t]+)([a-z]\w*)`, ByGroups(KeywordReserved, Text, NameClass), nil},
-			{`\b(Gosub|Goto)\b([ \t]+)([a-z]\w*)`, ByGroups(KeywordReserved, Text, NameLabel), nil},
-			{`\b(Object)\b([ \t]*)([.])([ \t]*)([a-z]\w*)\b`, ByGroups(Operator, Text, Punctuation, Text, NameClass), nil},
-			{`\b([a-z]\w*)(?:([ \t]*)(@{1,2}|[#$%])|([ \t]*)([.])([ \t]*)(?:([a-z]\w*)))?\b([ \t]*)(\()`, ByGroups(NameFunction, Text, KeywordType, Text, Punctuation, Text, NameClass, Text, Punctuation), nil},
-			{`\b(Function)\b([ \t]+)([a-z]\w*)(?:([ \t]*)(@{1,2}|[#$%])|([ \t]*)([.])([ \t]*)(?:([a-z]\w*)))?`, ByGroups(KeywordReserved, Text, NameFunction, Text, KeywordType, Text, Punctuation, Text, NameClass), nil},
-			{`\b(Type)([ \t]+)([a-z]\w*)`, ByGroups(KeywordReserved, Text, NameClass), nil},
-			{`\b(Pi|True|False|Null)\b`, KeywordConstant, nil},
-			{`\b(Local|Global|Const|Field|Dim)\b`, KeywordDeclaration, nil},
-			{Words(`\b`, `\b`, `End`, `Return`, `Exit`, `Chr`, `Len`, `Asc`, `New`, `Delete`, `Insert`, `Include`, `Function`, `Type`, `If`, `Then`, `Else`, `ElseIf`, `EndIf`, `For`, `To`, `Next`, `Step`, `Each`, `While`, `Wend`, `Repeat`, `Until`, `Forever`, `Select`, `Case`, `Default`, `Goto`, `Gosub`, `Data`, `Read`, `Restore`), KeywordReserved, nil},
-			{`([a-z]\w*)(?:([ \t]*)(@{1,2}|[#$%])|([ \t]*)([.])([ \t]*)(?:([a-z]\w*)))?`, ByGroups(NameVariable, Text, KeywordType, Text, Punctuation, Text, NameClass), nil},
-		},
-		"string": {
-			{`""`, LiteralStringDouble, nil},
-			{`"C?`, LiteralStringDouble, Pop(1)},
-			{`[^"]+`, LiteralStringDouble, nil},
-		},
-	}
-}

lexers/b/bnf.go

@@ -1,28 +0,0 @@
-package b
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Bnf lexer.
-var Bnf = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "BNF",
-		Aliases:   []string{"bnf"},
-		Filenames: []string{"*.bnf"},
-		MimeTypes: []string{"text/x-bnf"},
-	},
-	bnfRules,
-))
-
-func bnfRules() Rules {
-	return Rules{
-		"root": {
-			{`(<)([ -;=?-~]+)(>)`, ByGroups(Punctuation, NameClass, Punctuation), nil},
-			{`::=`, Operator, nil},
-			{`[^<>:]+`, Text, nil},
-			{`.`, Text, nil},
-		},
-	}
-}

lexers/b/brainfuck.go

@@ -1,38 +0,0 @@
-package b
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Brainfuck lexer.
-var Brainfuck = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Brainfuck",
-		Aliases:   []string{"brainfuck", "bf"},
-		Filenames: []string{"*.bf", "*.b"},
-		MimeTypes: []string{"application/x-brainfuck"},
-	},
-	brainfuckRules,
-))
-
-func brainfuckRules() Rules {
-	return Rules{
-		"common": {
-			{`[.,]+`, NameTag, nil},
-			{`[+-]+`, NameBuiltin, nil},
-			{`[<>]+`, NameVariable, nil},
-			{`[^.,+\-<>\[\]]+`, Comment, nil},
-		},
-		"root": {
-			{`\[`, Keyword, Push("loop")},
-			{`\]`, Error, nil},
-			Include("common"),
-		},
-		"loop": {
-			{`\[`, Keyword, Push()},
-			{`\]`, Keyword, Pop(1)},
-			Include("common"),
-		},
-	}
-}

lexers/bash.go

@@ -0,0 +1,17 @@
+package lexers
+
+import (
+	"regexp"
+)
+
+// TODO(moorereason): can this be factored away?
+var bashAnalyserRe = regexp.MustCompile(`(?m)^#!.*/bin/(?:env |)(?:bash|zsh|sh|ksh)`)
+
+func init() { // nolint: gochecknoinits
+	Get("bash").SetAnalyser(func(text string) float32 {
+		if bashAnalyserRe.FindString(text) != "" {
+			return 1.0
+		}
+		return 0.0
+	})
+}

lexers/bashsession.go

@@ -1,12 +1,11 @@
-package b
+package lexers
 
 import (
-	. "github.com/alecthomas/chroma" // nolint
+	. "github.com/alecthomas/chroma/v2" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
 )
 
 // BashSession lexer.
-var BashSession = internal.Register(MustNewLazyLexer(
+var BashSession = Register(MustNewLexer(
 	&Config{
 		Name:      "BashSession",
 		Aliases:   []string{"bash-session", "console", "shell-session"},
@@ -20,7 +19,7 @@ var BashSession = internal.Register(MustNewLazyLexer(
 func bashsessionRules() Rules {
 	return Rules{
 		"root": {
-			{`^((?:\[[^]]+@[^]]+\]\s?)?[#$%>])(\s*)(.*\n?)`, ByGroups(GenericPrompt, Text, Using(Bash)), nil},
+			{`^((?:\[[^]]+@[^]]+\]\s?)?[#$%>])(\s*)(.*\n?)`, ByGroups(GenericPrompt, Text, Using("Bash")), nil},
 			{`^.+\n?`, GenericOutput, nil},
 		},
 	}

lexers/c/c.go

@@ -1,96 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// C lexer.
-var C = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "C",
-		Aliases:   []string{"c"},
-		Filenames: []string{"*.c", "*.h", "*.idc", "*.x[bp]m"},
-		MimeTypes: []string{"text/x-chdr", "text/x-csrc", "image/x-xbitmap", "image/x-xpixmap"},
-		EnsureNL:  true,
-	},
-	cRules,
-))
-
-func cRules() Rules {
-	return Rules{
-		"whitespace": {
-			{`^#if\s+0`, CommentPreproc, Push("if0")},
-			{`^#`, CommentPreproc, Push("macro")},
-			{`^(\s*(?:/[*].*?[*]/\s*)?)(#if\s+0)`, ByGroups(UsingSelf("root"), CommentPreproc), Push("if0")},
-			{`^(\s*(?:/[*].*?[*]/\s*)?)(#)`, ByGroups(UsingSelf("root"), CommentPreproc), Push("macro")},
-			{`\n`, Text, nil},
-			{`\s+`, Text, nil},
-			{`\\\n`, Text, nil},
-			{`//(\n|[\w\W]*?[^\\]\n)`, CommentSingle, nil},
-			{`/(\\\n)?[*][\w\W]*?[*](\\\n)?/`, CommentMultiline, nil},
-			{`/(\\\n)?[*][\w\W]*`, CommentMultiline, nil},
-		},
-		"statements": {
-			{`(L?)(")`, ByGroups(LiteralStringAffix, LiteralString), Push("string")},
-			{`(L?)(')(\\.|\\[0-7]{1,3}|\\x[a-fA-F0-9]{1,2}|[^\\\'\n])(')`, ByGroups(LiteralStringAffix, LiteralStringChar, LiteralStringChar, LiteralStringChar), nil},
-			{`(\d+\.\d*|\.\d+|\d+)[eE][+-]?\d+[LlUu]*`, LiteralNumberFloat, nil},
-			{`(\d+\.\d*|\.\d+|\d+[fF])[fF]?`, LiteralNumberFloat, nil},
-			{`0x[0-9a-fA-F]+[LlUu]*`, LiteralNumberHex, nil},
-			{`0[0-7]+[LlUu]*`, LiteralNumberOct, nil},
-			{`\d+[LlUu]*`, LiteralNumberInteger, nil},
-			{`\*/`, Error, nil},
-			{`[~!%^&*+=|?:<>/-]`, Operator, nil},
-			{`[()\[\],.]`, Punctuation, nil},
-			{Words(``, `\b`, `asm`, `auto`, `break`, `case`, `const`, `continue`, `default`, `do`, `else`, `enum`, `extern`, `for`, `goto`, `if`, `register`, `restricted`, `return`, `sizeof`, `static`, `struct`, `switch`, `typedef`, `union`, `volatile`, `while`), Keyword, nil},
-			{`(bool|int|long|float|short|double|char((8|16|32)_t)?|unsigned|signed|void|u?int(_fast|_least|)(8|16|32|64)_t)\b`, KeywordType, nil},
-			{Words(``, `\b`, `inline`, `_inline`, `__inline`, `naked`, `restrict`, `thread`, `typename`), KeywordReserved, nil},
-			{`(__m(128i|128d|128|64))\b`, KeywordReserved, nil},
-			{Words(`__`, `\b`, `asm`, `int8`, `based`, `except`, `int16`, `stdcall`, `cdecl`, `fastcall`, `int32`, `declspec`, `finally`, `int64`, `try`, `leave`, `wchar_t`, `w64`, `unaligned`, `raise`, `noop`, `identifier`, `forceinline`, `assume`), KeywordReserved, nil},
-			{`(true|false|NULL)\b`, NameBuiltin, nil},
-			{`([a-zA-Z_]\w*)(\s*)(:)(?!:)`, ByGroups(NameLabel, Text, Punctuation), nil},
-			{`[a-zA-Z_]\w*`, Name, nil},
-		},
-		"root": {
-			Include("whitespace"),
-			{`((?:[\w*\s])+?(?:\s|[*]))([a-zA-Z_]\w*)(\s*\([^;]*?\))([^;{]*)(\{)`, ByGroups(UsingSelf("root"), NameFunction, UsingSelf("root"), UsingSelf("root"), Punctuation), Push("function")},
-			{`((?:[\w*\s])+?(?:\s|[*]))([a-zA-Z_]\w*)(\s*\([^;]*?\))([^;]*)(;)`, ByGroups(UsingSelf("root"), NameFunction, UsingSelf("root"), UsingSelf("root"), Punctuation), nil},
-			Default(Push("statement")),
-		},
-		"statement": {
-			Include("whitespace"),
-			Include("statements"),
-			{`[{}]`, Punctuation, nil},
-			{`;`, Punctuation, Pop(1)},
-		},
-		"function": {
-			Include("whitespace"),
-			Include("statements"),
-			{`;`, Punctuation, nil},
-			{`\{`, Punctuation, Push()},
-			{`\}`, Punctuation, Pop(1)},
-		},
-		"string": {
-			{`"`, LiteralString, Pop(1)},
-			{`\\([\\abfnrtv"\']|x[a-fA-F0-9]{2,4}|u[a-fA-F0-9]{4}|U[a-fA-F0-9]{8}|[0-7]{1,3})`, LiteralStringEscape, nil},
-			{`[^\\"\n]+`, LiteralString, nil},
-			{`\\\n`, LiteralString, nil},
-			{`\\`, LiteralString, nil},
-		},
-		"macro": {
-			{`(include)(\s*(?:/[*].*?[*]/\s*)?)([^\n]+)`, ByGroups(CommentPreproc, Text, CommentPreprocFile), nil},
-			{`[^/\n]+`, CommentPreproc, nil},
-			{`/[*](.|\n)*?[*]/`, CommentMultiline, nil},
-			{`//.*?\n`, CommentSingle, Pop(1)},
-			{`/`, CommentPreproc, nil},
-			{`(?<=\\)\n`, CommentPreproc, nil},
-			{`\n`, CommentPreproc, Pop(1)},
-		},
-		"if0": {
-			{`^\s*#if.*?(?<!\\)\n`, CommentPreproc, Push()},
-			{`^\s*#el(?:se|if).*\n`, CommentPreproc, Pop(1)},
-			{`^\s*#endif.*?(?<!\\)\n`, CommentPreproc, Pop(1)},
-			{`.*?\n`, Comment, nil},
-		},
-	}
-}

lexers/c/capnproto.go

@@ -1,65 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Cap'N'Proto Proto lexer.
-var CapNProto = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Cap'n Proto",
-		Aliases:   []string{"capnp"},
-		Filenames: []string{"*.capnp"},
-		MimeTypes: []string{},
-	},
-	capNProtoRules,
-))
-
-func capNProtoRules() Rules {
-	return Rules{
-		"root": {
-			{`#.*?$`, CommentSingle, nil},
-			{`@[0-9a-zA-Z]*`, NameDecorator, nil},
-			{`=`, Literal, Push("expression")},
-			{`:`, NameClass, Push("type")},
-			{`\$`, NameAttribute, Push("annotation")},
-			{`(struct|enum|interface|union|import|using|const|annotation|extends|in|of|on|as|with|from|fixed)\b`, Keyword, nil},
-			{`[\w.]+`, Name, nil},
-			{`[^#@=:$\w]+`, Text, nil},
-		},
-		"type": {
-			{`[^][=;,(){}$]+`, NameClass, nil},
-			{`[[(]`, NameClass, Push("parentype")},
-			Default(Pop(1)),
-		},
-		"parentype": {
-			{`[^][;()]+`, NameClass, nil},
-			{`[[(]`, NameClass, Push()},
-			{`[])]`, NameClass, Pop(1)},
-			Default(Pop(1)),
-		},
-		"expression": {
-			{`[^][;,(){}$]+`, Literal, nil},
-			{`[[(]`, Literal, Push("parenexp")},
-			Default(Pop(1)),
-		},
-		"parenexp": {
-			{`[^][;()]+`, Literal, nil},
-			{`[[(]`, Literal, Push()},
-			{`[])]`, Literal, Pop(1)},
-			Default(Pop(1)),
-		},
-		"annotation": {
-			{`[^][;,(){}=:]+`, NameAttribute, nil},
-			{`[[(]`, NameAttribute, Push("annexp")},
-			Default(Pop(1)),
-		},
-		"annexp": {
-			{`[^][;()]+`, NameAttribute, nil},
-			{`[[(]`, NameAttribute, Push()},
-			{`[])]`, NameAttribute, Pop(1)},
-			Default(Pop(1)),
-		},
-	}
-}

lexers/c/ceylon.go

@@ -1,67 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Ceylon lexer.
-var Ceylon = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Ceylon",
-		Aliases:   []string{"ceylon"},
-		Filenames: []string{"*.ceylon"},
-		MimeTypes: []string{"text/x-ceylon"},
-		DotAll:    true,
-	},
-	ceylonRules,
-))
-
-func ceylonRules() Rules {
-	return Rules{
-		"root": {
-			{`^(\s*(?:[a-zA-Z_][\w.\[\]]*\s+)+?)([a-zA-Z_]\w*)(\s*)(\()`, ByGroups(UsingSelf("root"), NameFunction, Text, Operator), nil},
-			{`[^\S\n]+`, Text, nil},
-			{`//.*?\n`, CommentSingle, nil},
-			{`/\*`, CommentMultiline, Push("comment")},
-			{`(shared|abstract|formal|default|actual|variable|deprecated|small|late|literal|doc|by|see|throws|optional|license|tagged|final|native|annotation|sealed)\b`, NameDecorator, nil},
-			{`(break|case|catch|continue|else|finally|for|in|if|return|switch|this|throw|try|while|is|exists|dynamic|nonempty|then|outer|assert|let)\b`, Keyword, nil},
-			{`(abstracts|extends|satisfies|super|given|of|out|assign)\b`, KeywordDeclaration, nil},
-			{`(function|value|void|new)\b`, KeywordType, nil},
-			{`(assembly|module|package)(\s+)`, ByGroups(KeywordNamespace, Text), nil},
-			{`(true|false|null)\b`, KeywordConstant, nil},
-			{`(class|interface|object|alias)(\s+)`, ByGroups(KeywordDeclaration, Text), Push("class")},
-			{`(import)(\s+)`, ByGroups(KeywordNamespace, Text), Push("import")},
-			{`"(\\\\|\\"|[^"])*"`, LiteralString, nil},
-			{`'\\.'|'[^\\]'|'\\\{#[0-9a-fA-F]{4}\}'`, LiteralStringChar, nil},
-			{"\".*``.*``.*\"", LiteralStringInterpol, nil},
-			{`(\.)([a-z_]\w*)`, ByGroups(Operator, NameAttribute), nil},
-			{`[a-zA-Z_]\w*:`, NameLabel, nil},
-			{`[a-zA-Z_]\w*`, Name, nil},
-			{`[~^*!%&\[\](){}<>|+=:;,./?-]`, Operator, nil},
-			{`\d{1,3}(_\d{3})+\.\d{1,3}(_\d{3})+[kMGTPmunpf]?`, LiteralNumberFloat, nil},
-			{`\d{1,3}(_\d{3})+\.[0-9]+([eE][+-]?[0-9]+)?[kMGTPmunpf]?`, LiteralNumberFloat, nil},
-			{`[0-9][0-9]*\.\d{1,3}(_\d{3})+[kMGTPmunpf]?`, LiteralNumberFloat, nil},
-			{`[0-9][0-9]*\.[0-9]+([eE][+-]?[0-9]+)?[kMGTPmunpf]?`, LiteralNumberFloat, nil},
-			{`#([0-9a-fA-F]{4})(_[0-9a-fA-F]{4})+`, LiteralNumberHex, nil},
-			{`#[0-9a-fA-F]+`, LiteralNumberHex, nil},
-			{`\$([01]{4})(_[01]{4})+`, LiteralNumberBin, nil},
-			{`\$[01]+`, LiteralNumberBin, nil},
-			{`\d{1,3}(_\d{3})+[kMGTP]?`, LiteralNumberInteger, nil},
-			{`[0-9]+[kMGTP]?`, LiteralNumberInteger, nil},
-			{`\n`, Text, nil},
-		},
-		"class": {
-			{`[A-Za-z_]\w*`, NameClass, Pop(1)},
-		},
-		"import": {
-			{`[a-z][\w.]*`, NameNamespace, Pop(1)},
-		},
-		"comment": {
-			{`[^*/]`, CommentMultiline, nil},
-			{`/\*`, CommentMultiline, Push()},
-			{`\*/`, CommentMultiline, Pop(1)},
-			{`[*/]`, CommentMultiline, nil},
-		},
-	}
-}

lexers/c/cfengine3.go

@@ -1,60 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Cfengine3 lexer.
-var Cfengine3 = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "CFEngine3",
-		Aliases:   []string{"cfengine3", "cf3"},
-		Filenames: []string{"*.cf"},
-		MimeTypes: []string{},
-	},
-	cfengine3Rules,
-))
-
-func cfengine3Rules() Rules {
-	return Rules{
-		"root": {
-			{`#.*?\n`, Comment, nil},
-			{`(body)(\s+)(\S+)(\s+)(control)`, ByGroups(Keyword, Text, Keyword, Text, Keyword), nil},
-			{`(body|bundle)(\s+)(\S+)(\s+)(\w+)(\()`, ByGroups(Keyword, Text, Keyword, Text, NameFunction, Punctuation), Push("arglist")},
-			{`(body|bundle)(\s+)(\S+)(\s+)(\w+)`, ByGroups(Keyword, Text, Keyword, Text, NameFunction), nil},
-			{`(")([^"]+)(")(\s+)(string|slist|int|real)(\s*)(=>)(\s*)`, ByGroups(Punctuation, NameVariable, Punctuation, Text, KeywordType, Text, Operator, Text), nil},
-			{`(\S+)(\s*)(=>)(\s*)`, ByGroups(KeywordReserved, Text, Operator, Text), nil},
-			{`"`, LiteralString, Push("string")},
-			{`(\w+)(\()`, ByGroups(NameFunction, Punctuation), nil},
-			{`([\w.!&|()]+)(::)`, ByGroups(NameClass, Punctuation), nil},
-			{`(\w+)(:)`, ByGroups(KeywordDeclaration, Punctuation), nil},
-			{`@[{(][^)}]+[})]`, NameVariable, nil},
-			{`[(){},;]`, Punctuation, nil},
-			{`=>`, Operator, nil},
-			{`->`, Operator, nil},
-			{`\d+\.\d+`, LiteralNumberFloat, nil},
-			{`\d+`, LiteralNumberInteger, nil},
-			{`\w+`, NameFunction, nil},
-			{`\s+`, Text, nil},
-		},
-		"string": {
-			{`\$[{(]`, LiteralStringInterpol, Push("interpol")},
-			{`\\.`, LiteralStringEscape, nil},
-			{`"`, LiteralString, Pop(1)},
-			{`\n`, LiteralString, nil},
-			{`.`, LiteralString, nil},
-		},
-		"interpol": {
-			{`\$[{(]`, LiteralStringInterpol, Push()},
-			{`[})]`, LiteralStringInterpol, Pop(1)},
-			{`[^${()}]+`, LiteralStringInterpol, nil},
-		},
-		"arglist": {
-			{`\)`, Punctuation, Pop(1)},
-			{`,`, Punctuation, nil},
-			{`\w+`, NameVariable, nil},
-			{`\s+`, Text, nil},
-		},
-	}
-}

lexers/c/chaiscript.go

@@ -1,67 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Chaiscript lexer.
-var Chaiscript = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "ChaiScript",
-		Aliases:   []string{"chai", "chaiscript"},
-		Filenames: []string{"*.chai"},
-		MimeTypes: []string{"text/x-chaiscript", "application/x-chaiscript"},
-		DotAll:    true,
-	},
-	chaiscriptRules,
-))
-
-func chaiscriptRules() Rules {
-	return Rules{
-		"commentsandwhitespace": {
-			{`\s+`, Text, nil},
-			{`//.*?\n`, CommentSingle, nil},
-			{`/\*.*?\*/`, CommentMultiline, nil},
-			{`^\#.*?\n`, CommentSingle, nil},
-		},
-		"slashstartsregex": {
-			Include("commentsandwhitespace"),
-			{`/(\\.|[^[/\\\n]|\[(\\.|[^\]\\\n])*])+/([gim]+\b|\B)`, LiteralStringRegex, Pop(1)},
-			{`(?=/)`, Text, Push("#pop", "badregex")},
-			Default(Pop(1)),
-		},
-		"badregex": {
-			{`\n`, Text, Pop(1)},
-		},
-		"root": {
-			Include("commentsandwhitespace"),
-			{`\n`, Text, nil},
-			{`[^\S\n]+`, Text, nil},
-			{`\+\+|--|~|&&|\?|:|\|\||\\(?=\n)|\.\.(<<|>>>?|==?|!=?|[-<>+*%&|^/])=?`, Operator, Push("slashstartsregex")},
-			{`[{(\[;,]`, Punctuation, Push("slashstartsregex")},
-			{`[})\].]`, Punctuation, nil},
-			{`[=+\-*/]`, Operator, nil},
-			{`(for|in|while|do|break|return|continue|if|else|throw|try|catch)\b`, Keyword, Push("slashstartsregex")},
-			{`(var)\b`, KeywordDeclaration, Push("slashstartsregex")},
-			{`(attr|def|fun)\b`, KeywordReserved, nil},
-			{`(true|false)\b`, KeywordConstant, nil},
-			{`(eval|throw)\b`, NameBuiltin, nil},
-			{"`\\S+`", NameBuiltin, nil},
-			{`[$a-zA-Z_]\w*`, NameOther, nil},
-			{`[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?`, LiteralNumberFloat, nil},
-			{`0x[0-9a-fA-F]+`, LiteralNumberHex, nil},
-			{`[0-9]+`, LiteralNumberInteger, nil},
-			{`"`, LiteralStringDouble, Push("dqstring")},
-			{`'(\\\\|\\'|[^'])*'`, LiteralStringSingle, nil},
-		},
-		"dqstring": {
-			{`\$\{[^"}]+?\}`, LiteralStringInterpol, nil},
-			{`\$`, LiteralStringDouble, nil},
-			{`\\\\`, LiteralStringDouble, nil},
-			{`\\"`, LiteralStringDouble, nil},
-			{`[^\\"$]+`, LiteralStringDouble, nil},
-			{`"`, LiteralStringDouble, Pop(1)},
-		},
-	}
-}

lexers/c/clojure.go

@@ -1,42 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Clojure lexer.
-var Clojure = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Clojure",
-		Aliases:   []string{"clojure", "clj"},
-		Filenames: []string{"*.clj"},
-		MimeTypes: []string{"text/x-clojure", "application/x-clojure"},
-	},
-	clojureRules,
-))
-
-func clojureRules() Rules {
-	return Rules{
-		"root": {
-			{`;.*$`, CommentSingle, nil},
-			{`[,\s]+`, Text, nil},
-			{`-?\d+\.\d+`, LiteralNumberFloat, nil},
-			{`-?\d+`, LiteralNumberInteger, nil},
-			{`0x-?[abcdef\d]+`, LiteralNumberHex, nil},
-			{`"(\\\\|\\"|[^"])*"`, LiteralString, nil},
-			{`'(?!#)[\w!$%*+<=>?/.#-]+`, LiteralStringSymbol, nil},
-			{`\\(.|[a-z]+)`, LiteralStringChar, nil},
-			{`::?#?(?!#)[\w!$%*+<=>?/.#-]+`, LiteralStringSymbol, nil},
-			{"~@|[`\\'#^~&@]", Operator, nil},
-			{Words(``, ` `, `.`, `def`, `do`, `fn`, `if`, `let`, `new`, `quote`, `var`, `loop`), Keyword, nil},
-			{Words(``, ` `, `def-`, `defn`, `defn-`, `defmacro`, `defmulti`, `defmethod`, `defstruct`, `defonce`, `declare`, `definline`, `definterface`, `defprotocol`, `defrecord`, `deftype`, `defproject`, `ns`), KeywordDeclaration, nil},
-			{Words(``, ` `, `*`, `+`, `-`, `->`, `/`, `<`, `<=`, `=`, `==`, `>`, `>=`, `..`, `accessor`, `agent`, `agent-errors`, `aget`, `alength`, `all-ns`, `alter`, `and`, `append-child`, `apply`, `array-map`, `aset`, `aset-boolean`, `aset-byte`, `aset-char`, `aset-double`, `aset-float`, `aset-int`, `aset-long`, `aset-short`, `assert`, `assoc`, `await`, `await-for`, `bean`, `binding`, `bit-and`, `bit-not`, `bit-or`, `bit-shift-left`, `bit-shift-right`, `bit-xor`, `boolean`, `branch?`, `butlast`, `byte`, `cast`, `char`, `children`, `class`, `clear-agent-errors`, `comment`, `commute`, `comp`, `comparator`, `complement`, `concat`, `conj`, `cons`, `constantly`, `cond`, `if-not`, `construct-proxy`, `contains?`, `count`, `create-ns`, `create-struct`, `cycle`, `dec`, `deref`, `difference`, `disj`, `dissoc`, `distinct`, `doall`, `doc`, `dorun`, `doseq`, `dosync`, `dotimes`, `doto`, `double`, `down`, `drop`, `drop-while`, `edit`, `end?`, `ensure`, `eval`, `every?`, `false?`, `ffirst`, `file-seq`, `filter`, `find`, `find-doc`, `find-ns`, `find-var`, `first`, `float`, `flush`, `for`, `fnseq`, `frest`, `gensym`, `get-proxy-class`, `get`, `hash-map`, `hash-set`, `identical?`, `identity`, `if-let`, `import`, `in-ns`, `inc`, `index`, `insert-child`, `insert-left`, `insert-right`, `inspect-table`, `inspect-tree`, `instance?`, `int`, `interleave`, `intersection`, `into`, `into-array`, `iterate`, `join`, `key`, `keys`, `keyword`, `keyword?`, `last`, `lazy-cat`, `lazy-cons`, `left`, `lefts`, `line-seq`, `list*`, `list`, `load`, `load-file`, `locking`, `long`, `loop`, `macroexpand`, `macroexpand-1`, `make-array`, `make-node`, `map`, `map-invert`, `map?`, `mapcat`, `max`, `max-key`, `memfn`, `merge`, `merge-with`, `meta`, `min`, `min-key`, `name`, `namespace`, `neg?`, `new`, `newline`, `next`, `nil?`, `node`, `not`, `not-any?`, `not-every?`, `not=`, `ns-imports`, `ns-interns`, `ns-map`, `ns-name`, `ns-publics`, `ns-refers`, `ns-resolve`, `ns-unmap`, `nth`, `nthrest`, `or`, `parse`, `partial`, `path`, `peek`, `pop`, `pos?`, `pr`, `pr-str`, `print`, `print-str`, `println`, `println-str`, `prn`, `prn-str`, `project`, `proxy`, `proxy-mappings`, `quot`, `rand`, `rand-int`, `range`, `re-find`, `re-groups`, `re-matcher`, `re-matches`, `re-pattern`, `re-seq`, `read`, `read-line`, `reduce`, `ref`, `ref-set`, `refer`, `rem`, `remove`, `remove-method`, `remove-ns`, `rename`, `rename-keys`, `repeat`, `replace`, `replicate`, `resolve`, `rest`, `resultset-seq`, `reverse`, `rfirst`, `right`, `rights`, `root`, `rrest`, `rseq`, `second`, `select`, `select-keys`, `send`, `send-off`, `seq`, `seq-zip`, `seq?`, `set`, `short`, `slurp`, `some`, `sort`, `sort-by`, `sorted-map`, `sorted-map-by`, `sorted-set`, `special-symbol?`, `split-at`, `split-with`, `str`, `string?`, `struct`, `struct-map`, `subs`, `subvec`, `symbol`, `symbol?`, `sync`, `take`, `take-nth`, `take-while`, `test`, `time`, `to-array`, `to-array-2d`, `tree-seq`, `true?`, `union`, `up`, `update-proxy`, `val`, `vals`, `var-get`, `var-set`, `var?`, `vector`, `vector-zip`, `vector?`, `when`, `when-first`, `when-let`, `when-not`, `with-local-vars`, `with-meta`, `with-open`, `with-out-str`, `xml-seq`, `xml-zip`, `zero?`, `zipmap`, `zipper`), NameBuiltin, nil},
-			{`(?<=\()(?!#)[\w!$%*+<=>?/.#-]+`, NameFunction, nil},
-			{`(?!#)[\w!$%*+<=>?/.#-]+`, NameVariable, nil},
-			{`(\[|\])`, Punctuation, nil},
-			{`(\{|\})`, Punctuation, nil},
-			{`(\(|\))`, Punctuation, nil},
-		},
-	}
-}

lexers/c/cmake.go

@@ -1,48 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Cmake lexer.
-var Cmake = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "CMake",
-		Aliases:   []string{"cmake"},
-		Filenames: []string{"*.cmake", "CMakeLists.txt"},
-		MimeTypes: []string{"text/x-cmake"},
-	},
-	cmakeRules,
-))
-
-func cmakeRules() Rules {
-	return Rules{
-		"root": {
-			{`\b(\w+)([ \t]*)(\()`, ByGroups(NameBuiltin, Text, Punctuation), Push("args")},
-			Include("keywords"),
-			Include("ws"),
-		},
-		"args": {
-			{`\(`, Punctuation, Push()},
-			{`\)`, Punctuation, Pop(1)},
-			{`(\$\{)(.+?)(\})`, ByGroups(Operator, NameVariable, Operator), nil},
-			{`(\$ENV\{)(.+?)(\})`, ByGroups(Operator, NameVariable, Operator), nil},
-			{`(\$<)(.+?)(>)`, ByGroups(Operator, NameVariable, Operator), nil},
-			{`(?s)".*?"`, LiteralStringDouble, nil},
-			{`\\\S+`, LiteralString, nil},
-			{`[^)$"# \t\n]+`, LiteralString, nil},
-			{`\n`, Text, nil},
-			Include("keywords"),
-			Include("ws"),
-		},
-		"string": {},
-		"keywords": {
-			{`\b(WIN32|UNIX|APPLE|CYGWIN|BORLAND|MINGW|MSVC|MSVC_IDE|MSVC60|MSVC70|MSVC71|MSVC80|MSVC90)\b`, Keyword, nil},
-		},
-		"ws": {
-			{`[ \t]+`, Text, nil},
-			{`#.*\n`, Comment, nil},
-		},
-	}
-}

lexers/c/cobol.go

@@ -1,55 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Cobol lexer.
-var Cobol = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "COBOL",
-		Aliases:         []string{"cobol"},
-		Filenames:       []string{"*.cob", "*.COB", "*.cpy", "*.CPY"},
-		MimeTypes:       []string{"text/x-cobol"},
-		CaseInsensitive: true,
-	},
-	cobolRules,
-))
-
-func cobolRules() Rules {
-	return Rules{
-		"root": {
-			Include("comment"),
-			Include("strings"),
-			Include("core"),
-			Include("nums"),
-			{`[a-z0-9]([\w\-]*[a-z0-9]+)?`, NameVariable, nil},
-			{`[ \t]+`, Text, nil},
-		},
-		"comment": {
-			{`(^.{6}[*/].*\n|^.{6}|\*>.*\n)`, Comment, nil},
-		},
-		"core": {
-			{`(^|(?<=[^\w\-]))(ALL\s+)?((ZEROES)|(HIGH-VALUE|LOW-VALUE|QUOTE|SPACE|ZERO)(S)?)\s*($|(?=[^\w\-]))`, NameConstant, nil},
-			{Words(`(^|(?<=[^\w\-]))`, `\s*($|(?=[^\w\-]))`, `ACCEPT`, `ADD`, `ALLOCATE`, `CALL`, `CANCEL`, `CLOSE`, `COMPUTE`, `CONFIGURATION`, `CONTINUE`, `DATA`, `DELETE`, `DISPLAY`, `DIVIDE`, `DIVISION`, `ELSE`, `END`, `END-ACCEPT`, `END-ADD`, `END-CALL`, `END-COMPUTE`, `END-DELETE`, `END-DISPLAY`, `END-DIVIDE`, `END-EVALUATE`, `END-IF`, `END-MULTIPLY`, `END-OF-PAGE`, `END-PERFORM`, `END-READ`, `END-RETURN`, `END-REWRITE`, `END-SEARCH`, `END-START`, `END-STRING`, `END-SUBTRACT`, `END-UNSTRING`, `END-WRITE`, `ENVIRONMENT`, `EVALUATE`, `EXIT`, `FD`, `FILE`, `FILE-CONTROL`, `FOREVER`, `FREE`, `GENERATE`, `GO`, `GOBACK`, `IDENTIFICATION`, `IF`, `INITIALIZE`, `INITIATE`, `INPUT-OUTPUT`, `INSPECT`, `INVOKE`, `I-O-CONTROL`, `LINKAGE`, `LOCAL-STORAGE`, `MERGE`, `MOVE`, `MULTIPLY`, `OPEN`, `PERFORM`, `PROCEDURE`, `PROGRAM-ID`, `RAISE`, `READ`, `RELEASE`, `RESUME`, `RETURN`, `REWRITE`, `SCREEN`, `SD`, `SEARCH`, `SECTION`, `SET`, `SORT`, `START`, `STOP`, `STRING`, `SUBTRACT`, `SUPPRESS`, `TERMINATE`, `THEN`, `UNLOCK`, `UNSTRING`, `USE`, `VALIDATE`, `WORKING-STORAGE`, `WRITE`), KeywordReserved, nil},
-			{Words(`(^|(?<=[^\w\-]))`, `\s*($|(?=[^\w\-]))`, `ACCESS`, `ADDRESS`, `ADVANCING`, `AFTER`, `ALL`, `ALPHABET`, `ALPHABETIC`, `ALPHABETIC-LOWER`, `ALPHABETIC-UPPER`, `ALPHANUMERIC`, `ALPHANUMERIC-EDITED`, `ALSO`, `ALTER`, `ALTERNATEANY`, `ARE`, `AREA`, `AREAS`, `ARGUMENT-NUMBER`, `ARGUMENT-VALUE`, `AS`, `ASCENDING`, `ASSIGN`, `AT`, `AUTO`, `AUTO-SKIP`, `AUTOMATIC`, `AUTOTERMINATE`, `BACKGROUND-COLOR`, `BASED`, `BEEP`, `BEFORE`, `BELL`, `BLANK`, `BLINK`, `BLOCK`, `BOTTOM`, `BY`, `BYTE-LENGTH`, `CHAINING`, `CHARACTER`, `CHARACTERS`, `CLASS`, `CODE`, `CODE-SET`, `COL`, `COLLATING`, `COLS`, `COLUMN`, `COLUMNS`, `COMMA`, `COMMAND-LINE`, `COMMIT`, `COMMON`, `CONSTANT`, `CONTAINS`, `CONTENT`, `CONTROL`, `CONTROLS`, `CONVERTING`, `COPY`, `CORR`, `CORRESPONDING`, `COUNT`, `CRT`, `CURRENCY`, `CURSOR`, `CYCLE`, `DATE`, `DAY`, `DAY-OF-WEEK`, `DE`, `DEBUGGING`, `DECIMAL-POINT`, `DECLARATIVES`, `DEFAULT`, `DELIMITED`, `DELIMITER`, `DEPENDING`, `DESCENDING`, `DETAIL`, `DISK`, `DOWN`, `DUPLICATES`, `DYNAMIC`, `EBCDIC`, `ENTRY`, `ENVIRONMENT-NAME`, `ENVIRONMENT-VALUE`, `EOL`, `EOP`, `EOS`, `ERASE`, `ERROR`, `ESCAPE`, `EXCEPTION`, `EXCLUSIVE`, `EXTEND`, `EXTERNAL`, `FILE-ID`, `FILLER`, `FINAL`, `FIRST`, `FIXED`, `FLOAT-LONG`, `FLOAT-SHORT`, `FOOTING`, `FOR`, `FOREGROUND-COLOR`, `FORMAT`, `FROM`, `FULL`, `FUNCTION`, `FUNCTION-ID`, `GIVING`, `GLOBAL`, `GROUP`, `HEADING`, `HIGHLIGHT`, `I-O`, `ID`, `IGNORE`, `IGNORING`, `IN`, `INDEX`, `INDEXED`, `INDICATE`, `INITIAL`, `INITIALIZED`, `INPUT`, `INTO`, `INTRINSIC`, `INVALID`, `IS`, `JUST`, `JUSTIFIED`, `KEY`, `LABEL`, `LAST`, `LEADING`, `LEFT`, `LENGTH`, `LIMIT`, `LIMITS`, `LINAGE`, `LINAGE-COUNTER`, `LINE`, `LINES`, `LOCALE`, `LOCK`, `LOWLIGHT`, `MANUAL`, `MEMORY`, `MINUS`, `MODE`, `MULTIPLE`, `NATIONAL`, `NATIONAL-EDITED`, `NATIVE`, `NEGATIVE`, `NEXT`, `NO`, `NULL`, `NULLS`, `NUMBER`, `NUMBERS`, `NUMERIC`, `NUMERIC-EDITED`, `OBJECT-COMPUTER`, `OCCURS`, `OF`, `OFF`, `OMITTED`, `ON`, `ONLY`, `OPTIONAL`, `ORDER`, `ORGANIZATION`, `OTHER`, `OUTPUT`, `OVERFLOW`, `OVERLINE`, `PACKED-DECIMAL`, `PADDING`, `PAGE`, `PARAGRAPH`, `PLUS`, `POINTER`, `POSITION`, `POSITIVE`, `PRESENT`, `PREVIOUS`, `PRINTER`, `PRINTING`, `PROCEDURE-POINTER`, `PROCEDURES`, `PROCEED`, `PROGRAM`, `PROGRAM-POINTER`, `PROMPT`, `QUOTE`, `QUOTES`, `RANDOM`, `RD`, `RECORD`, `RECORDING`, `RECORDS`, `RECURSIVE`, `REDEFINES`, `REEL`, `REFERENCE`, `RELATIVE`, `REMAINDER`, `REMOVAL`, `RENAMES`, `REPLACING`, `REPORT`, `REPORTING`, `REPORTS`, `REPOSITORY`, `REQUIRED`, `RESERVE`, `RETURNING`, `REVERSE-VIDEO`, `REWIND`, `RIGHT`, `ROLLBACK`, `ROUNDED`, `RUN`, `SAME`, `SCROLL`, `SECURE`, `SEGMENT-LIMIT`, `SELECT`, `SENTENCE`, `SEPARATE`, `SEQUENCE`, `SEQUENTIAL`, `SHARING`, `SIGN`, `SIGNED`, `SIGNED-INT`, `SIGNED-LONG`, `SIGNED-SHORT`, `SIZE`, `SORT-MERGE`, `SOURCE`, `SOURCE-COMPUTER`, `SPECIAL-NAMES`, `STANDARD`, `STANDARD-1`, `STANDARD-2`, `STATUS`, `SUM`, `SYMBOLIC`, `SYNC`, `SYNCHRONIZED`, `TALLYING`, `TAPE`, `TEST`, `THROUGH`, `THRU`, `TIME`, `TIMES`, `TO`, `TOP`, `TRAILING`, `TRANSFORM`, `TYPE`, `UNDERLINE`, `UNIT`, `UNSIGNED`, `UNSIGNED-INT`, `UNSIGNED-LONG`, `UNSIGNED-SHORT`, `UNTIL`, `UP`, `UPDATE`, `UPON`, `USAGE`, `USING`, `VALUE`, `VALUES`, `VARYING`, `WAIT`, `WHEN`, `WITH`, `WORDS`, `YYYYDDD`, `YYYYMMDD`), KeywordPseudo, nil},
-			{Words(`(^|(?<=[^\w\-]))`, `\s*($|(?=[^\w\-]))`, `ACTIVE-CLASS`, `ALIGNED`, `ANYCASE`, `ARITHMETIC`, `ATTRIBUTE`, `B-AND`, `B-NOT`, `B-OR`, `B-XOR`, `BIT`, `BOOLEAN`, `CD`, `CENTER`, `CF`, `CH`, `CHAIN`, `CLASS-ID`, `CLASSIFICATION`, `COMMUNICATION`, `CONDITION`, `DATA-POINTER`, `DESTINATION`, `DISABLE`, `EC`, `EGI`, `EMI`, `ENABLE`, `END-RECEIVE`, `ENTRY-CONVENTION`, `EO`, `ESI`, `EXCEPTION-OBJECT`, `EXPANDS`, `FACTORY`, `FLOAT-BINARY-16`, `FLOAT-BINARY-34`, `FLOAT-BINARY-7`, `FLOAT-DECIMAL-16`, `FLOAT-DECIMAL-34`, `FLOAT-EXTENDED`, `FORMAT`, `FUNCTION-POINTER`, `GET`, `GROUP-USAGE`, `IMPLEMENTS`, `INFINITY`, `INHERITS`, `INTERFACE`, `INTERFACE-ID`, `INVOKE`, `LC_ALL`, `LC_COLLATE`, `LC_CTYPE`, `LC_MESSAGES`, `LC_MONETARY`, `LC_NUMERIC`, `LC_TIME`, `LINE-COUNTER`, `MESSAGE`, `METHOD`, `METHOD-ID`, `NESTED`, `NONE`, `NORMAL`, `OBJECT`, `OBJECT-REFERENCE`, `OPTIONS`, `OVERRIDE`, `PAGE-COUNTER`, `PF`, `PH`, `PROPERTY`, `PROTOTYPE`, `PURGE`, `QUEUE`, `RAISE`, `RAISING`, `RECEIVE`, `RELATION`, `REPLACE`, `REPRESENTS-NOT-A-NUMBER`, `RESET`, `RESUME`, `RETRY`, `RF`, `RH`, `SECONDS`, `SEGMENT`, `SELF`, `SEND`, `SOURCES`, `STATEMENT`, `STEP`, `STRONG`, `SUB-QUEUE-1`, `SUB-QUEUE-2`, `SUB-QUEUE-3`, `SUPER`, `SYMBOL`, `SYSTEM-DEFAULT`, `TABLE`, `TERMINAL`, `TEXT`, `TYPEDEF`, `UCS-4`, `UNIVERSAL`, `USER-DEFAULT`, `UTF-16`, `UTF-8`, `VAL-STATUS`, `VALID`, `VALIDATE`, `VALIDATE-STATUS`), Error, nil},
-			{`(^|(?<=[^\w\-]))(PIC\s+.+?(?=(\s|\.\s))|PICTURE\s+.+?(?=(\s|\.\s))|(COMPUTATIONAL)(-[1-5X])?|(COMP)(-[1-5X])?|BINARY-C-LONG|BINARY-CHAR|BINARY-DOUBLE|BINARY-LONG|BINARY-SHORT|BINARY)\s*($|(?=[^\w\-]))`, KeywordType, nil},
-			{`(\*\*|\*|\+|-|/|<=|>=|<|>|==|/=|=)`, Operator, nil},
-			{`([(),;:&%.])`, Punctuation, nil},
-			{`(^|(?<=[^\w\-]))(ABS|ACOS|ANNUITY|ASIN|ATAN|BYTE-LENGTH|CHAR|COMBINED-DATETIME|CONCATENATE|COS|CURRENT-DATE|DATE-OF-INTEGER|DATE-TO-YYYYMMDD|DAY-OF-INTEGER|DAY-TO-YYYYDDD|EXCEPTION-(?:FILE|LOCATION|STATEMENT|STATUS)|EXP10|EXP|E|FACTORIAL|FRACTION-PART|INTEGER-OF-(?:DATE|DAY|PART)|INTEGER|LENGTH|LOCALE-(?:DATE|TIME(?:-FROM-SECONDS)?)|LOG(?:10)?|LOWER-CASE|MAX|MEAN|MEDIAN|MIDRANGE|MIN|MOD|NUMVAL(?:-C)?|ORD(?:-MAX|-MIN)?|PI|PRESENT-VALUE|RANDOM|RANGE|REM|REVERSE|SECONDS-FROM-FORMATTED-TIME|SECONDS-PAST-MIDNIGHT|SIGN|SIN|SQRT|STANDARD-DEVIATION|STORED-CHAR-LENGTH|SUBSTITUTE(?:-CASE)?|SUM|TAN|TEST-DATE-YYYYMMDD|TEST-DAY-YYYYDDD|TRIM|UPPER-CASE|VARIANCE|WHEN-COMPILED|YEAR-TO-YYYY)\s*($|(?=[^\w\-]))`, NameFunction, nil},
-			{`(^|(?<=[^\w\-]))(true|false)\s*($|(?=[^\w\-]))`, NameBuiltin, nil},
-			{`(^|(?<=[^\w\-]))(equal|equals|ne|lt|le|gt|ge|greater|less|than|not|and|or)\s*($|(?=[^\w\-]))`, OperatorWord, nil},
-		},
-		"strings": {
-			{`"[^"\n]*("|\n)`, LiteralStringDouble, nil},
-			{`'[^'\n]*('|\n)`, LiteralStringSingle, nil},
-		},
-		"nums": {
-			{`\d+(\s*|\.$|$)`, LiteralNumberInteger, nil},
-			{`[+-]?\d*\.\d+(E[-+]?\d+)?`, LiteralNumberFloat, nil},
-			{`[+-]?\d+\.\d*(E[-+]?\d+)?`, LiteralNumberFloat, nil},
-		},
-	}
-}

lexers/c/coffee.go

@@ -1,95 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Coffeescript lexer.
-var Coffeescript = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:         "CoffeeScript",
-		Aliases:      []string{"coffee-script", "coffeescript", "coffee"},
-		Filenames:    []string{"*.coffee"},
-		MimeTypes:    []string{"text/coffeescript"},
-		NotMultiline: true,
-		DotAll:       true,
-	},
-	coffeescriptRules,
-))
-
-func coffeescriptRules() Rules {
-	return Rules{
-		"commentsandwhitespace": {
-			{`\s+`, Text, nil},
-			{`###[^#].*?###`, CommentMultiline, nil},
-			{`#(?!##[^#]).*?\n`, CommentSingle, nil},
-		},
-		"multilineregex": {
-			{`[^/#]+`, LiteralStringRegex, nil},
-			{`///([gim]+\b|\B)`, LiteralStringRegex, Pop(1)},
-			{`#\{`, LiteralStringInterpol, Push("interpoling_string")},
-			{`[/#]`, LiteralStringRegex, nil},
-		},
-		"slashstartsregex": {
-			Include("commentsandwhitespace"),
-			{`///`, LiteralStringRegex, Push("#pop", "multilineregex")},
-			{`/(?! )(\\.|[^[/\\\n]|\[(\\.|[^\]\\\n])*])+/([gim]+\b|\B)`, LiteralStringRegex, Pop(1)},
-			{`/`, Operator, nil},
-			Default(Pop(1)),
-		},
-		"root": {
-			Include("commentsandwhitespace"),
-			{`^(?=\s|/)`, Text, Push("slashstartsregex")},
-			{"\\+\\+|~|&&|\\band\\b|\\bor\\b|\\bis\\b|\\bisnt\\b|\\bnot\\b|\\?|:|\\|\\||\\\\(?=\\n)|(<<|>>>?|==?(?!>)|!=?|=(?!>)|-(?!>)|[<>+*`%&\\|\\^/])=?", Operator, Push("slashstartsregex")},
-			{`(?:\([^()]*\))?\s*[=-]>`, NameFunction, Push("slashstartsregex")},
-			{`[{(\[;,]`, Punctuation, Push("slashstartsregex")},
-			{`[})\].]`, Punctuation, nil},
-			{`(?<![.$])(for|own|in|of|while|until|loop|break|return|continue|switch|when|then|if|unless|else|throw|try|catch|finally|new|delete|typeof|instanceof|super|extends|this|class|by)\b`, Keyword, Push("slashstartsregex")},
-			{`(?<![.$])(true|false|yes|no|on|off|null|NaN|Infinity|undefined)\b`, KeywordConstant, nil},
-			{`(Array|Boolean|Date|Error|Function|Math|netscape|Number|Object|Packages|RegExp|String|sun|decodeURI|decodeURIComponent|encodeURI|encodeURIComponent|eval|isFinite|isNaN|parseFloat|parseInt|document|window)\b`, NameBuiltin, nil},
-			{`[$a-zA-Z_][\w.:$]*\s*[:=]\s`, NameVariable, Push("slashstartsregex")},
-			{`@[$a-zA-Z_][\w.:$]*\s*[:=]\s`, NameVariableInstance, Push("slashstartsregex")},
-			{`@`, NameOther, Push("slashstartsregex")},
-			{`@?[$a-zA-Z_][\w$]*`, NameOther, nil},
-			{`[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?`, LiteralNumberFloat, nil},
-			{`0x[0-9a-fA-F]+`, LiteralNumberHex, nil},
-			{`[0-9]+`, LiteralNumberInteger, nil},
-			{`"""`, LiteralString, Push("tdqs")},
-			{`'''`, LiteralString, Push("tsqs")},
-			{`"`, LiteralString, Push("dqs")},
-			{`'`, LiteralString, Push("sqs")},
-		},
-		"strings": {
-			{`[^#\\\'"]+`, LiteralString, nil},
-		},
-		"interpoling_string": {
-			{`\}`, LiteralStringInterpol, Pop(1)},
-			Include("root"),
-		},
-		"dqs": {
-			{`"`, LiteralString, Pop(1)},
-			{`\\.|\'`, LiteralString, nil},
-			{`#\{`, LiteralStringInterpol, Push("interpoling_string")},
-			{`#`, LiteralString, nil},
-			Include("strings"),
-		},
-		"sqs": {
-			{`'`, LiteralString, Pop(1)},
-			{`#|\\.|"`, LiteralString, nil},
-			Include("strings"),
-		},
-		"tdqs": {
-			{`"""`, LiteralString, Pop(1)},
-			{`\\.|\'|"`, LiteralString, nil},
-			{`#\{`, LiteralStringInterpol, Push("interpoling_string")},
-			{`#`, LiteralString, nil},
-			Include("strings"),
-		},
-		"tsqs": {
-			{`'''`, LiteralString, Pop(1)},
-			{`#|\\.|\'|"`, LiteralString, nil},
-			Include("strings"),
-		},
-	}
-}

lexers/c/coldfusion.go

@@ -1,52 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Cfstatement lexer.
-var Cfstatement = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "cfstatement",
-		Aliases:         []string{"cfs"},
-		Filenames:       []string{},
-		MimeTypes:       []string{},
-		NotMultiline:    true,
-		CaseInsensitive: true,
-	},
-	cfstatementRules,
-))
-
-func cfstatementRules() Rules {
-	return Rules{
-		"root": {
-			{`//.*?\n`, CommentSingle, nil},
-			{`/\*(?:.|\n)*?\*/`, CommentMultiline, nil},
-			{`\+\+|--`, Operator, nil},
-			{`[-+*/^&=!]`, Operator, nil},
-			{`<=|>=|<|>|==`, Operator, nil},
-			{`mod\b`, Operator, nil},
-			{`(eq|lt|gt|lte|gte|not|is|and|or)\b`, Operator, nil},
-			{`\|\||&&`, Operator, nil},
-			{`\?`, Operator, nil},
-			{`"`, LiteralStringDouble, Push("string")},
-			{`'.*?'`, LiteralStringSingle, nil},
-			{`\d+`, LiteralNumber, nil},
-			{`(if|else|len|var|xml|default|break|switch|component|property|function|do|try|catch|in|continue|for|return|while|required|any|array|binary|boolean|component|date|guid|numeric|query|string|struct|uuid|case)\b`, Keyword, nil},
-			{`(true|false|null)\b`, KeywordConstant, nil},
-			{`(application|session|client|cookie|super|this|variables|arguments)\b`, NameConstant, nil},
-			{`([a-z_$][\w.]*)(\s*)(\()`, ByGroups(NameFunction, Text, Punctuation), nil},
-			{`[a-z_$][\w.]*`, NameVariable, nil},
-			{`[()\[\]{};:,.\\]`, Punctuation, nil},
-			{`\s+`, Text, nil},
-		},
-		"string": {
-			{`""`, LiteralStringDouble, nil},
-			{`#.+?#`, LiteralStringInterpol, nil},
-			{`[^"#]+`, LiteralStringDouble, nil},
-			{`#`, LiteralStringDouble, nil},
-			{`"`, LiteralStringDouble, Pop(1)},
-		},
-	}
-}

lexers/c/coq.go

@@ -1,67 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Coq lexer.
-var Coq = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Coq",
-		Aliases:   []string{"coq"},
-		Filenames: []string{"*.v"},
-		MimeTypes: []string{"text/x-coq"},
-	},
-	coqRules,
-))
-
-func coqRules() Rules {
-	return Rules{
-		"root": {
-			{`\s+`, Text, nil},
-			{`false|true|\(\)|\[\]`, NameBuiltinPseudo, nil},
-			{`\(\*`, Comment, Push("comment")},
-			{Words(`\b`, `\b`, `Section`, `Module`, `End`, `Require`, `Import`, `Export`, `Variable`, `Variables`, `Parameter`, `Parameters`, `Axiom`, `Hypothesis`, `Hypotheses`, `Notation`, `Local`, `Tactic`, `Reserved`, `Scope`, `Open`, `Close`, `Bind`, `Delimit`, `Definition`, `Let`, `Ltac`, `Fixpoint`, `CoFixpoint`, `Morphism`, `Relation`, `Implicit`, `Arguments`, `Set`, `Unset`, `Contextual`, `Strict`, `Prenex`, `Implicits`, `Inductive`, `CoInductive`, `Record`, `Structure`, `Canonical`, `Coercion`, `Theorem`, `Lemma`, `Corollary`, `Proposition`, `Fact`, `Remark`, `Example`, `Proof`, `Goal`, `Save`, `Qed`, `Defined`, `Hint`, `Resolve`, `Rewrite`, `View`, `Search`, `Show`, `Print`, `Printing`, `All`, `Graph`, `Projections`, `inside`, `outside`, `Check`, `Global`, `Instance`, `Class`, `Existing`, `Universe`, `Polymorphic`, `Monomorphic`, `Context`), KeywordNamespace, nil},
-			{Words(`\b`, `\b`, `forall`, `exists`, `exists2`, `fun`, `fix`, `cofix`, `struct`, `match`, `end`, `in`, `return`, `let`, `if`, `is`, `then`, `else`, `for`, `of`, `nosimpl`, `with`, `as`), Keyword, nil},
-			{Words(`\b`, `\b`, `Type`, `Prop`), KeywordType, nil},
-			{Words(`\b`, `\b`, `pose`, `set`, `move`, `case`, `elim`, `apply`, `clear`, `hnf`, `intro`, `intros`, `generalize`, `rename`, `pattern`, `after`, `destruct`, `induction`, `using`, `refine`, `inversion`, `injection`, `rewrite`, `congr`, `unlock`, `compute`, `ring`, `field`, `replace`, `fold`, `unfold`, `change`, `cutrewrite`, `simpl`, `have`, `suff`, `wlog`, `suffices`, `without`, `loss`, `nat_norm`, `assert`, `cut`, `trivial`, `revert`, `bool_congr`, `nat_congr`, `symmetry`, `transitivity`, `auto`, `split`, `left`, `right`, `autorewrite`, `tauto`, `setoid_rewrite`, `intuition`, `eauto`, `eapply`, `econstructor`, `etransitivity`, `constructor`, `erewrite`, `red`, `cbv`, `lazy`, `vm_compute`, `native_compute`, `subst`), Keyword, nil},
-			{Words(`\b`, `\b`, `by`, `done`, `exact`, `reflexivity`, `tauto`, `romega`, `omega`, `assumption`, `solve`, `contradiction`, `discriminate`, `congruence`), KeywordPseudo, nil},
-			{Words(`\b`, `\b`, `do`, `last`, `first`, `try`, `idtac`, `repeat`), KeywordReserved, nil},
-			{`\b([A-Z][\w\']*)`, Name, nil},
-			{"(\u03bb|\u03a0|\\|\\}|\\{\\||\\\\/|/\\\\|=>|~|\\}|\\|]|\\||\\{<|\\{|`|_|]|\\[\\||\\[>|\\[<|\\[|\\?\\?|\\?|>\\}|>]|>|=|<->|<-|<|;;|;|:>|:=|::|:|\\.\\.|\\.|->|-\\.|-|,|\\+|\\*|\\)|\\(|&&|&|#|!=)", Operator, nil},
-			{`([=<>@^|&+\*/$%-]|[!?~])?[!$%&*+\./:<=>?@^|~-]`, Operator, nil},
-			{`\b(unit|nat|bool|string|ascii|list)\b`, KeywordType, nil},
-			{`[^\W\d][\w']*`, Name, nil},
-			{`\d[\d_]*`, LiteralNumberInteger, nil},
-			{`0[xX][\da-fA-F][\da-fA-F_]*`, LiteralNumberHex, nil},
-			{`0[oO][0-7][0-7_]*`, LiteralNumberOct, nil},
-			{`0[bB][01][01_]*`, LiteralNumberBin, nil},
-			{`-?\d[\d_]*(.[\d_]*)?([eE][+\-]?\d[\d_]*)`, LiteralNumberFloat, nil},
-			{`'(?:(\\[\\\"'ntbr ])|(\\[0-9]{3})|(\\x[0-9a-fA-F]{2}))'`, LiteralStringChar, nil},
-			{`'.'`, LiteralStringChar, nil},
-			{`'`, Keyword, nil},
-			{`"`, LiteralStringDouble, Push("string")},
-			{`[~?][a-z][\w\']*:`, Name, nil},
-		},
-		"comment": {
-			{`[^(*)]+`, Comment, nil},
-			{`\(\*`, Comment, Push()},
-			{`\*\)`, Comment, Pop(1)},
-			{`[(*)]`, Comment, nil},
-		},
-		"string": {
-			{`[^"]+`, LiteralStringDouble, nil},
-			{`""`, LiteralStringDouble, nil},
-			{`"`, LiteralStringDouble, Pop(1)},
-		},
-		"dotted": {
-			{`\s+`, Text, nil},
-			{`\.`, Punctuation, nil},
-			{`[A-Z][\w\']*(?=\s*\.)`, NameNamespace, nil},
-			{`[A-Z][\w\']*`, NameClass, Pop(1)},
-			{`[a-z][a-z0-9_\']*`, Name, Pop(1)},
-			Default(Pop(1)),
-		},
-	}
-}

lexers/c/cpp.go

@@ -1,110 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// CPP lexer.
-var CPP = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "C++",
-		Aliases:   []string{"cpp", "c++"},
-		Filenames: []string{"*.cpp", "*.hpp", "*.c++", "*.h++", "*.cc", "*.hh", "*.cxx", "*.hxx", "*.C", "*.H", "*.cp", "*.CPP"},
-		MimeTypes: []string{"text/x-c++hdr", "text/x-c++src"},
-		EnsureNL:  true,
-	},
-	cppRules,
-))
-
-func cppRules() Rules {
-	return Rules{
-		"statements": {
-			{Words(``, `\b`, `catch`, `const_cast`, `delete`, `dynamic_cast`, `explicit`, `export`, `friend`, `mutable`, `namespace`, `new`, `operator`, `private`, `protected`, `public`, `reinterpret_cast`, `restrict`, `static_cast`, `template`, `this`, `throw`, `throws`, `try`, `typeid`, `typename`, `using`, `virtual`, `constexpr`, `nullptr`, `decltype`, `thread_local`, `alignas`, `alignof`, `static_assert`, `noexcept`, `override`, `final`, `concept`, `requires`, `consteval`, `co_await`, `co_return`, `co_yield`), Keyword, nil},
-			{`(enum)\b(\s+)(class)\b(\s*)`, ByGroups(Keyword, Text, Keyword, Text), Push("classname")},
-			{`(class|struct|enum|union)\b(\s*)`, ByGroups(Keyword, Text), Push("classname")},
-			{`\[\[.+\]\]`, NameAttribute, nil},
-			{`(R)(")([^\\()\s]{,16})(\()((?:.|\n)*?)(\)\3)(")`, ByGroups(LiteralStringAffix, LiteralString, LiteralStringDelimiter, LiteralStringDelimiter, LiteralString, LiteralStringDelimiter, LiteralString), nil},
-			{`(u8|u|U)(")`, ByGroups(LiteralStringAffix, LiteralString), Push("string")},
-			{`(L?)(")`, ByGroups(LiteralStringAffix, LiteralString), Push("string")},
-			{`(L?)(')(\\.|\\[0-7]{1,3}|\\x[a-fA-F0-9]{1,2}|[^\\\'\n])(')`, ByGroups(LiteralStringAffix, LiteralStringChar, LiteralStringChar, LiteralStringChar), nil},
-			{`(\d+\.\d*|\.\d+|\d+)[eE][+-]?\d+[LlUu]*`, LiteralNumberFloat, nil},
-			{`(\d+\.\d*|\.\d+|\d+[fF])[fF]?`, LiteralNumberFloat, nil},
-			{`0[xX]([0-9A-Fa-f]('?[0-9A-Fa-f]+)*)[LlUu]*`, LiteralNumberHex, nil},
-			{`0('?[0-7]+)+[LlUu]*`, LiteralNumberOct, nil},
-			{`0[Bb][01]('?[01]+)*[LlUu]*`, LiteralNumberBin, nil},
-			{`[0-9]('?[0-9]+)*[LlUu]*`, LiteralNumberInteger, nil},
-			{`\*/`, Error, nil},
-			{`[~!%^&*+=|?:<>/-]`, Operator, nil},
-			{`[()\[\],.]`, Punctuation, nil},
-			{Words(``, `\b`, `asm`, `auto`, `break`, `case`, `const`, `continue`, `default`, `do`, `else`, `enum`, `extern`, `for`, `goto`, `if`, `register`, `restricted`, `return`, `sizeof`, `static`, `struct`, `switch`, `typedef`, `union`, `volatile`, `while`), Keyword, nil},
-			{`(bool|int|long|float|short|double|char((8|16|32)_t)?|wchar_t|unsigned|signed|void|u?int(_fast|_least|)(8|16|32|64)_t)\b`, KeywordType, nil},
-			{Words(``, `\b`, `inline`, `_inline`, `__inline`, `naked`, `restrict`, `thread`, `typename`), KeywordReserved, nil},
-			{`(__m(128i|128d|128|64))\b`, KeywordReserved, nil},
-			{Words(`__`, `\b`, `asm`, `int8`, `based`, `except`, `int16`, `stdcall`, `cdecl`, `fastcall`, `int32`, `declspec`, `finally`, `int64`, `try`, `leave`, `w64`, `unaligned`, `raise`, `noop`, `identifier`, `forceinline`, `assume`), KeywordReserved, nil},
-			{`(true|false|NULL)\b`, NameBuiltin, nil},
-			{`([a-zA-Z_]\w*)(\s*)(:)(?!:)`, ByGroups(NameLabel, Text, Punctuation), nil},
-			{`[a-zA-Z_]\w*`, Name, nil},
-		},
-		"root": {
-			Include("whitespace"),
-			{`((?:[\w*\s])+?(?:\s|[*]))([a-zA-Z_]\w*)(\s*\([^;]*?\))([^;{]*)(\{)`, ByGroups(UsingSelf("root"), NameFunction, UsingSelf("root"), UsingSelf("root"), Punctuation), Push("function")},
-			{`((?:[\w*\s])+?(?:\s|[*]))([a-zA-Z_]\w*)(\s*\([^;]*?\))([^;]*)(;)`, ByGroups(UsingSelf("root"), NameFunction, UsingSelf("root"), UsingSelf("root"), Punctuation), nil},
-			Default(Push("statement")),
-			{Words(`__`, `\b`, `virtual_inheritance`, `uuidof`, `super`, `single_inheritance`, `multiple_inheritance`, `interface`, `event`), KeywordReserved, nil},
-			{`__(offload|blockingoffload|outer)\b`, KeywordPseudo, nil},
-		},
-		"classname": {
-			{`(\[\[.+\]\])(\s*)`, ByGroups(NameAttribute, Text), nil},
-			{`[a-zA-Z_]\w*`, NameClass, Pop(1)},
-			{`\s*(?=[>{])`, Text, Pop(1)},
-		},
-		"whitespace": {
-			{`^#if\s+0`, CommentPreproc, Push("if0")},
-			{`^#`, CommentPreproc, Push("macro")},
-			{`^(\s*(?:/[*].*?[*]/\s*)?)(#if\s+0)`, ByGroups(UsingSelf("root"), CommentPreproc), Push("if0")},
-			{`^(\s*(?:/[*].*?[*]/\s*)?)(#)`, ByGroups(UsingSelf("root"), CommentPreproc), Push("macro")},
-			{`\n`, Text, nil},
-			{`\s+`, Text, nil},
-			{`\\\n`, Text, nil},
-			{`//(\n|[\w\W]*?[^\\]\n)`, CommentSingle, nil},
-			{`/(\\\n)?[*][\w\W]*?[*](\\\n)?/`, CommentMultiline, nil},
-			{`/(\\\n)?[*][\w\W]*`, CommentMultiline, nil},
-		},
-		"statement": {
-			Include("whitespace"),
-			Include("statements"),
-			{`[{]`, Punctuation, Push("root")},
-			{`[;}]`, Punctuation, Pop(1)},
-		},
-		"function": {
-			Include("whitespace"),
-			Include("statements"),
-			{`;`, Punctuation, nil},
-			{`\{`, Punctuation, Push()},
-			{`\}`, Punctuation, Pop(1)},
-		},
-		"string": {
-			{`"`, LiteralString, Pop(1)},
-			{`\\([\\abfnrtv"\']|x[a-fA-F0-9]{2,4}|u[a-fA-F0-9]{4}|U[a-fA-F0-9]{8}|[0-7]{1,3})`, LiteralStringEscape, nil},
-			{`[^\\"\n]+`, LiteralString, nil},
-			{`\\\n`, LiteralString, nil},
-			{`\\`, LiteralString, nil},
-		},
-		"macro": {
-			{`(include)(\s*(?:/[*].*?[*]/\s*)?)([^\n]+)`, ByGroups(CommentPreproc, Text, CommentPreprocFile), nil},
-			{`[^/\n]+`, CommentPreproc, nil},
-			{`/[*](.|\n)*?[*]/`, CommentMultiline, nil},
-			{`//.*?\n`, CommentSingle, Pop(1)},
-			{`/`, CommentPreproc, nil},
-			{`(?<=\\)\n`, CommentPreproc, nil},
-			{`\n`, CommentPreproc, Pop(1)},
-		},
-		"if0": {
-			{`^\s*#if.*?(?<!\\)\n`, CommentPreproc, Push()},
-			{`^\s*#el(?:se|if).*\n`, CommentPreproc, Pop(1)},
-			{`^\s*#endif.*?(?<!\\)\n`, CommentPreproc, Pop(1)},
-			{`.*?\n`, Comment, nil},
-		},
-	}
-}

lexers/c/crystal.go

@@ -1,266 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Crystal lexer.
-var Crystal = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Crystal",
-		Aliases:   []string{"cr", "crystal"},
-		Filenames: []string{"*.cr"},
-		MimeTypes: []string{"text/x-crystal"},
-		DotAll:    true,
-	},
-	crystalRules,
-))
-
-func crystalRules() Rules {
-	return Rules{
-		"root": {
-			{`#.*?$`, CommentSingle, nil},
-			{Words(``, `\b`, `abstract`, `asm`, `as`, `begin`, `break`, `case`, `do`, `else`, `elsif`, `end`, `ensure`, `extend`, `ifdef`, `if`, `include`, `instance_sizeof`, `next`, `of`, `pointerof`, `private`, `protected`, `rescue`, `return`, `require`, `sizeof`, `super`, `then`, `typeof`, `unless`, `until`, `when`, `while`, `with`, `yield`), Keyword, nil},
-			{Words(``, `\b`, `true`, `false`, `nil`), KeywordConstant, nil},
-			{`(module|lib)(\s+)([a-zA-Z_]\w*(?:::[a-zA-Z_]\w*)*)`, ByGroups(Keyword, Text, NameNamespace), nil},
-			{`(def|fun|macro)(\s+)((?:[a-zA-Z_]\w*::)*)`, ByGroups(Keyword, Text, NameNamespace), Push("funcname")},
-			{"def(?=[*%&^`~+-/\\[<>=])", Keyword, Push("funcname")},
-			{`(class|struct|union|type|alias|enum)(\s+)((?:[a-zA-Z_]\w*::)*)`, ByGroups(Keyword, Text, NameNamespace), Push("classname")},
-			{`(self|out|uninitialized)\b|(is_a|responds_to)\?`, KeywordPseudo, nil},
-			{Words(``, `\b`, `debugger`, `record`, `pp`, `assert_responds_to`, `spawn`, `parallel`, `getter`, `setter`, `property`, `delegate`, `def_hash`, `def_equals`, `def_equals_and_hash`, `forward_missing_to`), NameBuiltinPseudo, nil},
-			{`getter[!?]|property[!?]|__(DIR|FILE|LINE)__\b`, NameBuiltinPseudo, nil},
-			{Words(`(?<!\.)`, `\b`, `Object`, `Value`, `Struct`, `Reference`, `Proc`, `Class`, `Nil`, `Symbol`, `Enum`, `Void`, `Bool`, `Number`, `Int`, `Int8`, `Int16`, `Int32`, `Int64`, `UInt8`, `UInt16`, `UInt32`, `UInt64`, `Float`, `Float32`, `Float64`, `Char`, `String`, `Pointer`, `Slice`, `Range`, `Exception`, `Regex`, `Mutex`, `StaticArray`, `Array`, `Hash`, `Set`, `Tuple`, `Deque`, `Box`, `Process`, `File`, `Dir`, `Time`, `Channel`, `Concurrent`, `Scheduler`, `abort`, `at_exit`, `caller`, `delay`, `exit`, `fork`, `future`, `get_stack_top`, `gets`, `lazy`, `loop`, `main`, `p`, `print`, `printf`, `puts`, `raise`, `rand`, `read_line`, `sleep`, `sprintf`, `system`, `with_color`), NameBuiltin, nil},
-			{"(?<!\\w)(<<-?)([\"`\\']?)([a-zA-Z_]\\w*)(\\2)(.*?\\n)", StringHeredoc, nil},
-			{`(<<-?)("|\')()(\2)(.*?\n)`, StringHeredoc, nil},
-			{`__END__`, CommentPreproc, Push("end-part")},
-			{`(?:^|(?<=[=<>~!:])|(?<=(?:\s|;)when\s)|(?<=(?:\s|;)or\s)|(?<=(?:\s|;)and\s)|(?<=\.index\s)|(?<=\.scan\s)|(?<=\.sub\s)|(?<=\.sub!\s)|(?<=\.gsub\s)|(?<=\.gsub!\s)|(?<=\.match\s)|(?<=(?:\s|;)if\s)|(?<=(?:\s|;)elsif\s)|(?<=^when\s)|(?<=^index\s)|(?<=^scan\s)|(?<=^sub\s)|(?<=^gsub\s)|(?<=^sub!\s)|(?<=^gsub!\s)|(?<=^match\s)|(?<=^if\s)|(?<=^elsif\s))(\s*)(/)`, ByGroups(Text, LiteralStringRegex), Push("multiline-regex")},
-			{`(?<=\(|,|\[)/`, LiteralStringRegex, Push("multiline-regex")},
-			{`(\s+)(/)(?![\s=])`, ByGroups(Text, LiteralStringRegex), Push("multiline-regex")},
-			{`(0o[0-7]+(?:_[0-7]+)*(?:_?[iu][0-9]+)?)\b(\s*)([/?])?`, ByGroups(LiteralNumberOct, Text, Operator), nil},
-			{`(0x[0-9A-Fa-f]+(?:_[0-9A-Fa-f]+)*(?:_?[iu][0-9]+)?)\b(\s*)([/?])?`, ByGroups(LiteralNumberHex, Text, Operator), nil},
-			{`(0b[01]+(?:_[01]+)*(?:_?[iu][0-9]+)?)\b(\s*)([/?])?`, ByGroups(LiteralNumberBin, Text, Operator), nil},
-			{`((?:0(?![0-9])|[1-9][\d_]*)(?:\.\d[\d_]*)(?:e[+-]?[0-9]+)?(?:_?f[0-9]+)?)(\s*)([/?])?`, ByGroups(LiteralNumberFloat, Text, Operator), nil},
-			{`((?:0(?![0-9])|[1-9][\d_]*)(?:\.\d[\d_]*)?(?:e[+-]?[0-9]+)(?:_?f[0-9]+)?)(\s*)([/?])?`, ByGroups(LiteralNumberFloat, Text, Operator), nil},
-			{`((?:0(?![0-9])|[1-9][\d_]*)(?:\.\d[\d_]*)?(?:e[+-]?[0-9]+)?(?:_?f[0-9]+))(\s*)([/?])?`, ByGroups(LiteralNumberFloat, Text, Operator), nil},
-			{`(0\b|[1-9][\d]*(?:_\d+)*(?:_?[iu][0-9]+)?)\b(\s*)([/?])?`, ByGroups(LiteralNumberInteger, Text, Operator), nil},
-			{`@@[a-zA-Z_]\w*`, NameVariableClass, nil},
-			{`@[a-zA-Z_]\w*`, NameVariableInstance, nil},
-			{`\$\w+`, NameVariableGlobal, nil},
-			{"\\$[!@&`\\'+~=/\\\\,;.<>_*$?:\"^-]", NameVariableGlobal, nil},
-			{`\$-[0adFiIlpvw]`, NameVariableGlobal, nil},
-			{`::`, Operator, nil},
-			Include("strings"),
-			{`\?(\\[MC]-)*(\\([\\befnrtv#"\']|x[a-fA-F0-9]{1,2}|[0-7]{1,3})|\S)(?!\w)`, LiteralStringChar, nil},
-			{`[A-Z][A-Z_]+\b`, NameConstant, nil},
-			{`\{%`, LiteralStringInterpol, Push("in-macro-control")},
-			{`\{\{`, LiteralStringInterpol, Push("in-macro-expr")},
-			{`(@\[)(\s*)([A-Z]\w*)`, ByGroups(Operator, Text, NameDecorator), Push("in-attr")},
-			{Words(`(\.|::)`, ``, `!=`, `!~`, `!`, `%`, `&&`, `&`, `**`, `*`, `+`, `-`, `/`, `<=>`, `<<`, `<=`, `<`, `===`, `==`, `=~`, `=`, `>=`, `>>`, `>`, `[]=`, `[]?`, `[]`, `^`, `||`, `|`, `~`), ByGroups(Operator, NameOperator), nil},
-			{"(\\.|::)([a-zA-Z_]\\w*[!?]?|[*%&^`~+\\-/\\[<>=])", ByGroups(Operator, Name), nil},
-			{`[a-zA-Z_]\w*(?:[!?](?!=))?`, Name, nil},
-			{`(\[|\]\??|\*\*|<=>?|>=|<<?|>>?|=~|===|!~|&&?|\|\||\.{1,3})`, Operator, nil},
-			{`[-+/*%=<>&!^|~]=?`, Operator, nil},
-			{`[(){};,/?:\\]`, Punctuation, nil},
-			{`\s+`, Text, nil},
-		},
-		"funcname": {
-			{"(?:([a-zA-Z_]\\w*)(\\.))?([a-zA-Z_]\\w*[!?]?|\\*\\*?|[-+]@?|[/%&|^`~]|\\[\\]=?|<<|>>|<=?>|>=?|===?)", ByGroups(NameClass, Operator, NameFunction), Pop(1)},
-			Default(Pop(1)),
-		},
-		"classname": {
-			{`[A-Z_]\w*`, NameClass, nil},
-			{`(\()(\s*)([A-Z_]\w*)(\s*)(\))`, ByGroups(Punctuation, Text, NameClass, Text, Punctuation), nil},
-			Default(Pop(1)),
-		},
-		"in-intp": {
-			{`\{`, LiteralStringInterpol, Push()},
-			{`\}`, LiteralStringInterpol, Pop(1)},
-			Include("root"),
-		},
-		"string-intp": {
-			{`#\{`, LiteralStringInterpol, Push("in-intp")},
-		},
-		"string-escaped": {
-			{`\\([\\befnstv#"\']|x[a-fA-F0-9]{1,2}|[0-7]{1,3})`, LiteralStringEscape, nil},
-		},
-		"string-intp-escaped": {
-			Include("string-intp"),
-			Include("string-escaped"),
-		},
-		"interpolated-regex": {
-			Include("string-intp"),
-			{`[\\#]`, LiteralStringRegex, nil},
-			{`[^\\#]+`, LiteralStringRegex, nil},
-		},
-		"interpolated-string": {
-			Include("string-intp"),
-			{`[\\#]`, LiteralStringOther, nil},
-			{`[^\\#]+`, LiteralStringOther, nil},
-		},
-		"multiline-regex": {
-			Include("string-intp"),
-			{`\\\\`, LiteralStringRegex, nil},
-			{`\\/`, LiteralStringRegex, nil},
-			{`[\\#]`, LiteralStringRegex, nil},
-			{`[^\\/#]+`, LiteralStringRegex, nil},
-			{`/[imsx]*`, LiteralStringRegex, Pop(1)},
-		},
-		"end-part": {
-			{`.+`, CommentPreproc, Pop(1)},
-		},
-		"in-macro-control": {
-			{`\{%`, LiteralStringInterpol, Push()},
-			{`%\}`, LiteralStringInterpol, Pop(1)},
-			{`for\b|in\b`, Keyword, nil},
-			Include("root"),
-		},
-		"in-macro-expr": {
-			{`\{\{`, LiteralStringInterpol, Push()},
-			{`\}\}`, LiteralStringInterpol, Pop(1)},
-			Include("root"),
-		},
-		"in-attr": {
-			{`\[`, Operator, Push()},
-			{`\]`, Operator, Pop(1)},
-			Include("root"),
-		},
-		"strings": {
-			{`\:@{0,2}[a-zA-Z_]\w*[!?]?`, LiteralStringSymbol, nil},
-			{Words(`\:@{0,2}`, ``, `!=`, `!~`, `!`, `%`, `&&`, `&`, `**`, `*`, `+`, `-`, `/`, `<=>`, `<<`, `<=`, `<`, `===`, `==`, `=~`, `=`, `>=`, `>>`, `>`, `[]=`, `[]?`, `[]`, `^`, `||`, `|`, `~`), LiteralStringSymbol, nil},
-			{`:'(\\\\|\\'|[^'])*'`, LiteralStringSymbol, nil},
-			{`'(\\\\|\\'|[^']|\\[^'\\]+)'`, LiteralStringChar, nil},
-			{`:"`, LiteralStringSymbol, Push("simple-sym")},
-			{`([a-zA-Z_]\w*)(:)(?!:)`, ByGroups(LiteralStringSymbol, Punctuation), nil},
-			{`"`, LiteralStringDouble, Push("simple-string")},
-			{"(?<!\\.)`", LiteralStringBacktick, Push("simple-backtick")},
-			{`%\{`, LiteralStringOther, Push("cb-intp-string")},
-			{`%[wi]\{`, LiteralStringOther, Push("cb-string")},
-			{`%r\{`, LiteralStringRegex, Push("cb-regex")},
-			{`%\[`, LiteralStringOther, Push("sb-intp-string")},
-			{`%[wi]\[`, LiteralStringOther, Push("sb-string")},
-			{`%r\[`, LiteralStringRegex, Push("sb-regex")},
-			{`%\(`, LiteralStringOther, Push("pa-intp-string")},
-			{`%[wi]\(`, LiteralStringOther, Push("pa-string")},
-			{`%r\(`, LiteralStringRegex, Push("pa-regex")},
-			{`%<`, LiteralStringOther, Push("ab-intp-string")},
-			{`%[wi]<`, LiteralStringOther, Push("ab-string")},
-			{`%r<`, LiteralStringRegex, Push("ab-regex")},
-			{`(%r([\W_]))((?:\\\2|(?!\2).)*)(\2[imsx]*)`, String, nil},
-			{`(%[wi]([\W_]))((?:\\\2|(?!\2).)*)(\2)`, String, nil},
-			{`(?<=[-+/*%=<>&!^|~,(])(\s*)(%([\t ])(?:(?:\\\3|(?!\3).)*)\3)`, ByGroups(Text, LiteralStringOther, None), nil},
-			{`^(\s*)(%([\t ])(?:(?:\\\3|(?!\3).)*)\3)`, ByGroups(Text, LiteralStringOther, None), nil},
-			{`(%([\[{(<]))((?:\\\2|(?!\2).)*)(\2)`, String, nil},
-		},
-		"simple-string": {
-			Include("string-intp-escaped"),
-			{`[^\\"#]+`, LiteralStringDouble, nil},
-			{`[\\#]`, LiteralStringDouble, nil},
-			{`"`, LiteralStringDouble, Pop(1)},
-		},
-		"simple-sym": {
-			Include("string-escaped"),
-			{`[^\\"#]+`, LiteralStringSymbol, nil},
-			{`[\\#]`, LiteralStringSymbol, nil},
-			{`"`, LiteralStringSymbol, Pop(1)},
-		},
-		"simple-backtick": {
-			Include("string-intp-escaped"),
-			{"[^\\\\`#]+", LiteralStringBacktick, nil},
-			{`[\\#]`, LiteralStringBacktick, nil},
-			{"`", LiteralStringBacktick, Pop(1)},
-		},
-		"cb-intp-string": {
-			{`\\[\{]`, LiteralStringOther, nil},
-			{`\{`, LiteralStringOther, Push()},
-			{`\}`, LiteralStringOther, Pop(1)},
-			Include("string-intp-escaped"),
-			{`[\\#{}]`, LiteralStringOther, nil},
-			{`[^\\#{}]+`, LiteralStringOther, nil},
-		},
-		"cb-string": {
-			{`\\[\\{}]`, LiteralStringOther, nil},
-			{`\{`, LiteralStringOther, Push()},
-			{`\}`, LiteralStringOther, Pop(1)},
-			{`[\\#{}]`, LiteralStringOther, nil},
-			{`[^\\#{}]+`, LiteralStringOther, nil},
-		},
-		"cb-regex": {
-			{`\\[\\{}]`, LiteralStringRegex, nil},
-			{`\{`, LiteralStringRegex, Push()},
-			{`\}[imsx]*`, LiteralStringRegex, Pop(1)},
-			Include("string-intp"),
-			{`[\\#{}]`, LiteralStringRegex, nil},
-			{`[^\\#{}]+`, LiteralStringRegex, nil},
-		},
-		"sb-intp-string": {
-			{`\\[\[]`, LiteralStringOther, nil},
-			{`\[`, LiteralStringOther, Push()},
-			{`\]`, LiteralStringOther, Pop(1)},
-			Include("string-intp-escaped"),
-			{`[\\#\[\]]`, LiteralStringOther, nil},
-			{`[^\\#\[\]]+`, LiteralStringOther, nil},
-		},
-		"sb-string": {
-			{`\\[\\\[\]]`, LiteralStringOther, nil},
-			{`\[`, LiteralStringOther, Push()},
-			{`\]`, LiteralStringOther, Pop(1)},
-			{`[\\#\[\]]`, LiteralStringOther, nil},
-			{`[^\\#\[\]]+`, LiteralStringOther, nil},
-		},
-		"sb-regex": {
-			{`\\[\\\[\]]`, LiteralStringRegex, nil},
-			{`\[`, LiteralStringRegex, Push()},
-			{`\][imsx]*`, LiteralStringRegex, Pop(1)},
-			Include("string-intp"),
-			{`[\\#\[\]]`, LiteralStringRegex, nil},
-			{`[^\\#\[\]]+`, LiteralStringRegex, nil},
-		},
-		"pa-intp-string": {
-			{`\\[\(]`, LiteralStringOther, nil},
-			{`\(`, LiteralStringOther, Push()},
-			{`\)`, LiteralStringOther, Pop(1)},
-			Include("string-intp-escaped"),
-			{`[\\#()]`, LiteralStringOther, nil},
-			{`[^\\#()]+`, LiteralStringOther, nil},
-		},
-		"pa-string": {
-			{`\\[\\()]`, LiteralStringOther, nil},
-			{`\(`, LiteralStringOther, Push()},
-			{`\)`, LiteralStringOther, Pop(1)},
-			{`[\\#()]`, LiteralStringOther, nil},
-			{`[^\\#()]+`, LiteralStringOther, nil},
-		},
-		"pa-regex": {
-			{`\\[\\()]`, LiteralStringRegex, nil},
-			{`\(`, LiteralStringRegex, Push()},
-			{`\)[imsx]*`, LiteralStringRegex, Pop(1)},
-			Include("string-intp"),
-			{`[\\#()]`, LiteralStringRegex, nil},
-			{`[^\\#()]+`, LiteralStringRegex, nil},
-		},
-		"ab-intp-string": {
-			{`\\[<]`, LiteralStringOther, nil},
-			{`<`, LiteralStringOther, Push()},
-			{`>`, LiteralStringOther, Pop(1)},
-			Include("string-intp-escaped"),
-			{`[\\#<>]`, LiteralStringOther, nil},
-			{`[^\\#<>]+`, LiteralStringOther, nil},
-		},
-		"ab-string": {
-			{`\\[\\<>]`, LiteralStringOther, nil},
-			{`<`, LiteralStringOther, Push()},
-			{`>`, LiteralStringOther, Pop(1)},
-			{`[\\#<>]`, LiteralStringOther, nil},
-			{`[^\\#<>]+`, LiteralStringOther, nil},
-		},
-		"ab-regex": {
-			{`\\[\\<>]`, LiteralStringRegex, nil},
-			{`<`, LiteralStringRegex, Push()},
-			{`>[imsx]*`, LiteralStringRegex, Pop(1)},
-			Include("string-intp"),
-			{`[\\#<>]`, LiteralStringRegex, nil},
-			{`[^\\#<>]+`, LiteralStringRegex, nil},
-		},
-	}
-}

lexers/c/csharp.go

@@ -1,56 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// CSharp lexer.
-var CSharp = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "C#",
-		Aliases:   []string{"csharp", "c#"},
-		Filenames: []string{"*.cs"},
-		MimeTypes: []string{"text/x-csharp"},
-		DotAll:    true,
-		EnsureNL:  true,
-	},
-	cSharpRules,
-))
-
-func cSharpRules() Rules {
-	return Rules{
-		"root": {
-			{`^\s*\[.*?\]`, NameAttribute, nil},
-			{`[^\S\n]+`, Text, nil},
-			{`\\\n`, Text, nil},
-			{`///[^\n\r]+`, CommentSpecial, nil},
-			{`//[^\n\r]+`, CommentSingle, nil},
-			{`/[*].*?[*]/`, CommentMultiline, nil},
-			{`\n`, Text, nil},
-			{`[~!%^&*()+=|\[\]:;,.<>/?-]`, Punctuation, nil},
-			{`[{}]`, Punctuation, nil},
-			{`@"(""|[^"])*"`, LiteralString, nil},
-			{`\$@?"(""|[^"])*"`, LiteralString, nil},
-			{`"(\\\\|\\"|[^"\n])*["\n]`, LiteralString, nil},
-			{`'\\.'|'[^\\]'`, LiteralStringChar, nil},
-			{`0[xX][0-9a-fA-F]+[Ll]?|[0-9_](\.[0-9]*)?([eE][+-]?[0-9]+)?[flFLdD]?`, LiteralNumber, nil},
-			{`#[ \t]*(if|endif|else|elif|define|undef|line|error|warning|region|endregion|pragma|nullable)\b[^\n\r]+`, CommentPreproc, nil},
-			{`\b(extern)(\s+)(alias)\b`, ByGroups(Keyword, Text, Keyword), nil},
-			{`(abstract|as|async|await|base|break|by|case|catch|checked|const|continue|default|delegate|do|else|enum|event|explicit|extern|false|finally|fixed|for|foreach|goto|if|implicit|in|init|internal|is|let|lock|new|null|on|operator|out|override|params|private|protected|public|readonly|ref|return|sealed|sizeof|stackalloc|static|switch|this|throw|true|try|typeof|unchecked|unsafe|virtual|void|while|get|set|new|partial|yield|add|remove|value|alias|ascending|descending|from|group|into|orderby|select|thenby|where|join|equals)\b`, Keyword, nil},
-			{`(global)(::)`, ByGroups(Keyword, Punctuation), nil},
-			{`(bool|byte|char|decimal|double|dynamic|float|int|long|object|sbyte|short|string|uint|ulong|ushort|var)\b\??`, KeywordType, nil},
-			{`(class|struct|record|interface)(\s+)`, ByGroups(Keyword, Text), Push("class")},
-			{`(namespace|using)(\s+)`, ByGroups(Keyword, Text), Push("namespace")},
-			{`@?[_a-zA-Z]\w*`, Name, nil},
-		},
-		"class": {
-			{`@?[_a-zA-Z]\w*`, NameClass, Pop(1)},
-			Default(Pop(1)),
-		},
-		"namespace": {
-			{`(?=\()`, Text, Pop(1)},
-			{`(@?[_a-zA-Z]\w*|\.)+`, NameNamespace, Pop(1)},
-		},
-	}
-}

lexers/c/cython.go

@@ -1,139 +0,0 @@
-package c
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Cython lexer.
-var Cython = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Cython",
-		Aliases:   []string{"cython", "pyx", "pyrex"},
-		Filenames: []string{"*.pyx", "*.pxd", "*.pxi"},
-		MimeTypes: []string{"text/x-cython", "application/x-cython"},
-	},
-	cythonRules,
-))
-
-func cythonRules() Rules {
-	return Rules{
-		"root": {
-			{`\n`, Text, nil},
-			{`^(\s*)("""(?:.|\n)*?""")`, ByGroups(Text, LiteralStringDoc), nil},
-			{`^(\s*)('''(?:.|\n)*?''')`, ByGroups(Text, LiteralStringDoc), nil},
-			{`[^\S\n]+`, Text, nil},
-			{`#.*$`, Comment, nil},
-			{`[]{}:(),;[]`, Punctuation, nil},
-			{`\\\n`, Text, nil},
-			{`\\`, Text, nil},
-			{`(in|is|and|or|not)\b`, OperatorWord, nil},
-			{`(<)([a-zA-Z0-9.?]+)(>)`, ByGroups(Punctuation, KeywordType, Punctuation), nil},
-			{`!=|==|<<|>>|[-~+/*%=<>&^|.?]`, Operator, nil},
-			{`(from)(\d+)(<=)(\s+)(<)(\d+)(:)`, ByGroups(Keyword, LiteralNumberInteger, Operator, Name, Operator, Name, Punctuation), nil},
-			Include("keywords"),
-			{`(def|property)(\s+)`, ByGroups(Keyword, Text), Push("funcname")},
-			{`(cp?def)(\s+)`, ByGroups(Keyword, Text), Push("cdef")},
-			{`(cdef)(:)`, ByGroups(Keyword, Punctuation), nil},
-			{`(class|struct)(\s+)`, ByGroups(Keyword, Text), Push("classname")},
-			{`(from)(\s+)`, ByGroups(Keyword, Text), Push("fromimport")},
-			{`(c?import)(\s+)`, ByGroups(Keyword, Text), Push("import")},
-			Include("builtins"),
-			Include("backtick"),
-			{`(?:[rR]|[uU][rR]|[rR][uU])"""`, LiteralString, Push("tdqs")},
-			{`(?:[rR]|[uU][rR]|[rR][uU])'''`, LiteralString, Push("tsqs")},
-			{`(?:[rR]|[uU][rR]|[rR][uU])"`, LiteralString, Push("dqs")},
-			{`(?:[rR]|[uU][rR]|[rR][uU])'`, LiteralString, Push("sqs")},
-			{`[uU]?"""`, LiteralString, Combined("stringescape", "tdqs")},
-			{`[uU]?'''`, LiteralString, Combined("stringescape", "tsqs")},
-			{`[uU]?"`, LiteralString, Combined("stringescape", "dqs")},
-			{`[uU]?'`, LiteralString, Combined("stringescape", "sqs")},
-			Include("name"),
-			Include("numbers"),
-		},
-		"keywords": {
-			{Words(``, `\b`, `assert`, `break`, `by`, `continue`, `ctypedef`, `del`, `elif`, `else`, `except`, `except?`, `exec`, `finally`, `for`, `fused`, `gil`, `global`, `if`, `include`, `lambda`, `nogil`, `pass`, `print`, `raise`, `return`, `try`, `while`, `yield`, `as`, `with`), Keyword, nil},
-			{`(DEF|IF|ELIF|ELSE)\b`, CommentPreproc, nil},
-		},
-		"builtins": {
-			{Words(`(?<!\.)`, `\b`, `__import__`, `abs`, `all`, `any`, `apply`, `basestring`, `bin`, `bool`, `buffer`, `bytearray`, `bytes`, `callable`, `chr`, `classmethod`, `cmp`, `coerce`, `compile`, `complex`, `delattr`, `dict`, `dir`, `divmod`, `enumerate`, `eval`, `execfile`, `exit`, `file`, `filter`, `float`, `frozenset`, `getattr`, `globals`, `hasattr`, `hash`, `hex`, `id`, `input`, `int`, `intern`, `isinstance`, `issubclass`, `iter`, `len`, `list`, `locals`, `long`, `map`, `max`, `min`, `next`, `object`, `oct`, `open`, `ord`, `pow`, `property`, `range`, `raw_input`, `reduce`, `reload`, `repr`, `reversed`, `round`, `set`, `setattr`, `slice`, `sorted`, `staticmethod`, `str`, `sum`, `super`, `tuple`, `type`, `unichr`, `unicode`, `unsigned`, `vars`, `xrange`, `zip`), NameBuiltin, nil},
-			{`(?<!\.)(self|None|Ellipsis|NotImplemented|False|True|NULL)\b`, NameBuiltinPseudo, nil},
-			{Words(`(?<!\.)`, `\b`, `ArithmeticError`, `AssertionError`, `AttributeError`, `BaseException`, `DeprecationWarning`, `EOFError`, `EnvironmentError`, `Exception`, `FloatingPointError`, `FutureWarning`, `GeneratorExit`, `IOError`, `ImportError`, `ImportWarning`, `IndentationError`, `IndexError`, `KeyError`, `KeyboardInterrupt`, `LookupError`, `MemoryError`, `NameError`, `NotImplemented`, `NotImplementedError`, `OSError`, `OverflowError`, `OverflowWarning`, `PendingDeprecationWarning`, `ReferenceError`, `RuntimeError`, `RuntimeWarning`, `StandardError`, `StopIteration`, `SyntaxError`, `SyntaxWarning`, `SystemError`, `SystemExit`, `TabError`, `TypeError`, `UnboundLocalError`, `UnicodeDecodeError`, `UnicodeEncodeError`, `UnicodeError`, `UnicodeTranslateError`, `UnicodeWarning`, `UserWarning`, `ValueError`, `Warning`, `ZeroDivisionError`), NameException, nil},
-		},
-		"numbers": {
-			{`(\d+\.?\d*|\d*\.\d+)([eE][+-]?[0-9]+)?`, LiteralNumberFloat, nil},
-			{`0\d+`, LiteralNumberOct, nil},
-			{`0[xX][a-fA-F0-9]+`, LiteralNumberHex, nil},
-			{`\d+L`, LiteralNumberIntegerLong, nil},
-			{`\d+`, LiteralNumberInteger, nil},
-		},
-		"backtick": {
-			{"`.*?`", LiteralStringBacktick, nil},
-		},
-		"name": {
-			{`@\w+`, NameDecorator, nil},
-			{`[a-zA-Z_]\w*`, Name, nil},
-		},
-		"funcname": {
-			{`[a-zA-Z_]\w*`, NameFunction, Pop(1)},
-		},
-		"cdef": {
-			{`(public|readonly|extern|api|inline)\b`, KeywordReserved, nil},
-			{`(struct|enum|union|class)\b`, Keyword, nil},
-			{`([a-zA-Z_]\w*)(\s*)(?=[(:#=]|$)`, ByGroups(NameFunction, Text), Pop(1)},
-			{`([a-zA-Z_]\w*)(\s*)(,)`, ByGroups(NameFunction, Text, Punctuation), nil},
-			{`from\b`, Keyword, Pop(1)},
-			{`as\b`, Keyword, nil},
-			{`:`, Punctuation, Pop(1)},
-			{`(?=["\'])`, Text, Pop(1)},
-			{`[a-zA-Z_]\w*`, KeywordType, nil},
-			{`.`, Text, nil},
-		},
-		"classname": {
-			{`[a-zA-Z_]\w*`, NameClass, Pop(1)},
-		},
-		"import": {
-			{`(\s+)(as)(\s+)`, ByGroups(Text, Keyword, Text), nil},
-			{`[a-zA-Z_][\w.]*`, NameNamespace, nil},
-			{`(\s*)(,)(\s*)`, ByGroups(Text, Operator, Text), nil},
-			Default(Pop(1)),
-		},
-		"fromimport": {
-			{`(\s+)(c?import)\b`, ByGroups(Text, Keyword), Pop(1)},
-			{`[a-zA-Z_.][\w.]*`, NameNamespace, nil},
-			Default(Pop(1)),
-		},
-		"stringescape": {
-			{`\\([\\abfnrtv"\']|\n|N\{.*?\}|u[a-fA-F0-9]{4}|U[a-fA-F0-9]{8}|x[a-fA-F0-9]{2}|[0-7]{1,3})`, LiteralStringEscape, nil},
-		},
-		"strings": {
-			{`%(\([a-zA-Z0-9]+\))?[-#0 +]*([0-9]+|[*])?(\.([0-9]+|[*]))?[hlL]?[E-GXc-giorsux%]`, LiteralStringInterpol, nil},
-			{`[^\\\'"%\n]+`, LiteralString, nil},
-			{`[\'"\\]`, LiteralString, nil},
-			{`%`, LiteralString, nil},
-		},
-		"nl": {
-			{`\n`, LiteralString, nil},
-		},
-		"dqs": {
-			{`"`, LiteralString, Pop(1)},
-			{`\\\\|\\"|\\\n`, LiteralStringEscape, nil},
-			Include("strings"),
-		},
-		"sqs": {
-			{`'`, LiteralString, Pop(1)},
-			{`\\\\|\\'|\\\n`, LiteralStringEscape, nil},
-			Include("strings"),
-		},
-		"tdqs": {
-			{`"""`, LiteralString, Pop(1)},
-			Include("strings"),
-			Include("nl"),
-		},
-		"tsqs": {
-			{`'''`, LiteralString, Pop(1)},
-			Include("strings"),
-			Include("nl"),
-		},
-	}
-}

lexers/caddyfile.go

@@ -1,8 +1,7 @@
-package c
+package lexers
 
 import (
-	. "github.com/alecthomas/chroma" // nolint
+	. "github.com/alecthomas/chroma/v2" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
 )
 
 // caddyfileCommon are the rules common to both of the lexer variants
@@ -137,7 +136,7 @@ func caddyfileCommonRules() Rules {
 }
 
 // Caddyfile lexer.
-var Caddyfile = internal.Register(MustNewLazyLexer(
+var Caddyfile = Register(MustNewLexer(
 	&Config{
 		Name:      "Caddyfile",
 		Aliases:   []string{"caddyfile", "caddy"},
@@ -196,7 +195,7 @@ func caddyfileRules() Rules {
 }
 
 // Caddyfile directive-only lexer.
-var CaddyfileDirectives = internal.Register(MustNewLazyLexer(
+var CaddyfileDirectives = Register(MustNewLexer(
 	&Config{
 		Name:      "Caddyfile Directives",
 		Aliases:   []string{"caddyfile-directives", "caddyfile-d", "caddy-d"},

lexers/cheetah.go

@@ -1,13 +1,11 @@
-package c
+package lexers
 
 import (
-	. "github.com/alecthomas/chroma" // nolint
+	. "github.com/alecthomas/chroma/v2" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-	. "github.com/alecthomas/chroma/lexers/p" // nolint
 )
 
 // Cheetah lexer.
-var Cheetah = internal.Register(MustNewLazyLexer(
+var Cheetah = Register(MustNewLexer(
 	&Config{
 		Name:      "Cheetah",
 		Aliases:   []string{"cheetah", "spitfire"},
@@ -24,9 +22,9 @@ func cheetahRules() Rules {
 			{`#[*](.|\n)*?[*]#`, Comment, nil},
 			{`#end[^#\n]*(?:#|$)`, CommentPreproc, nil},
 			{`#slurp$`, CommentPreproc, nil},
-			{`(#[a-zA-Z]+)([^#\n]*)(#|$)`, ByGroups(CommentPreproc, Using(Python), CommentPreproc), nil},
+			{`(#[a-zA-Z]+)([^#\n]*)(#|$)`, ByGroups(CommentPreproc, Using("Python"), CommentPreproc), nil},
-			{`(\$)([a-zA-Z_][\w.]*\w)`, ByGroups(CommentPreproc, Using(Python)), nil},
+			{`(\$)([a-zA-Z_][\w.]*\w)`, ByGroups(CommentPreproc, Using("Python")), nil},
-			{`(\$\{!?)(.*?)(\})(?s)`, ByGroups(CommentPreproc, Using(Python), CommentPreproc), nil},
+			{`(\$\{!?)(.*?)(\})(?s)`, ByGroups(CommentPreproc, Using("Python"), CommentPreproc), nil},
 			{`(?sx)
                 (.+?)               # anything, followed by:
                 (?:

lexers/circular/doc.go

@@ -1,2 +0,0 @@
-// Package circular exists to break circular dependencies between lexers.
-package circular

lexers/circular/php.go

@@ -1,86 +0,0 @@
-package circular
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// PHP lexer for pure PHP code (not embedded in HTML).
-var PHP = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "PHP",
-		Aliases:         []string{"php", "php3", "php4", "php5"},
-		Filenames:       []string{"*.php", "*.php[345]", "*.inc"},
-		MimeTypes:       []string{"text/x-php"},
-		DotAll:          true,
-		CaseInsensitive: true,
-		EnsureNL:        true,
-	},
-	phpRules,
-))
-
-func phpRules() Rules {
-	return phpCommonRules().Rename("php", "root")
-}
-
-func phpCommonRules() Rules {
-	return Rules{
-		"php": {
-			{`\?>`, CommentPreproc, Pop(1)},
-			{`(<<<)([\'"]?)((?:[\\_a-z]|[^\x00-\x7f])(?:[\\\w]|[^\x00-\x7f])*)(\2\n.*?\n\s*)(\3)(;?)(\n)`, ByGroups(LiteralString, LiteralString, LiteralStringDelimiter, LiteralString, LiteralStringDelimiter, Punctuation, Text), nil},
-			{`\s+`, Text, nil},
-			{`#.*?\n`, CommentSingle, nil},
-			{`//.*?\n`, CommentSingle, nil},
-			{`/\*\*/`, CommentMultiline, nil},
-			{`/\*\*.*?\*/`, LiteralStringDoc, nil},
-			{`/\*.*?\*/`, CommentMultiline, nil},
-			{`(->|::)(\s*)((?:[\\_a-z]|[^\x00-\x7f])(?:[\\\w]|[^\x00-\x7f])*)`, ByGroups(Operator, Text, NameAttribute), nil},
-			{`[~!%^&*+=|:.<>/@-]+`, Operator, nil},
-			{`\?`, Operator, nil},
-			{`[\[\]{}();,]+`, Punctuation, nil},
-			{`(class)(\s+)`, ByGroups(Keyword, Text), Push("classname")},
-			{`(function)(\s*)(?=\()`, ByGroups(Keyword, Text), nil},
-			{`(function)(\s+)(&?)(\s*)`, ByGroups(Keyword, Text, Operator, Text), Push("functionname")},
-			{`(const)(\s+)((?:[\\_a-z]|[^\x00-\x7f])(?:[\\\w]|[^\x00-\x7f])*)`, ByGroups(Keyword, Text, NameConstant), nil},
-			{`(and|E_PARSE|old_function|E_ERROR|or|as|E_WARNING|parent|eval|PHP_OS|break|exit|case|extends|PHP_VERSION|cfunction|FALSE|print|for|require|continue|foreach|require_once|declare|return|default|static|do|switch|die|stdClass|echo|else|TRUE|elseif|var|empty|if|xor|enddeclare|include|virtual|endfor|include_once|while|endforeach|global|endif|list|endswitch|new|endwhile|not|array|E_ALL|NULL|final|php_user_filter|interface|implements|public|private|protected|abstract|clone|try|catch|throw|this|use|namespace|trait|yield|finally)\b`, Keyword, nil},
-			{`(true|false|null)\b`, KeywordConstant, nil},
-			Include("magicconstants"),
-			{`\$\{\$+(?:[\\_a-z]|[^\x00-\x7f])(?:[\\\w]|[^\x00-\x7f])*\}`, NameVariable, nil},
-			{`\$+(?:[\\_a-z]|[^\x00-\x7f])(?:[\\\w]|[^\x00-\x7f])*`, NameVariable, nil},
-			{`(?:[\\_a-z]|[^\x00-\x7f])(?:[\\\w]|[^\x00-\x7f])*`, NameOther, nil},
-			{`(\d+\.\d*|\d*\.\d+)(e[+-]?[0-9]+)?`, LiteralNumberFloat, nil},
-			{`\d+e[+-]?[0-9]+`, LiteralNumberFloat, nil},
-			{`0[0-7]+`, LiteralNumberOct, nil},
-			{`0x[a-f0-9_]+`, LiteralNumberHex, nil},
-			{`\d[\d_]*`, LiteralNumberInteger, nil},
-			{`0b[01]+`, LiteralNumberBin, nil},
-			{`'([^'\\]*(?:\\.[^'\\]*)*)'`, LiteralStringSingle, nil},
-			{"`([^`\\\\]*(?:\\\\.[^`\\\\]*)*)`", LiteralStringBacktick, nil},
-			{`"`, LiteralStringDouble, Push("string")},
-		},
-		"magicfuncs": {
-			{Words(``, `\b`, `__construct`, `__destruct`, `__call`, `__callStatic`, `__get`, `__set`, `__isset`, `__unset`, `__sleep`, `__wakeup`, `__toString`, `__invoke`, `__set_state`, `__clone`, `__debugInfo`), NameFunctionMagic, nil},
-		},
-		"magicconstants": {
-			{Words(``, `\b`, `__LINE__`, `__FILE__`, `__DIR__`, `__FUNCTION__`, `__CLASS__`, `__TRAIT__`, `__METHOD__`, `__NAMESPACE__`), NameConstant, nil},
-		},
-		"classname": {
-			{`(?:[\\_a-z]|[^\x00-\x7f])(?:[\\\w]|[^\x00-\x7f])*`, NameClass, Pop(1)},
-		},
-		"functionname": {
-			Include("magicfuncs"),
-			{`(?:[\\_a-z]|[^\x00-\x7f])(?:[\\\w]|[^\x00-\x7f])*`, NameFunction, Pop(1)},
-			Default(Pop(1)),
-		},
-		"string": {
-			{`"`, LiteralStringDouble, Pop(1)},
-			{`[^{$"\\]+`, LiteralStringDouble, nil},
-			{`\\([nrt"$\\]|[0-7]{1,3}|x[0-9a-f]{1,2})`, LiteralStringEscape, nil},
-			{`\$(?:[\\_a-z]|[^\x00-\x7f])(?:[\\\w]|[^\x00-\x7f])*(\[\S+?\]|->(?:[\\_a-z]|[^\x00-\x7f])(?:[\\\w]|[^\x00-\x7f])*)?`, LiteralStringInterpol, nil},
-			{`(\{\$\{)(.*?)(\}\})`, ByGroups(LiteralStringInterpol, UsingSelf("root"), LiteralStringInterpol), nil},
-			{`(\{)(\$.*?)(\})`, ByGroups(LiteralStringInterpol, UsingSelf("root"), LiteralStringInterpol), nil},
-			{`(\$\{)(\S+)(\})`, ByGroups(LiteralStringInterpol, NameVariable, LiteralStringInterpol), nil},
-			{`[${\\]`, LiteralStringDouble, nil},
-		},
-	}
-}

lexers/cl.go

@@ -1,8 +1,7 @@
-package c
+package lexers
 
 import (
-	. "github.com/alecthomas/chroma" // nolint
+	. "github.com/alecthomas/chroma/v2" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
 )
 
 var (
@@ -230,15 +229,9 @@ var (
 )
 
 // Common Lisp lexer.
-var CommonLisp = internal.Register(TypeRemappingLexer(MustNewLazyLexer(
+var CommonLisp = Register(TypeRemappingLexer(MustNewXMLLexer(
-	&Config{
+	embedded,
-		Name:            "Common Lisp",
+	"embedded/common_lisp.xml",
-		Aliases:         []string{"common-lisp", "cl", "lisp"},
-		Filenames:       []string{"*.cl", "*.lisp"},
-		MimeTypes:       []string{"text/x-common-lisp"},
-		CaseInsensitive: true,
-	},
-	commonLispRules,
 ), TypeMapping{
 	{NameVariable, NameFunction, clBuiltinFunctions},
 	{NameVariable, Keyword, clSpecialForms},
@@ -248,63 +241,3 @@ var CommonLisp = internal.Register(TypeRemappingLexer(MustNewLazyLexer(
 	{NameVariable, KeywordType, clBuiltinTypes},
 	{NameVariable, NameClass, clBuiltinClasses},
 }))
-
-func commonLispRules() Rules {
-	return Rules{
-		"root": {
-			Default(Push("body")),
-		},
-		"multiline-comment": {
-			{`#\|`, CommentMultiline, Push()},
-			{`\|#`, CommentMultiline, Pop(1)},
-			{`[^|#]+`, CommentMultiline, nil},
-			{`[|#]`, CommentMultiline, nil},
-		},
-		"commented-form": {
-			{`\(`, CommentPreproc, Push()},
-			{`\)`, CommentPreproc, Pop(1)},
-			{`[^()]+`, CommentPreproc, nil},
-		},
-		"body": {
-			{`\s+`, Text, nil},
-			{`;.*$`, CommentSingle, nil},
-			{`#\|`, CommentMultiline, Push("multiline-comment")},
-			{`#\d*Y.*$`, CommentSpecial, nil},
-			{`"(\\.|\\\n|[^"\\])*"`, LiteralString, nil},
-			{`:(\|[^|]+\||(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~])(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~]|[#.:])*)`, LiteralStringSymbol, nil},
-			{`::(\|[^|]+\||(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~])(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~]|[#.:])*)`, LiteralStringSymbol, nil},
-			{`:#(\|[^|]+\||(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~])(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~]|[#.:])*)`, LiteralStringSymbol, nil},
-			{`'(\|[^|]+\||(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~])(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~]|[#.:])*)`, LiteralStringSymbol, nil},
-			{`'`, Operator, nil},
-			{"`", Operator, nil},
-			{"[-+]?\\d+\\.?(?=[ \"()\\'\\n,;`])", LiteralNumberInteger, nil},
-			{"[-+]?\\d+/\\d+(?=[ \"()\\'\\n,;`])", LiteralNumber, nil},
-			{"[-+]?(\\d*\\.\\d+([defls][-+]?\\d+)?|\\d+(\\.\\d*)?[defls][-+]?\\d+)(?=[ \"()\\'\\n,;`])", LiteralNumberFloat, nil},
-			{"#\\\\.(?=[ \"()\\'\\n,;`])", LiteralStringChar, nil},
-			{`#\\(\|[^|]+\||(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~])(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~]|[#.:])*)`, LiteralStringChar, nil},
-			{`#\(`, Operator, Push("body")},
-			{`#\d*\*[01]*`, LiteralOther, nil},
-			{`#:(\|[^|]+\||(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~])(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~]|[#.:])*)`, LiteralStringSymbol, nil},
-			{`#[.,]`, Operator, nil},
-			{`#\'`, NameFunction, nil},
-			{`#b[+-]?[01]+(/[01]+)?`, LiteralNumberBin, nil},
-			{`#o[+-]?[0-7]+(/[0-7]+)?`, LiteralNumberOct, nil},
-			{`#x[+-]?[0-9a-f]+(/[0-9a-f]+)?`, LiteralNumberHex, nil},
-			{`#\d+r[+-]?[0-9a-z]+(/[0-9a-z]+)?`, LiteralNumber, nil},
-			{`(#c)(\()`, ByGroups(LiteralNumber, Punctuation), Push("body")},
-			{`(#\d+a)(\()`, ByGroups(LiteralOther, Punctuation), Push("body")},
-			{`(#s)(\()`, ByGroups(LiteralOther, Punctuation), Push("body")},
-			{`#p?"(\\.|[^"])*"`, LiteralOther, nil},
-			{`#\d+=`, Operator, nil},
-			{`#\d+#`, Operator, nil},
-			{"#+nil(?=[ \"()\\'\\n,;`])\\s*\\(", CommentPreproc, Push("commented-form")},
-			{`#[+-]`, Operator, nil},
-			{`(,@|,|\.)`, Operator, nil},
-			{"(t|nil)(?=[ \"()\\'\\n,;`])", NameConstant, nil},
-			{`\*(\|[^|]+\||(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~])(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~]|[#.:])*)\*`, NameVariableGlobal, nil},
-			{`(\|[^|]+\||(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~])(?:\\.|[\w!$%&*+-/<=>?@\[\]^{}~]|[#.:])*)`, NameVariable, nil},
-			{`\(`, Punctuation, Push("body")},
-			{`\)`, Punctuation, Pop(1)},
-		},
-	}
-}

lexers/cpp_test.go

@@ -1,12 +1,12 @@
-package c_test
+package lexers_test
 
 import (
 	"testing"
 
 	"github.com/stretchr/testify/assert"
 
-	"github.com/alecthomas/chroma"
+	"github.com/alecthomas/chroma/v2"
-	"github.com/alecthomas/chroma/lexers/c"
+	"github.com/alecthomas/chroma/v2/lexers"
 )
 
 func TestIssue290(t *testing.T) {
@@ -16,7 +16,7 @@ double b =     0011111111001001100110011001100110011001100110011001100110011010;
 double c =     0011111111010011001100110011001100110011001100110011001100110011; // imperfect representation of 0.3
 double a + b = 0011111111010011001100110011001100110011001100110011001100110100; // Note that this is not quite equal to the "canonical" 0.3!a
 `
-	it, err := c.CPP.Tokenise(nil, input)
+	it, err := lexers.GlobalLexerRegistry.Get("C++").Tokenise(nil, input)
 	assert.NoError(t, err)
 	for {
 		token := it()

lexers/cql.go

@@ -1,12 +1,11 @@
-package c
+package lexers
 
 import (
-	. "github.com/alecthomas/chroma" // nolint
+	. "github.com/alecthomas/chroma/v2" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
 )
 
 // CassandraCQL lexer.
-var CassandraCQL = internal.Register(MustNewLazyLexer(
+var CassandraCQL = Register(MustNewLexer(
 	&Config{
 		Name:            "Cassandra CQL",
 		Aliases:         []string{"cassandra", "cql"},
@@ -29,12 +28,9 @@ func cassandraCQLRules() Rules {
 			{"[+*/<>=~!@#%^&|`?-]+", Operator, nil},
 			{
 				`(?s)(java|javascript)(\s+)(AS)(\s+)('|\$\$)(.*?)(\5)`,
-				UsingByGroup(
+				UsingByGroup(1, 6,
-					internal.Get,
-					1, 6,
 					NameBuiltin, TextWhitespace, Keyword, TextWhitespace,
-					LiteralStringHeredoc, LiteralStringHeredoc, LiteralStringHeredoc,
+					LiteralStringHeredoc, LiteralStringHeredoc, LiteralStringHeredoc),
-				),
 				nil,
 			},
 			{`(true|false|null)\b`, KeywordConstant, nil},

lexers/d/d.go

@@ -1,73 +0,0 @@
-package d
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// D lexer. https://dlang.org/spec/lex.html
-var D = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "D",
-		Aliases:   []string{"d"},
-		Filenames: []string{"*.d", "*.di"},
-		MimeTypes: []string{"text/x-d"},
-		EnsureNL:  true,
-	},
-	dRules,
-))
-
-func dRules() Rules {
-	return Rules{
-		"root": {
-			{`[^\S\n]+`, Text, nil},
-
-			// https://dlang.org/spec/lex.html#comment
-			{`//.*?\n`, CommentSingle, nil},
-			{`/\*.*?\*/`, CommentMultiline, nil},
-			{`/\+.*?\+/`, CommentMultiline, nil},
-
-			// https://dlang.org/spec/lex.html#keywords
-			{`(asm|assert|body|break|case|cast|catch|continue|default|debug|delete|deprecated|do|else|finally|for|foreach|foreach_reverse|goto|if|in|invariant|is|macro|mixin|new|out|pragma|return|super|switch|this|throw|try|version|while|with)\b`, Keyword, nil},
-			{`__(FILE|FILE_FULL_PATH|MODULE|LINE|FUNCTION|PRETTY_FUNCTION|DATE|EOF|TIME|TIMESTAMP|VENDOR|VERSION)__\b`, NameBuiltin, nil},
-			{`__(traits|vector|parameters)\b`, NameBuiltin, nil},
-			{`((?:(?:[^\W\d]|\$)[\w.\[\]$<>]*\s+)+?)((?:[^\W\d]|\$)[\w$]*)(\s*)(\()`, ByGroups(UsingSelf("root"), NameFunction, Text, Operator), nil},
-
-			// https://dlang.org/spec/attribute.html#uda
-			{`@[\w.]*`, NameDecorator, nil},
-			{`(abstract|auto|alias|align|const|delegate|enum|export|final|function|inout|lazy|nothrow|override|package|private|protected|public|pure|static|synchronized|template|volatile|__gshared)\b`, KeywordDeclaration, nil},
-
-			// https://dlang.org/spec/type.html#basic-data-types
-			{`(void|bool|byte|ubyte|short|ushort|int|uint|long|ulong|cent|ucent|float|double|real|ifloat|idouble|ireal|cfloat|cdouble|creal|char|wchar|dchar|string|wstring|dstring)\b`, KeywordType, nil},
-			{`(module)(\s+)`, ByGroups(KeywordNamespace, Text), Push("import")},
-			{`(true|false|null)\b`, KeywordConstant, nil},
-			{`(class|interface|struct|template|union)(\s+)`, ByGroups(KeywordDeclaration, Text), Push("class")},
-			{`(import)(\s+)`, ByGroups(KeywordNamespace, Text), Push("import")},
-
-			// https://dlang.org/spec/lex.html#string_literals
-			// TODO support delimited strings
-			{`[qr]?"(\\\\|\\"|[^"])*"[cwd]?`, LiteralString, nil},
-			{"(`)([^`]*)(`)[cwd]?", LiteralString, nil},
-			{`'\\.'|'[^\\]'|'\\u[0-9a-fA-F]{4}'`, LiteralStringChar, nil},
-			{`(\.)((?:[^\W\d]|\$)[\w$]*)`, ByGroups(Operator, NameAttribute), nil},
-			{`^\s*([^\W\d]|\$)[\w$]*:`, NameLabel, nil},
-
-			// https://dlang.org/spec/lex.html#floatliteral
-			{`([0-9][0-9_]*\.([0-9][0-9_]*)?|\.[0-9][0-9_]*)([eE][+\-]?[0-9][0-9_]*)?[fFL]?i?|[0-9][eE][+\-]?[0-9][0-9_]*[fFL]?|[0-9]([eE][+\-]?[0-9][0-9_]*)?[fFL]|0[xX]([0-9a-fA-F][0-9a-fA-F_]*\.?|([0-9a-fA-F][0-9a-fA-F_]*)?\.[0-9a-fA-F][0-9a-fA-F_]*)[pP][+\-]?[0-9][0-9_]*[fFL]?`, LiteralNumberFloat, nil},
-			// https://dlang.org/spec/lex.html#integerliteral
-			{`0[xX][0-9a-fA-F][0-9a-fA-F_]*[lL]?`, LiteralNumberHex, nil},
-			{`0[bB][01][01_]*[lL]?`, LiteralNumberBin, nil},
-			{`0[0-7_]+[lL]?`, LiteralNumberOct, nil},
-			{`0|[1-9][0-9_]*[lL]?`, LiteralNumberInteger, nil},
-			{`([~^*!%&\[\](){}<>|+=:;,./?-]|q{)`, Operator, nil},
-			{`([^\W\d]|\$)[\w$]*`, Name, nil},
-			{`\n`, Text, nil},
-		},
-		"class": {
-			{`([^\W\d]|\$)[\w$]*`, NameClass, Pop(1)},
-		},
-		"import": {
-			{`[\w.]+\*?`, NameNamespace, Pop(1)},
-		},
-	}
-}

lexers/d/dart.go

@@ -1,95 +0,0 @@
-package d
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Dart lexer.
-var Dart = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Dart",
-		Aliases:   []string{"dart"},
-		Filenames: []string{"*.dart"},
-		MimeTypes: []string{"text/x-dart"},
-		DotAll:    true,
-	},
-	dartRules,
-))
-
-func dartRules() Rules {
-	return Rules{
-		"root": {
-			Include("string_literal"),
-			{`#!(.*?)$`, CommentPreproc, nil},
-			{`\b(import|export)\b`, Keyword, Push("import_decl")},
-			{`\b(library|source|part of|part)\b`, Keyword, nil},
-			{`[^\S\n]+`, Text, nil},
-			{`//.*?\n`, CommentSingle, nil},
-			{`/\*.*?\*/`, CommentMultiline, nil},
-			{`\b(class)\b(\s+)`, ByGroups(KeywordDeclaration, Text), Push("class")},
-			{`\b(assert|break|case|catch|continue|default|do|else|finally|for|if|in|is|new|return|super|switch|this|throw|try|while)\b`, Keyword, nil},
-			{`\b(abstract|async|await|const|extends|factory|final|get|implements|native|operator|set|static|sync|typedef|var|with|yield)\b`, KeywordDeclaration, nil},
-			{`\b(bool|double|dynamic|int|num|Object|String|void)\b`, KeywordType, nil},
-			{`\b(false|null|true)\b`, KeywordConstant, nil},
-			{`[~!%^&*+=|?:<>/-]|as\b`, Operator, nil},
-			{`[a-zA-Z_$]\w*:`, NameLabel, nil},
-			{`[a-zA-Z_$]\w*`, Name, nil},
-			{`[(){}\[\],.;]`, Punctuation, nil},
-			{`0[xX][0-9a-fA-F]+`, LiteralNumberHex, nil},
-			{`\d+(\.\d*)?([eE][+-]?\d+)?`, LiteralNumber, nil},
-			{`\.\d+([eE][+-]?\d+)?`, LiteralNumber, nil},
-			{`\n`, Text, nil},
-		},
-		"class": {
-			{`[a-zA-Z_$]\w*`, NameClass, Pop(1)},
-		},
-		"import_decl": {
-			Include("string_literal"),
-			{`\s+`, Text, nil},
-			{`\b(as|show|hide)\b`, Keyword, nil},
-			{`[a-zA-Z_$]\w*`, Name, nil},
-			{`\,`, Punctuation, nil},
-			{`\;`, Punctuation, Pop(1)},
-		},
-		"string_literal": {
-			{`r"""([\w\W]*?)"""`, LiteralStringDouble, nil},
-			{`r'''([\w\W]*?)'''`, LiteralStringSingle, nil},
-			{`r"(.*?)"`, LiteralStringDouble, nil},
-			{`r'(.*?)'`, LiteralStringSingle, nil},
-			{`"""`, LiteralStringDouble, Push("string_double_multiline")},
-			{`'''`, LiteralStringSingle, Push("string_single_multiline")},
-			{`"`, LiteralStringDouble, Push("string_double")},
-			{`'`, LiteralStringSingle, Push("string_single")},
-		},
-		"string_common": {
-			{`\\(x[0-9A-Fa-f]{2}|u[0-9A-Fa-f]{4}|u\{[0-9A-Fa-f]*\}|[a-z'\"$\\])`, LiteralStringEscape, nil},
-			{`(\$)([a-zA-Z_]\w*)`, ByGroups(LiteralStringInterpol, Name), nil},
-			{`(\$\{)(.*?)(\})`, ByGroups(LiteralStringInterpol, UsingSelf("root"), LiteralStringInterpol), nil},
-		},
-		"string_double": {
-			{`"`, LiteralStringDouble, Pop(1)},
-			{`[^"$\\\n]+`, LiteralStringDouble, nil},
-			Include("string_common"),
-			{`\$+`, LiteralStringDouble, nil},
-		},
-		"string_double_multiline": {
-			{`"""`, LiteralStringDouble, Pop(1)},
-			{`[^"$\\]+`, LiteralStringDouble, nil},
-			Include("string_common"),
-			{`(\$|\")+`, LiteralStringDouble, nil},
-		},
-		"string_single": {
-			{`'`, LiteralStringSingle, Pop(1)},
-			{`[^'$\\\n]+`, LiteralStringSingle, nil},
-			Include("string_common"),
-			{`\$+`, LiteralStringSingle, nil},
-		},
-		"string_single_multiline": {
-			{`'''`, LiteralStringSingle, Pop(1)},
-			{`[^\'$\\]+`, LiteralStringSingle, nil},
-			Include("string_common"),
-			{`(\$|\')+`, LiteralStringSingle, nil},
-		},
-	}
-}

lexers/d/diff.go

@@ -1,33 +0,0 @@
-package d
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Diff lexer.
-var Diff = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Diff",
-		Aliases:   []string{"diff", "udiff"},
-		EnsureNL:  true,
-		Filenames: []string{"*.diff", "*.patch"},
-		MimeTypes: []string{"text/x-diff", "text/x-patch"},
-	},
-	diffRules,
-))
-
-func diffRules() Rules {
-	return Rules{
-		"root": {
-			{` .*\n`, Text, nil},
-			{`\+.*\n`, GenericInserted, nil},
-			{`-.*\n`, GenericDeleted, nil},
-			{`!.*\n`, GenericStrong, nil},
-			{`@.*\n`, GenericSubheading, nil},
-			{`([Ii]ndex|diff).*\n`, GenericHeading, nil},
-			{`=.*\n`, GenericHeading, nil},
-			{`.*\n`, Text, nil},
-		},
-	}
-}

lexers/d/django.go

@@ -1,57 +0,0 @@
-package d
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Django/Jinja lexer.
-var DjangoJinja = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Django/Jinja",
-		Aliases:   []string{"django", "jinja"},
-		Filenames: []string{},
-		MimeTypes: []string{"application/x-django-templating", "application/x-jinja"},
-		DotAll:    true,
-	},
-	djangoJinjaRules,
-))
-
-func djangoJinjaRules() Rules {
-	return Rules{
-		"root": {
-			{`[^{]+`, Other, nil},
-			{`\{\{`, CommentPreproc, Push("var")},
-			{`\{[*#].*?[*#]\}`, Comment, nil},
-			{`(\{%)(-?\s*)(comment)(\s*-?)(%\})(.*?)(\{%)(-?\s*)(endcomment)(\s*-?)(%\})`, ByGroups(CommentPreproc, Text, Keyword, Text, CommentPreproc, Comment, CommentPreproc, Text, Keyword, Text, CommentPreproc), nil},
-			{`(\{%)(-?\s*)(raw)(\s*-?)(%\})(.*?)(\{%)(-?\s*)(endraw)(\s*-?)(%\})`, ByGroups(CommentPreproc, Text, Keyword, Text, CommentPreproc, Text, CommentPreproc, Text, Keyword, Text, CommentPreproc), nil},
-			{`(\{%)(-?\s*)(filter)(\s+)([a-zA-Z_]\w*)`, ByGroups(CommentPreproc, Text, Keyword, Text, NameFunction), Push("block")},
-			{`(\{%)(-?\s*)([a-zA-Z_]\w*)`, ByGroups(CommentPreproc, Text, Keyword), Push("block")},
-			{`\{`, Other, nil},
-		},
-		"varnames": {
-			{`(\|)(\s*)([a-zA-Z_]\w*)`, ByGroups(Operator, Text, NameFunction), nil},
-			{`(is)(\s+)(not)?(\s+)?([a-zA-Z_]\w*)`, ByGroups(Keyword, Text, Keyword, Text, NameFunction), nil},
-			{`(_|true|false|none|True|False|None)\b`, KeywordPseudo, nil},
-			{`(in|as|reversed|recursive|not|and|or|is|if|else|import|with(?:(?:out)?\s*context)?|scoped|ignore\s+missing)\b`, Keyword, nil},
-			{`(loop|block|super|forloop)\b`, NameBuiltin, nil},
-			{`[a-zA-Z_][\w-]*`, NameVariable, nil},
-			{`\.\w+`, NameVariable, nil},
-			{`:?"(\\\\|\\"|[^"])*"`, LiteralStringDouble, nil},
-			{`:?'(\\\\|\\'|[^'])*'`, LiteralStringSingle, nil},
-			{`([{}()\[\]+\-*/,:~]|[><=]=?)`, Operator, nil},
-			{`[0-9](\.[0-9]*)?(eE[+-][0-9])?[flFLdD]?|0[xX][0-9a-fA-F]+[Ll]?`, LiteralNumber, nil},
-		},
-		"var": {
-			{`\s+`, Text, nil},
-			{`(-?)(\}\})`, ByGroups(Text, CommentPreproc), Pop(1)},
-			Include("varnames"),
-		},
-		"block": {
-			{`\s+`, Text, nil},
-			{`(-?)(%\})`, ByGroups(Text, CommentPreproc), Pop(1)},
-			Include("varnames"),
-			{`.`, Punctuation, nil},
-		},
-	}
-}

lexers/d/dtd.go

@@ -1,73 +0,0 @@
-package d
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Dtd lexer.
-var Dtd = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "DTD",
-		Aliases:   []string{"dtd"},
-		Filenames: []string{"*.dtd"},
-		MimeTypes: []string{"application/xml-dtd"},
-		DotAll:    true,
-	},
-	dtdRules,
-))
-
-func dtdRules() Rules {
-	return Rules{
-		"root": {
-			Include("common"),
-			{`(<!ELEMENT)(\s+)(\S+)`, ByGroups(Keyword, Text, NameTag), Push("element")},
-			{`(<!ATTLIST)(\s+)(\S+)`, ByGroups(Keyword, Text, NameTag), Push("attlist")},
-			{`(<!ENTITY)(\s+)(\S+)`, ByGroups(Keyword, Text, NameEntity), Push("entity")},
-			{`(<!NOTATION)(\s+)(\S+)`, ByGroups(Keyword, Text, NameTag), Push("notation")},
-			{`(<!\[)([^\[\s]+)(\s*)(\[)`, ByGroups(Keyword, NameEntity, Text, Keyword), nil},
-			{`(<!DOCTYPE)(\s+)([^>\s]+)`, ByGroups(Keyword, Text, NameTag), nil},
-			{`PUBLIC|SYSTEM`, KeywordConstant, nil},
-			{`[\[\]>]`, Keyword, nil},
-		},
-		"common": {
-			{`\s+`, Text, nil},
-			{`(%|&)[^;]*;`, NameEntity, nil},
-			{`<!--`, Comment, Push("comment")},
-			{`[(|)*,?+]`, Operator, nil},
-			{`"[^"]*"`, LiteralStringDouble, nil},
-			{`\'[^\']*\'`, LiteralStringSingle, nil},
-		},
-		"comment": {
-			{`[^-]+`, Comment, nil},
-			{`-->`, Comment, Pop(1)},
-			{`-`, Comment, nil},
-		},
-		"element": {
-			Include("common"),
-			{`EMPTY|ANY|#PCDATA`, KeywordConstant, nil},
-			{`[^>\s|()?+*,]+`, NameTag, nil},
-			{`>`, Keyword, Pop(1)},
-		},
-		"attlist": {
-			Include("common"),
-			{`CDATA|IDREFS|IDREF|ID|NMTOKENS|NMTOKEN|ENTITIES|ENTITY|NOTATION`, KeywordConstant, nil},
-			{`#REQUIRED|#IMPLIED|#FIXED`, KeywordConstant, nil},
-			{`xml:space|xml:lang`, KeywordReserved, nil},
-			{`[^>\s|()?+*,]+`, NameAttribute, nil},
-			{`>`, Keyword, Pop(1)},
-		},
-		"entity": {
-			Include("common"),
-			{`SYSTEM|PUBLIC|NDATA`, KeywordConstant, nil},
-			{`[^>\s|()?+*,]+`, NameEntity, nil},
-			{`>`, Keyword, Pop(1)},
-		},
-		"notation": {
-			Include("common"),
-			{`SYSTEM|PUBLIC`, KeywordConstant, nil},
-			{`[^>\s|()?+*,]+`, NameAttribute, nil},
-			{`>`, Keyword, Pop(1)},
-		},
-	}
-}

lexers/d/dylan.go

@@ -1,76 +0,0 @@
-package d
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Dylan lexer.
-var Dylan = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:            "Dylan",
-		Aliases:         []string{"dylan"},
-		Filenames:       []string{"*.dylan", "*.dyl", "*.intr"},
-		MimeTypes:       []string{"text/x-dylan"},
-		CaseInsensitive: true,
-	},
-	func() Rules {
-		return Rules{
-			"root": {
-				{`\s+`, Whitespace, nil},
-				{`//.*?\n`, CommentSingle, nil},
-				{`([a-z0-9-]+:)([ \t]*)(.*(?:\n[ \t].+)*)`, ByGroups(NameAttribute, Whitespace, LiteralString), nil},
-				Default(Push("code")),
-			},
-			"code": {
-				{`\s+`, Whitespace, nil},
-				{`//.*?\n`, CommentSingle, nil},
-				{`/\*`, CommentMultiline, Push("comment")},
-				{`"`, LiteralString, Push("string")},
-				{`'(\\.|\\[0-7]{1,3}|\\x[a-f0-9]{1,2}|[^\\\'\n])'`, LiteralStringChar, nil},
-				{`#b[01]+`, LiteralNumberBin, nil},
-				{`#o[0-7]+`, LiteralNumberOct, nil},
-				{`[-+]?(\d*\.\d+([ed][-+]?\d+)?|\d+(\.\d*)?e[-+]?\d+)`, LiteralNumberFloat, nil},
-				{`[-+]?\d+`, LiteralNumberInteger, nil},
-				{`#x[0-9a-f]+`, LiteralNumberHex, nil},
-
-				{`(\?\\?)([\w!&*<>|^$%@+~?/=-]+)(:)(token|name|variable|expression|body|case-body|\*)`,
-					ByGroups(Operator, NameVariable, Operator, NameBuiltin), nil},
-				{`(\?)(:)(token|name|variable|expression|body|case-body|\*)`,
-					ByGroups(Operator, Operator, NameVariable), nil},
-				{`(\?\\?)([\w!&*<>|^$%@+~?/=-]+)`, ByGroups(Operator, NameVariable), nil},
-
-				{`(=>|::|#\(|#\[|##|\?\?|\?=|\?|[(){}\[\],.;])`, Punctuation, nil},
-				{`:=`, Operator, nil},
-				{`#[tf]`, Literal, nil},
-				{`#"`, LiteralStringSymbol, Push("symbol")},
-				{`#[a-z0-9-]+`, Keyword, nil},
-				{`#(all-keys|include|key|next|rest)`, Keyword, nil},
-				{`[\w!&*<>|^$%@+~?/=-]+:`, KeywordConstant, nil},
-				{`<[\w!&*<>|^$%@+~?/=-]+>`, NameClass, nil},
-				{`\*[\w!&*<>|^$%@+~?/=-]+\*`, NameVariableGlobal, nil},
-				{`\$[\w!&*<>|^$%@+~?/=-]+`, NameConstant, nil},
-				{`(let|method|function)([ \t]+)([\w!&*<>|^$%@+~?/=-]+)`, ByGroups(NameBuiltin, Whitespace, NameVariable), nil},
-				{`(error|signal|return|break)`, NameException, nil},
-				{`(\\?)([\w!&*<>|^$%@+~?/=-]+)`, ByGroups(Operator, Name), nil},
-			},
-			"comment": {
-				{`[^*/]`, CommentMultiline, nil},
-				{`/\*`, CommentMultiline, Push()},
-				{`\*/`, CommentMultiline, Pop(1)},
-				{`[*/]`, CommentMultiline, nil},
-			},
-			"symbol": {
-				{`"`, LiteralStringSymbol, Pop(1)},
-				{`[^\\"]+`, LiteralStringSymbol, nil},
-			},
-			"string": {
-				{`"`, LiteralString, Pop(1)},
-				{`\\([\\abfnrtv"\']|x[a-f0-9]{2,4}|[0-7]{1,3})`, LiteralStringEscape, nil},
-				{`[^\\"\n]+`, LiteralString, nil},
-				{`\\\n`, LiteralString, nil},
-				{`\\`, LiteralString, nil},
-			},
-		}
-	},
-))

lexers/docker.go

@@ -1,14 +1,11 @@
-package d
+package lexers
 
 import (
-	. "github.com/alecthomas/chroma" // nolint
+	. "github.com/alecthomas/chroma/v2" // nolint
-	"github.com/alecthomas/chroma/lexers/b"
-	"github.com/alecthomas/chroma/lexers/internal"
-	"github.com/alecthomas/chroma/lexers/j"
 )
 
 // Docker lexer.
-var Docker = internal.Register(MustNewLazyLexer(
+var Docker = Register(MustNewLexer(
 	&Config{
 		Name:            "Docker",
 		Aliases:         []string{"docker", "dockerfile"},
@@ -23,13 +20,13 @@ func dockerRules() Rules {
 	return Rules{
 		"root": {
 			{`#.*`, Comment, nil},
-			{`(ONBUILD)((?:\s*\\?\s*))`, ByGroups(Keyword, Using(b.Bash)), nil},
+			{`(ONBUILD)((?:\s*\\?\s*))`, ByGroups(Keyword, Using("Bash")), nil},
-			{`(HEALTHCHECK)(((?:\s*\\?\s*)--\w+=\w+(?:\s*\\?\s*))*)`, ByGroups(Keyword, Using(b.Bash)), nil},
+			{`(HEALTHCHECK)(((?:\s*\\?\s*)--\w+=\w+(?:\s*\\?\s*))*)`, ByGroups(Keyword, Using("Bash")), nil},
-			{`(VOLUME|ENTRYPOINT|CMD|SHELL)((?:\s*\\?\s*))(\[.*?\])`, ByGroups(Keyword, Using(b.Bash), Using(j.JSON)), nil},
+			{`(VOLUME|ENTRYPOINT|CMD|SHELL)((?:\s*\\?\s*))(\[.*?\])`, ByGroups(Keyword, Using("Bash"), Using("JSON")), nil},
-			{`(LABEL|ENV|ARG)((?:(?:\s*\\?\s*)\w+=\w+(?:\s*\\?\s*))*)`, ByGroups(Keyword, Using(b.Bash)), nil},
+			{`(LABEL|ENV|ARG)((?:(?:\s*\\?\s*)\w+=\w+(?:\s*\\?\s*))*)`, ByGroups(Keyword, Using("Bash")), nil},
 			{`((?:FROM|MAINTAINER|EXPOSE|WORKDIR|USER|STOPSIGNAL)|VOLUME)\b(.*)`, ByGroups(Keyword, LiteralString), nil},
 			{`((?:RUN|CMD|ENTRYPOINT|ENV|ARG|LABEL|ADD|COPY))`, Keyword, nil},
-			{`(.*\\\n)*.+`, Using(b.Bash), nil},
+			{`(.*\\\n)*.+`, Using("Bash"), nil},
 		},
 	}
 }

lexers/e/ebnf.go

@@ -1,55 +0,0 @@
-package e
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Ebnf lexer.
-var Ebnf = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "EBNF",
-		Aliases:   []string{"ebnf"},
-		Filenames: []string{"*.ebnf"},
-		MimeTypes: []string{"text/x-ebnf"},
-	},
-	ebnfRules,
-))
-
-func ebnfRules() Rules {
-	return Rules{
-		"root": {
-			Include("whitespace"),
-			Include("comment_start"),
-			Include("identifier"),
-			{`=`, Operator, Push("production")},
-		},
-		"production": {
-			Include("whitespace"),
-			Include("comment_start"),
-			Include("identifier"),
-			{`"[^"]*"`, LiteralStringDouble, nil},
-			{`'[^']*'`, LiteralStringSingle, nil},
-			{`(\?[^?]*\?)`, NameEntity, nil},
-			{`[\[\]{}(),|]`, Punctuation, nil},
-			{`-`, Operator, nil},
-			{`;`, Punctuation, Pop(1)},
-			{`\.`, Punctuation, Pop(1)},
-		},
-		"whitespace": {
-			{`\s+`, Text, nil},
-		},
-		"comment_start": {
-			{`\(\*`, CommentMultiline, Push("comment")},
-		},
-		"comment": {
-			{`[^*)]`, CommentMultiline, nil},
-			Include("comment_start"),
-			{`\*\)`, CommentMultiline, Pop(1)},
-			{`[*)]`, CommentMultiline, nil},
-		},
-		"identifier": {
-			{`([a-zA-Z][\w \-]*)`, Keyword, nil},
-		},
-	}
-}

lexers/e/elixir.go

@@ -1,281 +0,0 @@
-package e
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Elixir lexer.
-var Elixir = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Elixir",
-		Aliases:   []string{"elixir", "ex", "exs"},
-		Filenames: []string{"*.ex", "*.exs"},
-		MimeTypes: []string{"text/x-elixir"},
-	},
-	elixirRules,
-))
-
-func elixirRules() Rules {
-	return Rules{
-		"root": {
-			{`\s+`, Text, nil},
-			{`#.*$`, CommentSingle, nil},
-			{`(\?)(\\x\{)([\da-fA-F]+)(\})`, ByGroups(LiteralStringChar, LiteralStringEscape, LiteralNumberHex, LiteralStringEscape), nil},
-			{`(\?)(\\x[\da-fA-F]{1,2})`, ByGroups(LiteralStringChar, LiteralStringEscape), nil},
-			{`(\?)(\\[abdefnrstv])`, ByGroups(LiteralStringChar, LiteralStringEscape), nil},
-			{`\?\\?.`, LiteralStringChar, nil},
-			{`:::`, LiteralStringSymbol, nil},
-			{`::`, Operator, nil},
-			{`:(?:\.\.\.|<<>>|%\{\}|%|\{\})`, LiteralStringSymbol, nil},
-			{`:(?:(?:\.\.\.|[a-z_]\w*[!?]?)|[A-Z]\w*(?:\.[A-Z]\w*)*|(?:\<\<\<|\>\>\>|\|\|\||\&\&\&|\^\^\^|\~\~\~|\=\=\=|\!\=\=|\~\>\>|\<\~\>|\|\~\>|\<\|\>|\=\=|\!\=|\<\=|\>\=|\&\&|\|\||\<\>|\+\+|\-\-|\|\>|\=\~|\-\>|\<\-|\||\.|\=|\~\>|\<\~|\<|\>|\+|\-|\*|\/|\!|\^|\&))`, LiteralStringSymbol, nil},
-			{`:"`, LiteralStringSymbol, Push("string_double_atom")},
-			{`:'`, LiteralStringSymbol, Push("string_single_atom")},
-			{`((?:\.\.\.|<<>>|%\{\}|%|\{\})|(?:(?:\.\.\.|[a-z_]\w*[!?]?)|[A-Z]\w*(?:\.[A-Z]\w*)*|(?:\<\<\<|\>\>\>|\|\|\||\&\&\&|\^\^\^|\~\~\~|\=\=\=|\!\=\=|\~\>\>|\<\~\>|\|\~\>|\<\|\>|\=\=|\!\=|\<\=|\>\=|\&\&|\|\||\<\>|\+\+|\-\-|\|\>|\=\~|\-\>|\<\-|\||\.|\=|\~\>|\<\~|\<|\>|\+|\-|\*|\/|\!|\^|\&)))(:)(?=\s|\n)`, ByGroups(LiteralStringSymbol, Punctuation), nil},
-			{`(fn|do|end|after|else|rescue|catch)\b`, Keyword, nil},
-			{`(not|and|or|when|in)\b`, OperatorWord, nil},
-			{`(case|cond|for|if|unless|try|receive|raise|quote|unquote|unquote_splicing|throw|super|while)\b`, Keyword, nil},
-			{`(def|defp|defmodule|defprotocol|defmacro|defmacrop|defdelegate|defexception|defstruct|defimpl|defcallback)\b`, KeywordDeclaration, nil},
-			{`(import|require|use|alias)\b`, KeywordNamespace, nil},
-			{`(nil|true|false)\b`, NameConstant, nil},
-			{`(_|__MODULE__|__DIR__|__ENV__|__CALLER__)\b`, NamePseudo, nil},
-			{`@(?:\.\.\.|[a-z_]\w*[!?]?)`, NameAttribute, nil},
-			{`(?:\.\.\.|[a-z_]\w*[!?]?)`, Name, nil},
-			{`(%?)([A-Z]\w*(?:\.[A-Z]\w*)*)`, ByGroups(Punctuation, NameClass), nil},
-			{`\<\<\<|\>\>\>|\|\|\||\&\&\&|\^\^\^|\~\~\~|\=\=\=|\!\=\=|\~\>\>|\<\~\>|\|\~\>|\<\|\>`, Operator, nil},
-			{`\=\=|\!\=|\<\=|\>\=|\&\&|\|\||\<\>|\+\+|\-\-|\|\>|\=\~|\-\>|\<\-|\||\.|\=|\~\>|\<\~`, Operator, nil},
-			{`\\\\|\<\<|\>\>|\=\>|\(|\)|\:|\;|\,|\[|\]`, Punctuation, nil},
-			{`&\d`, NameEntity, nil},
-			{`\<|\>|\+|\-|\*|\/|\!|\^|\&`, Operator, nil},
-			{`0b[01](_?[01])*`, LiteralNumberBin, nil},
-			{`0o[0-7](_?[0-7])*`, LiteralNumberOct, nil},
-			{`0x[\da-fA-F](_?[\dA-Fa-f])*`, LiteralNumberHex, nil},
-			{`\d(_?\d)*\.\d(_?\d)*([eE][-+]?\d(_?\d)*)?`, LiteralNumberFloat, nil},
-			{`\d(_?\d)*`, LiteralNumberInteger, nil},
-			{`"""\s*`, LiteralStringHeredoc, Push("heredoc_double")},
-			{`'''\s*$`, LiteralStringHeredoc, Push("heredoc_single")},
-			{`"`, LiteralStringDouble, Push("string_double")},
-			{`'`, LiteralStringSingle, Push("string_single")},
-			Include("sigils"),
-			{`%\{`, Punctuation, Push("map_key")},
-			{`\{`, Punctuation, Push("tuple")},
-		},
-		"heredoc_double": {
-			{`^\s*"""`, LiteralStringHeredoc, Pop(1)},
-			Include("heredoc_interpol"),
-		},
-		"heredoc_single": {
-			{`^\s*'''`, LiteralStringHeredoc, Pop(1)},
-			Include("heredoc_interpol"),
-		},
-		"heredoc_interpol": {
-			{`[^#\\\n]+`, LiteralStringHeredoc, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringHeredoc, nil},
-			{`\n+`, LiteralStringHeredoc, nil},
-			Include("interpol"),
-		},
-		"heredoc_no_interpol": {
-			{`[^\\\n]+`, LiteralStringHeredoc, nil},
-			{`\\.`, LiteralStringHeredoc, nil},
-			{`\n+`, LiteralStringHeredoc, nil},
-		},
-		"escapes": {
-			{`(\\x\{)([\da-fA-F]+)(\})`, ByGroups(LiteralStringEscape, LiteralNumberHex, LiteralStringEscape), nil},
-			{`(\\x[\da-fA-F]{1,2})`, LiteralStringEscape, nil},
-			{`(\\[abdefnrstv])`, LiteralStringEscape, nil},
-		},
-		"interpol": {
-			{`#\{`, LiteralStringInterpol, Push("interpol_string")},
-		},
-		"interpol_string": {
-			{`\}`, LiteralStringInterpol, Pop(1)},
-			Include("root"),
-		},
-		"map_key": {
-			Include("root"),
-			{`:`, Punctuation, Push("map_val")},
-			{`=>`, Punctuation, Push("map_val")},
-			{`\}`, Punctuation, Pop(1)},
-		},
-		"map_val": {
-			Include("root"),
-			{`,`, Punctuation, Pop(1)},
-			{`(?=\})`, Punctuation, Pop(1)},
-		},
-		"tuple": {
-			Include("root"),
-			{`\}`, Punctuation, Pop(1)},
-		},
-		"string_double": {
-			{`[^#"\\]+`, LiteralStringDouble, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringDouble, nil},
-			{`(")`, ByGroups(LiteralStringDouble), Pop(1)},
-			Include("interpol"),
-		},
-		"string_single": {
-			{`[^#'\\]+`, LiteralStringSingle, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringSingle, nil},
-			{`(')`, ByGroups(LiteralStringSingle), Pop(1)},
-			Include("interpol"),
-		},
-		"string_double_atom": {
-			{`[^#"\\]+`, LiteralStringSymbol, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringSymbol, nil},
-			{`(")`, ByGroups(LiteralStringSymbol), Pop(1)},
-			Include("interpol"),
-		},
-		"string_single_atom": {
-			{`[^#'\\]+`, LiteralStringSymbol, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringSymbol, nil},
-			{`(')`, ByGroups(LiteralStringSymbol), Pop(1)},
-			Include("interpol"),
-		},
-		"sigils": {
-			{`(~[a-z])(""")`, ByGroups(LiteralStringOther, LiteralStringHeredoc), Push("triquot-end", "triquot-intp")},
-			{`(~[A-Z])(""")`, ByGroups(LiteralStringOther, LiteralStringHeredoc), Push("triquot-end", "triquot-no-intp")},
-			{`(~[a-z])(''')`, ByGroups(LiteralStringOther, LiteralStringHeredoc), Push("triapos-end", "triapos-intp")},
-			{`(~[A-Z])(''')`, ByGroups(LiteralStringOther, LiteralStringHeredoc), Push("triapos-end", "triapos-no-intp")},
-			{`~[a-z]\{`, LiteralStringOther, Push("cb-intp")},
-			{`~[A-Z]\{`, LiteralStringOther, Push("cb-no-intp")},
-			{`~[a-z]\[`, LiteralStringOther, Push("sb-intp")},
-			{`~[A-Z]\[`, LiteralStringOther, Push("sb-no-intp")},
-			{`~[a-z]\(`, LiteralStringOther, Push("pa-intp")},
-			{`~[A-Z]\(`, LiteralStringOther, Push("pa-no-intp")},
-			{`~[a-z]<`, LiteralStringOther, Push("ab-intp")},
-			{`~[A-Z]<`, LiteralStringOther, Push("ab-no-intp")},
-			{`~[a-z]/`, LiteralStringOther, Push("slas-intp")},
-			{`~[A-Z]/`, LiteralStringOther, Push("slas-no-intp")},
-			{`~[a-z]\|`, LiteralStringOther, Push("pipe-intp")},
-			{`~[A-Z]\|`, LiteralStringOther, Push("pipe-no-intp")},
-			{`~[a-z]"`, LiteralStringOther, Push("quot-intp")},
-			{`~[A-Z]"`, LiteralStringOther, Push("quot-no-intp")},
-			{`~[a-z]'`, LiteralStringOther, Push("apos-intp")},
-			{`~[A-Z]'`, LiteralStringOther, Push("apos-no-intp")},
-		},
-		"triquot-end": {
-			{`[a-zA-Z]+`, LiteralStringOther, Pop(1)},
-			Default(Pop(1)),
-		},
-		"triquot-intp": {
-			{`^\s*"""`, LiteralStringHeredoc, Pop(1)},
-			Include("heredoc_interpol"),
-		},
-		"triquot-no-intp": {
-			{`^\s*"""`, LiteralStringHeredoc, Pop(1)},
-			Include("heredoc_no_interpol"),
-		},
-		"triapos-end": {
-			{`[a-zA-Z]+`, LiteralStringOther, Pop(1)},
-			Default(Pop(1)),
-		},
-		"triapos-intp": {
-			{`^\s*'''`, LiteralStringHeredoc, Pop(1)},
-			Include("heredoc_interpol"),
-		},
-		"triapos-no-intp": {
-			{`^\s*'''`, LiteralStringHeredoc, Pop(1)},
-			Include("heredoc_no_interpol"),
-		},
-		"cb-intp": {
-			{`[^#\}\\]+`, LiteralStringOther, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringOther, nil},
-			{`\}[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-			Include("interpol"),
-		},
-		"cb-no-intp": {
-			{`[^\}\\]+`, LiteralStringOther, nil},
-			{`\\.`, LiteralStringOther, nil},
-			{`\}[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-		},
-		"sb-intp": {
-			{`[^#\]\\]+`, LiteralStringOther, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringOther, nil},
-			{`\][a-zA-Z]*`, LiteralStringOther, Pop(1)},
-			Include("interpol"),
-		},
-		"sb-no-intp": {
-			{`[^\]\\]+`, LiteralStringOther, nil},
-			{`\\.`, LiteralStringOther, nil},
-			{`\][a-zA-Z]*`, LiteralStringOther, Pop(1)},
-		},
-		"pa-intp": {
-			{`[^#\)\\]+`, LiteralStringOther, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringOther, nil},
-			{`\)[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-			Include("interpol"),
-		},
-		"pa-no-intp": {
-			{`[^\)\\]+`, LiteralStringOther, nil},
-			{`\\.`, LiteralStringOther, nil},
-			{`\)[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-		},
-		"ab-intp": {
-			{`[^#>\\]+`, LiteralStringOther, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringOther, nil},
-			{`>[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-			Include("interpol"),
-		},
-		"ab-no-intp": {
-			{`[^>\\]+`, LiteralStringOther, nil},
-			{`\\.`, LiteralStringOther, nil},
-			{`>[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-		},
-		"slas-intp": {
-			{`[^#/\\]+`, LiteralStringOther, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringOther, nil},
-			{`/[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-			Include("interpol"),
-		},
-		"slas-no-intp": {
-			{`[^/\\]+`, LiteralStringOther, nil},
-			{`\\.`, LiteralStringOther, nil},
-			{`/[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-		},
-		"pipe-intp": {
-			{`[^#\|\\]+`, LiteralStringOther, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringOther, nil},
-			{`\|[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-			Include("interpol"),
-		},
-		"pipe-no-intp": {
-			{`[^\|\\]+`, LiteralStringOther, nil},
-			{`\\.`, LiteralStringOther, nil},
-			{`\|[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-		},
-		"quot-intp": {
-			{`[^#"\\]+`, LiteralStringOther, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringOther, nil},
-			{`"[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-			Include("interpol"),
-		},
-		"quot-no-intp": {
-			{`[^"\\]+`, LiteralStringOther, nil},
-			{`\\.`, LiteralStringOther, nil},
-			{`"[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-		},
-		"apos-intp": {
-			{`[^#'\\]+`, LiteralStringOther, nil},
-			Include("escapes"),
-			{`\\.`, LiteralStringOther, nil},
-			{`'[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-			Include("interpol"),
-		},
-		"apos-no-intp": {
-			{`[^'\\]+`, LiteralStringOther, nil},
-			{`\\.`, LiteralStringOther, nil},
-			{`'[a-zA-Z]*`, LiteralStringOther, Pop(1)},
-		},
-	}
-}

lexers/e/elm.go

@@ -1,63 +0,0 @@
-package e
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Elm lexer.
-var Elm = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Elm",
-		Aliases:   []string{"elm"},
-		Filenames: []string{"*.elm"},
-		MimeTypes: []string{"text/x-elm"},
-	},
-	elmRules,
-))
-
-func elmRules() Rules {
-	return Rules{
-		"root": {
-			{`\{-`, CommentMultiline, Push("comment")},
-			{`--.*`, CommentSingle, nil},
-			{`\s+`, Text, nil},
-			{`"`, LiteralString, Push("doublequote")},
-			{`^\s*module\s*`, KeywordNamespace, Push("imports")},
-			{`^\s*import\s*`, KeywordNamespace, Push("imports")},
-			{`\[glsl\|.*`, NameEntity, Push("shader")},
-			{Words(``, `\b`, `alias`, `as`, `case`, `else`, `if`, `import`, `in`, `let`, `module`, `of`, `port`, `then`, `type`, `where`), KeywordReserved, nil},
-			{`[A-Z]\w*`, KeywordType, nil},
-			{`^main `, KeywordReserved, nil},
-			{Words(`\(`, `\)`, `~`, `||`, `|>`, `|`, "`", `^`, `\`, `'`, `>>`, `>=`, `>`, `==`, `=`, `<~`, `<|`, `<=`, `<<`, `<-`, `<`, `::`, `:`, `/=`, `//`, `/`, `..`, `.`, `->`, `-`, `++`, `+`, `*`, `&&`, `%`), NameFunction, nil},
-			{Words(``, ``, `~`, `||`, `|>`, `|`, "`", `^`, `\`, `'`, `>>`, `>=`, `>`, `==`, `=`, `<~`, `<|`, `<=`, `<<`, `<-`, `<`, `::`, `:`, `/=`, `//`, `/`, `..`, `.`, `->`, `-`, `++`, `+`, `*`, `&&`, `%`), NameFunction, nil},
-			Include("numbers"),
-			{`[a-z_][a-zA-Z_\']*`, NameVariable, nil},
-			{`[,()\[\]{}]`, Punctuation, nil},
-		},
-		"comment": {
-			{`-(?!\})`, CommentMultiline, nil},
-			{`\{-`, CommentMultiline, Push("comment")},
-			{`[^-}]`, CommentMultiline, nil},
-			{`-\}`, CommentMultiline, Pop(1)},
-		},
-		"doublequote": {
-			{`\\u[0-9a-fA-F]{4}`, LiteralStringEscape, nil},
-			{`\\[nrfvb\\"]`, LiteralStringEscape, nil},
-			{`[^"]`, LiteralString, nil},
-			{`"`, LiteralString, Pop(1)},
-		},
-		"imports": {
-			{`\w+(\.\w+)*`, NameClass, Pop(1)},
-		},
-		"numbers": {
-			{`_?\d+\.(?=\d+)`, LiteralNumberFloat, nil},
-			{`_?\d+`, LiteralNumberInteger, nil},
-		},
-		"shader": {
-			{`\|(?!\])`, NameEntity, nil},
-			{`\|\]`, NameEntity, Pop(1)},
-			{`.*\n`, NameEntity, nil},
-		},
-	}
-}

lexers/e/erlang.go

@@ -1,70 +0,0 @@
-package e
-
-import (
-	. "github.com/alecthomas/chroma" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
-)
-
-// Erlang lexer.
-var Erlang = internal.Register(MustNewLazyLexer(
-	&Config{
-		Name:      "Erlang",
-		Aliases:   []string{"erlang"},
-		Filenames: []string{"*.erl", "*.hrl", "*.es", "*.escript"},
-		MimeTypes: []string{"text/x-erlang"},
-	},
-	erlangRules,
-))
-
-func erlangRules() Rules {
-	return Rules{
-		"root": {
-			{`\s+`, Text, nil},
-			{`%.*\n`, Comment, nil},
-			{Words(``, `\b`, `after`, `begin`, `case`, `catch`, `cond`, `end`, `fun`, `if`, `let`, `of`, `query`, `receive`, `try`, `when`), Keyword, nil},
-			{Words(``, `\b`, `abs`, `append_element`, `apply`, `atom_to_list`, `binary_to_list`, `bitstring_to_list`, `binary_to_term`, `bit_size`, `bump_reductions`, `byte_size`, `cancel_timer`, `check_process_code`, `delete_module`, `demonitor`, `disconnect_node`, `display`, `element`, `erase`, `exit`, `float`, `float_to_list`, `fun_info`, `fun_to_list`, `function_exported`, `garbage_collect`, `get`, `get_keys`, `group_leader`, `hash`, `hd`, `integer_to_list`, `iolist_to_binary`, `iolist_size`, `is_atom`, `is_binary`, `is_bitstring`, `is_boolean`, `is_builtin`, `is_float`, `is_function`, `is_integer`, `is_list`, `is_number`, `is_pid`, `is_port`, `is_process_alive`, `is_record`, `is_reference`, `is_tuple`, `length`, `link`, `list_to_atom`, `list_to_binary`, `list_to_bitstring`, `list_to_existing_atom`, `list_to_float`, `list_to_integer`, `list_to_pid`, `list_to_tuple`, `load_module`, `localtime_to_universaltime`, `make_tuple`, `md5`, `md5_final`, `md5_update`, `memory`, `module_loaded`, `monitor`, `monitor_node`, `node`, `nodes`, `open_port`, `phash`, `phash2`, `pid_to_list`, `port_close`, `port_command`, `port_connect`, `port_control`, `port_call`, `port_info`, `port_to_list`, `process_display`, `process_flag`, `process_info`, `purge_module`, `put`, `read_timer`, `ref_to_list`, `register`, `resume_process`, `round`, `send`, `send_after`, `send_nosuspend`, `set_cookie`, `setelement`, `size`, `spawn`, `spawn_link`, `spawn_monitor`, `spawn_opt`, `split_binary`, `start_timer`, `statistics`, `suspend_process`, `system_flag`, `system_info`, `system_monitor`, `system_profile`, `term_to_binary`, `tl`, `trace`, `trace_delivered`, `trace_info`, `trace_pattern`, `trunc`, `tuple_size`, `tuple_to_list`, `universaltime_to_localtime`, `unlink`, `unregister`, `whereis`), NameBuiltin, nil},
-			{Words(``, `\b`, `and`, `andalso`, `band`, `bnot`, `bor`, `bsl`, `bsr`, `bxor`, `div`, `not`, `or`, `orelse`, `rem`, `xor`), OperatorWord, nil},
-			{`^-`, Punctuation, Push("directive")},
-			{`(\+\+?|--?|\*|/|<|>|/=|=:=|=/=|=<|>=|==?|<-|!|\?)`, Operator, nil},
-			{`"`, LiteralString, Push("string")},
-			{`<<`, NameLabel, nil},
-			{`>>`, NameLabel, nil},
-			{`((?:[a-z]\w*|'[^\n']*[^\\]'))(:)`, ByGroups(NameNamespace, Punctuation), nil},
-			{`(?:^|(?<=:))((?:[a-z]\w*|'[^\n']*[^\\]'))(\s*)(\()`, ByGroups(NameFunction, Text, Punctuation), nil},
-			{`[+-]?(?:[2-9]|[12][0-9]|3[0-6])#[0-9a-zA-Z]+`, LiteralNumberInteger, nil},
-			{`[+-]?\d+`, LiteralNumberInteger, nil},
-			{`[+-]?\d+.\d+`, LiteralNumberFloat, nil},
-			{`[]\[:_@\".{}()|;,]`, Punctuation, nil},
-			{`(?:[A-Z_]\w*)`, NameVariable, nil},
-			{`(?:[a-z]\w*|'[^\n']*[^\\]')`, Name, nil},
-			{`\?(?:(?:[A-Z_]\w*)|(?:[a-z]\w*|'[^\n']*[^\\]'))`, NameConstant, nil},
-			{`\$(?:(?:\\(?:[bdefnrstv\'"\\]|[0-7][0-7]?[0-7]?|(?:x[0-9a-fA-F]{2}|x\{[0-9a-fA-F]+\})|\^[a-zA-Z]))|\\[ %]|[^\\])`, LiteralStringChar, nil},
-			{`#(?:[a-z]\w*|'[^\n']*[^\\]')(:?\.(?:[a-z]\w*|'[^\n']*[^\\]'))?`, NameLabel, nil},
-			{`\A#!.+\n`, CommentHashbang, nil},
-			{`#\{`, Punctuation, Push("map_key")},
-		},
-		"string": {
-			{`(?:\\(?:[bdefnrstv\'"\\]|[0-7][0-7]?[0-7]?|(?:x[0-9a-fA-F]{2}|x\{[0-9a-fA-F]+\})|\^[a-zA-Z]))`, LiteralStringEscape, nil},
-			{`"`, LiteralString, Pop(1)},
-			{`~[0-9.*]*[~#+BPWXb-ginpswx]`, LiteralStringInterpol, nil},
-			{`[^"\\~]+`, LiteralString, nil},
-			{`~`, LiteralString, nil},
-		},
-		"directive": {
-			{`(define)(\s*)(\()((?:(?:[A-Z_]\w*)|(?:[a-z]\w*|'[^\n']*[^\\]')))`, ByGroups(NameEntity, Text, Punctuation, NameConstant), Pop(1)},
-			{`(record)(\s*)(\()((?:(?:[A-Z_]\w*)|(?:[a-z]\w*|'[^\n']*[^\\]')))`, ByGroups(NameEntity, Text, Punctuation, NameLabel), Pop(1)},
-			{`(?:[a-z]\w*|'[^\n']*[^\\]')`, NameEntity, Pop(1)},
-		},
-		"map_key": {
-			Include("root"),
-			{`=>`, Punctuation, Push("map_val")},
-			{`:=`, Punctuation, Push("map_val")},
-			{`\}`, Punctuation, Pop(1)},
-		},
-		"map_val": {
-			Include("root"),
-			{`,`, Punctuation, Pop(1)},
-			{`(?=\})`, Punctuation, Pop(1)},
-		},
-	}
-}

lexers/emacs.go

@@ -1,8 +1,7 @@
-package e
+package lexers
 
 import (
-	. "github.com/alecthomas/chroma" // nolint
+	. "github.com/alecthomas/chroma/v2" // nolint
-	"github.com/alecthomas/chroma/lexers/internal"
 )
 
 var (
@@ -522,14 +521,9 @@ var (
 )
 
 // EmacsLisp lexer.
-var EmacsLisp = internal.Register(TypeRemappingLexer(MustNewLazyLexer(
+var EmacsLisp = Register(TypeRemappingLexer(MustNewXMLLexer(
-	&Config{
+	embedded,
-		Name:      "EmacsLisp",
+	"embedded/emacslisp.xml",
-		Aliases:   []string{"emacs", "elisp", "emacs-lisp"},
-		Filenames: []string{"*.el"},
-		MimeTypes: []string{"text/x-elisp", "application/x-elisp"},
-	},
-	emacsLispRules,
 ), TypeMapping{
 	{NameVariable, NameFunction, emacsBuiltinFunction},
 	{NameVariable, NameBuiltin, emacsSpecialForms},
@@ -537,50 +531,3 @@ var EmacsLisp = internal.Register(TypeRemappingLexer(MustNewLazyLexer(
 	{NameVariable, NameBuiltin, append(emacsBuiltinFunctionHighlighted, emacsMacros...)},
 	{NameVariable, KeywordPseudo, emacsLambdaListKeywords},
 }))
-
-func emacsLispRules() Rules {
-	return Rules{
-		"root": {
-			Default(Push("body")),
-		},
-		"body": {
-			{`\s+`, Text, nil},
-			{`;.*$`, CommentSingle, nil},
-			{`"`, LiteralString, Push("string")},
-			{`\?([^\\]|\\.)`, LiteralStringChar, nil},
-			{`:((?:\\.|[\w!$%&*+-/<=>?@^{}~|])(?:\\.|[\w!$%&*+-/<=>?@^{}~|]|[#.:])*)`, NameBuiltin, nil},
-			{`::((?:\\.|[\w!$%&*+-/<=>?@^{}~|])(?:\\.|[\w!$%&*+-/<=>?@^{}~|]|[#.:])*)`, LiteralStringSymbol, nil},
-			{`'((?:\\.|[\w!$%&*+-/<=>?@^{}~|])(?:\\.|[\w!$%&*+-/<=>?@^{}~|]|[#.:])*)`, LiteralStringSymbol, nil},
-			{`'`, Operator, nil},
-			{"`", Operator, nil},
-			{"[-+]?\\d+\\.?(?=[ \"()\\]\\'\\n,;`])", LiteralNumberInteger, nil},
-			{"[-+]?\\d+/\\d+(?=[ \"()\\]\\'\\n,;`])", LiteralNumber, nil},
-			{"[-+]?(\\d*\\.\\d+([defls][-+]?\\d+)?|\\d+(\\.\\d*)?[defls][-+]?\\d+)(?=[ \"()\\]\\'\\n,;`])", LiteralNumberFloat, nil},
-			{`\[|\]`, Punctuation, nil},
-			{`#:((?:\\.|[\w!$%&*+-/<=>?@^{}~|])(?:\\.|[\w!$%&*+-/<=>?@^{}~|]|[#.:])*)`, LiteralStringSymbol, nil},
-			{`#\^\^?`, Operator, nil},
-			{`#\'`, NameFunction, nil},
-			{`#[bB][+-]?[01]+(/[01]+)?`, LiteralNumberBin, nil},
-			{`#[oO][+-]?[0-7]+(/[0-7]+)?`, LiteralNumberOct, nil},
-			{`#[xX][+-]?[0-9a-fA-F]+(/[0-9a-fA-F]+)?`, LiteralNumberHex, nil},
-			{`#\d+r[+-]?[0-9a-zA-Z]+(/[0-9a-zA-Z]+)?`, LiteralNumber, nil},
-			{`#\d+=`, Operator, nil},
-			{`#\d+#`, Operator, nil},
-			{`(,@|,|\.|:)`, Operator, nil},
-			{"(t|nil)(?=[ \"()\\]\\'\\n,;`])", NameConstant, nil},
-			{`\*((?:\\.|[\w!$%&*+-/<=>?@^{}~|])(?:\\.|[\w!$%&*+-/<=>?@^{}~|]|[#.:])*)\*`, NameVariableGlobal, nil},
-			{`((?:\\.|[\w!$%&*+-/<=>?@^{}~|])(?:\\.|[\w!$%&*+-/<=>?@^{}~|]|[#.:])*)`, NameVariable, nil},
-			{`#\(`, Operator, Push("body")},
-			{`\(`, Punctuation, Push("body")},
-			{`\)`, Punctuation, Pop(1)},
-		},
-		"string": {
-			{"[^\"\\\\`]+", LiteralString, nil},
-			{"`((?:\\\\.|[\\w!$%&*+-/<=>?@^{}~|])(?:\\\\.|[\\w!$%&*+-/<=>?@^{}~|]|[#.:])*)\\'", LiteralStringSymbol, nil},
-			{"`", LiteralString, nil},
-			{`\\.`, LiteralString, nil},
-			{`\\\n`, LiteralString, nil},
-			{`"`, LiteralString, Pop(1)},
-		},
-	}
-}

lexers/embedded/abap.xml

@@ -0,0 +1,154 @@
+<lexer>
+  <config>
+    <name>ABAP</name>
+    <alias>abap</alias>
+    <filename>*.abap</filename>
+    <filename>*.ABAP</filename>
+    <mime_type>text/x-abap</mime_type>
+    <case_insensitive>true</case_insensitive>
+  </config>
+  <rules>
+    <state name="common">
+      <rule pattern="\s+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="^\*.*$">
+        <token type="CommentSingle"/>
+      </rule>
+      <rule pattern="\&#34;.*?\n">
+        <token type="CommentSingle"/>
+      </rule>
+      <rule pattern="##\w+">
+        <token type="CommentSpecial"/>
+      </rule>
+    </state>
+    <state name="variable-names">
+      <rule pattern="&lt;\S+&gt;">
+        <token type="NameVariable"/>
+      </rule>
+      <rule pattern="\w[\w~]*(?:(\[\])|-&gt;\*)?">
+        <token type="NameVariable"/>
+      </rule>
+    </state>
+    <state name="root">
+      <rule>
+        <include state="common"/>
+      </rule>
+      <rule pattern="CALL\s+(?:BADI|CUSTOMER-FUNCTION|FUNCTION)">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="(CALL\s+(?:DIALOG|SCREEN|SUBSCREEN|SELECTION-SCREEN|TRANSACTION|TRANSFORMATION))\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="(FORM|PERFORM)(\s+)(\w+)">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="Text"/>
+          <token type="NameFunction"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(PERFORM)(\s+)(\()(\w+)(\))">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="Text"/>
+          <token type="Punctuation"/>
+          <token type="NameVariable"/>
+          <token type="Punctuation"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(MODULE)(\s+)(\S+)(\s+)(INPUT|OUTPUT)">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="Text"/>
+          <token type="NameFunction"/>
+          <token type="Text"/>
+          <token type="Keyword"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(METHOD)(\s+)([\w~]+)">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="Text"/>
+          <token type="NameFunction"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(\s+)([\w\-]+)([=\-]&gt;)([\w\-~]+)">
+        <bygroups>
+          <token type="Text"/>
+          <token type="NameVariable"/>
+          <token type="Operator"/>
+          <token type="NameFunction"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(?&lt;=(=|-)&gt;)([\w\-~]+)(?=\()">
+        <token type="NameFunction"/>
+      </rule>
+      <rule pattern="(TEXT)(-)(\d{3})">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="Punctuation"/>
+          <token type="LiteralNumberInteger"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(TEXT)(-)(\w{3})">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="Punctuation"/>
+          <token type="NameVariable"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(ADD-CORRESPONDING|AUTHORITY-CHECK|CLASS-DATA|CLASS-EVENTS|CLASS-METHODS|CLASS-POOL|DELETE-ADJACENT|DIVIDE-CORRESPONDING|EDITOR-CALL|ENHANCEMENT-POINT|ENHANCEMENT-SECTION|EXIT-COMMAND|FIELD-GROUPS|FIELD-SYMBOLS|FUNCTION-POOL|INTERFACE-POOL|INVERTED-DATE|LOAD-OF-PROGRAM|LOG-POINT|MESSAGE-ID|MOVE-CORRESPONDING|MULTIPLY-CORRESPONDING|NEW-LINE|NEW-PAGE|NEW-SECTION|NO-EXTENSION|OUTPUT-LENGTH|PRINT-CONTROL|SELECT-OPTIONS|START-OF-SELECTION|SUBTRACT-CORRESPONDING|SYNTAX-CHECK|SYSTEM-EXCEPTIONS|TYPE-POOL|TYPE-POOLS|NO-DISPLAY)\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="(?&lt;![-\&gt;])(CREATE\s+(PUBLIC|PRIVATE|DATA|OBJECT)|(PUBLIC|PRIVATE|PROTECTED)\s+SECTION|(TYPE|LIKE)\s+((LINE\s+OF|REF\s+TO|(SORTED|STANDARD|HASHED)\s+TABLE\s+OF))?|FROM\s+(DATABASE|MEMORY)|CALL\s+METHOD|(GROUP|ORDER) BY|HAVING|SEPARATED BY|GET\s+(BADI|BIT|CURSOR|DATASET|LOCALE|PARAMETER|PF-STATUS|(PROPERTY|REFERENCE)\s+OF|RUN\s+TIME|TIME\s+(STAMP)?)?|SET\s+(BIT|BLANK\s+LINES|COUNTRY|CURSOR|DATASET|EXTENDED\s+CHECK|HANDLER|HOLD\s+DATA|LANGUAGE|LEFT\s+SCROLL-BOUNDARY|LOCALE|MARGIN|PARAMETER|PF-STATUS|PROPERTY\s+OF|RUN\s+TIME\s+(ANALYZER|CLOCK\s+RESOLUTION)|SCREEN|TITLEBAR|UPADTE\s+TASK\s+LOCAL|USER-COMMAND)|CONVERT\s+((INVERTED-)?DATE|TIME|TIME\s+STAMP|TEXT)|(CLOSE|OPEN)\s+(DATASET|CURSOR)|(TO|FROM)\s+(DATA BUFFER|INTERNAL TABLE|MEMORY ID|DATABASE|SHARED\s+(MEMORY|BUFFER))|DESCRIBE\s+(DISTANCE\s+BETWEEN|FIELD|LIST|TABLE)|FREE\s(MEMORY|OBJECT)?|PROCESS\s+(BEFORE\s+OUTPUT|AFTER\s+INPUT|ON\s+(VALUE-REQUEST|HELP-REQUEST))|AT\s+(LINE-SELECTION|USER-COMMAND|END\s+OF|NEW)|AT\s+SELECTION-SCREEN(\s+(ON(\s+(BLOCK|(HELP|VALUE)-REQUEST\s+FOR|END\s+OF|RADIOBUTTON\s+GROUP))?|OUTPUT))?|SELECTION-SCREEN:?\s+((BEGIN|END)\s+OF\s+((TABBED\s+)?BLOCK|LINE|SCREEN)|COMMENT|FUNCTION\s+KEY|INCLUDE\s+BLOCKS|POSITION|PUSHBUTTON|SKIP|ULINE)|LEAVE\s+(LIST-PROCESSING|PROGRAM|SCREEN|TO LIST-PROCESSING|TO TRANSACTION)(ENDING|STARTING)\s+AT|FORMAT\s+(COLOR|INTENSIFIED|INVERSE|HOTSPOT|INPUT|FRAMES|RESET)|AS\s+(CHECKBOX|SUBSCREEN|WINDOW)|WITH\s+(((NON-)?UNIQUE)?\s+KEY|FRAME)|(BEGIN|END)\s+OF|DELETE(\s+ADJACENT\s+DUPLICATES\sFROM)?|COMPARING(\s+ALL\s+FIELDS)?|(INSERT|APPEND)(\s+INITIAL\s+LINE\s+(IN)?TO|\s+LINES\s+OF)?|IN\s+((BYTE|CHARACTER)\s+MODE|PROGRAM)|END-OF-(DEFINITION|PAGE|SELECTION)|WITH\s+FRAME(\s+TITLE)|(REPLACE|FIND)\s+((FIRST|ALL)\s+OCCURRENCES?\s+OF\s+)?(SUBSTRING|REGEX)?|MATCH\s+(LENGTH|COUNT|LINE|OFFSET)|(RESPECTING|IGNORING)\s+CASE|IN\s+UPDATE\s+TASK|(SOURCE|RESULT)\s+(XML)?|REFERENCE\s+INTO|AND\s+(MARK|RETURN)|CLIENT\s+SPECIFIED|CORRESPONDING\s+FIELDS\s+OF|IF\s+FOUND|FOR\s+EVENT|INHERITING\s+FROM|LEAVE\s+TO\s+SCREEN|LOOP\s+AT\s+(SCREEN)?|LOWER\s+CASE|MATCHCODE\s+OBJECT|MODIF\s+ID|MODIFY\s+SCREEN|NESTING\s+LEVEL|NO\s+INTERVALS|OF\s+STRUCTURE|RADIOBUTTON\s+GROUP|RANGE\s+OF|REF\s+TO|SUPPRESS DIALOG|TABLE\s+OF|UPPER\s+CASE|TRANSPORTING\s+NO\s+FIELDS|VALUE\s+CHECK|VISIBLE\s+LENGTH|HEADER\s+LINE|COMMON\s+PART)\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="(^|(?&lt;=(\s|\.)))(ABBREVIATED|ABSTRACT|ADD|ALIASES|ALIGN|ALPHA|ASSERT|AS|ASSIGN(ING)?|AT(\s+FIRST)?|BACK|BLOCK|BREAK-POINT|CASE|CATCH|CHANGING|CHECK|CLASS|CLEAR|COLLECT|COLOR|COMMIT|CREATE|COMMUNICATION|COMPONENTS?|COMPUTE|CONCATENATE|CONDENSE|CONSTANTS|CONTEXTS|CONTINUE|CONTROLS|COUNTRY|CURRENCY|DATA|DATE|DECIMALS|DEFAULT|DEFINE|DEFINITION|DEFERRED|DEMAND|DETAIL|DIRECTORY|DIVIDE|DO|DUMMY|ELSE(IF)?|ENDAT|ENDCASE|ENDCATCH|ENDCLASS|ENDDO|ENDFORM|ENDFUNCTION|ENDIF|ENDINTERFACE|ENDLOOP|ENDMETHOD|ENDMODULE|ENDSELECT|ENDTRY|ENDWHILE|ENHANCEMENT|EVENTS|EXACT|EXCEPTIONS?|EXIT|EXPONENT|EXPORT|EXPORTING|EXTRACT|FETCH|FIELDS?|FOR|FORM|FORMAT|FREE|FROM|FUNCTION|HIDE|ID|IF|IMPORT|IMPLEMENTATION|IMPORTING|IN|INCLUDE|INCLUDING|INDEX|INFOTYPES|INITIALIZATION|INTERFACE|INTERFACES|INTO|LANGUAGE|LEAVE|LENGTH|LINES|LOAD|LOCAL|JOIN|KEY|NEXT|MAXIMUM|MESSAGE|METHOD[S]?|MINIMUM|MODULE|MODIFIER|MODIFY|MOVE|MULTIPLY|NODES|NUMBER|OBLIGATORY|OBJECT|OF|OFF|ON|OTHERS|OVERLAY|PACK|PAD|PARAMETERS|PERCENTAGE|POSITION|PROGRAM|PROVIDE|PUBLIC|PUT|PF\d\d|RAISE|RAISING|RANGES?|READ|RECEIVE|REDEFINITION|REFRESH|REJECT|REPORT|RESERVE|RESUME|RETRY|RETURN|RETURNING|RIGHT|ROLLBACK|REPLACE|SCROLL|SEARCH|SELECT|SHIFT|SIGN|SINGLE|SIZE|SKIP|SORT|SPLIT|STATICS|STOP|STYLE|SUBMATCHES|SUBMIT|SUBTRACT|SUM(?!\()|SUMMARY|SUMMING|SUPPLY|TABLE|TABLES|TIMESTAMP|TIMES?|TIMEZONE|TITLE|\??TO|TOP-OF-PAGE|TRANSFER|TRANSLATE|TRY|TYPES|ULINE|UNDER|UNPACK|UPDATE|USING|VALUE|VALUES|VIA|VARYING|VARY|WAIT|WHEN|WHERE|WIDTH|WHILE|WITH|WINDOW|WRITE|XSD|ZERO)\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="(abs|acos|asin|atan|boolc|boolx|bit_set|char_off|charlen|ceil|cmax|cmin|condense|contains|contains_any_of|contains_any_not_of|concat_lines_of|cos|cosh|count|count_any_of|count_any_not_of|dbmaxlen|distance|escape|exp|find|find_end|find_any_of|find_any_not_of|floor|frac|from_mixed|insert|lines|log|log10|match|matches|nmax|nmin|numofchar|repeat|replace|rescale|reverse|round|segment|shift_left|shift_right|sign|sin|sinh|sqrt|strlen|substring|substring_after|substring_from|substring_before|substring_to|tan|tanh|to_upper|to_lower|to_mixed|translate|trunc|xstrlen)(\()\b">
+        <bygroups>
+          <token type="NameBuiltin"/>
+          <token type="Punctuation"/>
+        </bygroups>
+      </rule>
+      <rule pattern="&amp;[0-9]">
+        <token type="Name"/>
+      </rule>
+      <rule pattern="[0-9]+">
+        <token type="LiteralNumberInteger"/>
+      </rule>
+      <rule pattern="(?&lt;=(\s|.))(AND|OR|EQ|NE|GT|LT|GE|LE|CO|CN|CA|NA|CS|NOT|NS|CP|NP|BYTE-CO|BYTE-CN|BYTE-CA|BYTE-NA|BYTE-CS|BYTE-NS|IS\s+(NOT\s+)?(INITIAL|ASSIGNED|REQUESTED|BOUND))\b">
+        <token type="OperatorWord"/>
+      </rule>
+      <rule>
+        <include state="variable-names"/>
+      </rule>
+      <rule pattern="[?*&lt;&gt;=\-+&amp;]">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="&#39;(&#39;&#39;|[^&#39;])*&#39;">
+        <token type="LiteralStringSingle"/>
+      </rule>
+      <rule pattern="`([^`])*`">
+        <token type="LiteralStringSingle"/>
+      </rule>
+      <rule pattern="([|}])([^{}|]*?)([|{])">
+        <bygroups>
+          <token type="Punctuation"/>
+          <token type="LiteralStringSingle"/>
+          <token type="Punctuation"/>
+        </bygroups>
+      </rule>
+      <rule pattern="[/;:()\[\],.]">
+        <token type="Punctuation"/>
+      </rule>
+      <rule pattern="(!)(\w+)">
+        <bygroups>
+          <token type="Operator"/>
+          <token type="Name"/>
+        </bygroups>
+      </rule>
+    </state>
+  </rules>
+</lexer>

lexers/embedded/abnf.xml

@@ -0,0 +1,66 @@
+<lexer>
+  <config>
+    <name>ABNF</name>
+    <alias>abnf</alias>
+    <filename>*.abnf</filename>
+    <mime_type>text/x-abnf</mime_type>
+  </config>
+  <rules>
+    <state name="root">
+      <rule pattern=";.*$">
+        <token type="CommentSingle"/>
+      </rule>
+      <rule pattern="(%[si])?&#34;[^&#34;]*&#34;">
+        <token type="Literal"/>
+      </rule>
+      <rule pattern="%b[01]+\-[01]+\b">
+        <token type="Literal"/>
+      </rule>
+      <rule pattern="%b[01]+(\.[01]+)*\b">
+        <token type="Literal"/>
+      </rule>
+      <rule pattern="%d[0-9]+\-[0-9]+\b">
+        <token type="Literal"/>
+      </rule>
+      <rule pattern="%d[0-9]+(\.[0-9]+)*\b">
+        <token type="Literal"/>
+      </rule>
+      <rule pattern="%x[0-9a-fA-F]+\-[0-9a-fA-F]+\b">
+        <token type="Literal"/>
+      </rule>
+      <rule pattern="%x[0-9a-fA-F]+(\.[0-9a-fA-F]+)*\b">
+        <token type="Literal"/>
+      </rule>
+      <rule pattern="\b[0-9]+\*[0-9]+">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="\b[0-9]+\*">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="\b[0-9]+">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="\*">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="(HEXDIG|DQUOTE|DIGIT|VCHAR|OCTET|ALPHA|CHAR|CRLF|HTAB|LWSP|BIT|CTL|WSP|LF|SP|CR)\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="[a-zA-Z][a-zA-Z0-9-]+\b">
+        <token type="NameClass"/>
+      </rule>
+      <rule pattern="(=/|=|/)">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="[\[\]()]">
+        <token type="Punctuation"/>
+      </rule>
+      <rule pattern="\s+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern=".">
+        <token type="Text"/>
+      </rule>
+    </state>
+  </rules>
+</lexer>

lexers/embedded/actionscript.xml

@@ -0,0 +1,68 @@
+<lexer>
+  <config>
+    <name>ActionScript</name>
+    <alias>as</alias>
+    <alias>actionscript</alias>
+    <filename>*.as</filename>
+    <mime_type>application/x-actionscript</mime_type>
+    <mime_type>text/x-actionscript</mime_type>
+    <mime_type>text/actionscript</mime_type>
+    <dot_all>true</dot_all>
+    <not_multiline>true</not_multiline>
+  </config>
+  <rules>
+    <state name="root">
+      <rule pattern="\s+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="//.*?\n">
+        <token type="CommentSingle"/>
+      </rule>
+      <rule pattern="/\*.*?\*/">
+        <token type="CommentMultiline"/>
+      </rule>
+      <rule pattern="/(\\\\|\\/|[^/\n])*/[gim]*">
+        <token type="LiteralStringRegex"/>
+      </rule>
+      <rule pattern="[~^*!%&amp;&lt;&gt;|+=:;,/?\\-]+">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="[{}\[\]();.]+">
+        <token type="Punctuation"/>
+      </rule>
+      <rule pattern="(instanceof|arguments|continue|default|typeof|switch|return|catch|break|while|throw|each|this|with|else|case|var|new|for|try|if|do|in)\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="(implements|protected|namespace|interface|intrinsic|override|function|internal|private|package|extends|dynamic|import|native|return|public|static|class|const|super|final|get|set)\b">
+        <token type="KeywordDeclaration"/>
+      </rule>
+      <rule pattern="(true|false|null|NaN|Infinity|-Infinity|undefined|Void)\b">
+        <token type="KeywordConstant"/>
+      </rule>
+      <rule pattern="(IDynamicPropertyOutputIDynamicPropertyWriter|DisplacmentMapFilterMode|AccessibilityProperties|ContextMenuBuiltInItems|SharedObjectFlushStatus|DisplayObjectContainer|IllegalOperationError|DisplacmentMapFilter|InterpolationMethod|URLLoaderDataFormat|PrintJobOrientation|ActionScriptVersion|BitmapFilterQuality|GradientBevelFilter|GradientGlowFilter|DeleteObjectSample|StackOverflowError|SoundLoaderContext|ScriptTimeoutError|SecurityErrorEvent|InteractiveObject|StageDisplayState|FileReferenceList|TextFieldAutoSize|ApplicationDomain|BitmapDataChannel|ColorMatrixFilter|ExternalInterface|IMEConversionMode|DropShadowFilter|URLRequestHeader|ContextMenuEvent|ConvultionFilter|URLRequestMethod|BitmapFilterType|IEventDispatcher|ContextMenuItem|LocalConnection|InvalidSWFError|AsyncErrorEvent|MovieClipLoader|IBitmapDrawable|PrintJobOptions|EventDispatcher|NewObjectSample|HTTPStatusEvent|TextFormatAlign|IExternalizable|FullScreenEvent|DefinitionError|TextLineMetrics|NetStatusEvent|ColorTransform|ObjectEncoding|SecurityDomain|StageScaleMode|FocusDirection|ReferenceError|SoundTransform|KeyboardEvent|DisplayObject|PixelSnapping|LoaderContext|NetConnection|SecurityPanel|SecurityError|FileReference|AsBroadcaster|LineScaleMode|AntiAliasType|Accessibility|TextFieldType|URLVariabeles|ActivityEvent|ProgressEvent|TextColorType|StageQuality|TextSnapshot|Capabilities|BitmapFilter|SpreadMethod|GradientType|TextRenderer|SoundChannel|SharedObject|IOErrorEvent|SimpleButton|ContextMenu|InvokeEvent|CSMSettings|SyntaxError|StatusEvent|KeyLocation|IDataOutput|VerifyError|XMLDocument|XMLNodeType|MemoryError|GridFitType|BevelFilter|ErrorEvent|FrameLabel|GlowFilter|LoaderInfo|Microphone|MorphShape|BlurFilter|MouseEvent|FocusEvent|SoundMixer|FileFilter|TimerEvent|JointStyle|EventPhase|StageAlign|Dictionary|URLRequest|StyleSheet|SWFVersion|IDataInput|StaticText|RangeError|BitmapData|TextFormat|StackFrame|Namespace|SyncEvent|Rectangle|URLLoader|TypeError|Responder|NetStream|BlendMode|CapsStyle|DataEvent|ByteArray|MovieClip|Transform|TextField|Selection|AVM1Movie|XMLSocket|URLStream|FontStyle|EvalError|FontType|LoadVars|Graphics|Security|IMEEvent|URIError|Keyboard|Function|EOFError|PrintJob|IOError|XMLList|Boolean|ID3Info|XMLNode|Bitmap|String|RegExp|Sample|Object|Sprite|System|Endian|Matrix|Camera|Locale|Number|Loader|Socket|QName|Class|Timer|Sound|Shape|XMLUI|Mouse|Scene|Stage|Color|Point|Video|Error|Event|Proxy|Array|Date|uint|Math|Font|int|Key|IME|XML)\b">
+        <token type="NameBuiltin"/>
+      </rule>
+      <rule pattern="(decodeURIComponent|updateAfterEvent|clearInterval|setInterval|getVersion|parseFloat|fscommand|isXMLName|encodeURI|decodeURI|getTimer|unescape|isFinite|parseInt|getURL|escape|trace|isNaN|eval)\b">
+        <token type="NameFunction"/>
+      </rule>
+      <rule pattern="[$a-zA-Z_]\w*">
+        <token type="NameOther"/>
+      </rule>
+      <rule pattern="[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?">
+        <token type="LiteralNumberFloat"/>
+      </rule>
+      <rule pattern="0x[0-9a-f]+">
+        <token type="LiteralNumberHex"/>
+      </rule>
+      <rule pattern="[0-9]+">
+        <token type="LiteralNumberInteger"/>
+      </rule>
+      <rule pattern="&#34;(\\\\|\\&#34;|[^&#34;])*&#34;">
+        <token type="LiteralStringDouble"/>
+      </rule>
+      <rule pattern="&#39;(\\\\|\\&#39;|[^&#39;])*&#39;">
+        <token type="LiteralStringSingle"/>
+      </rule>
+    </state>
+  </rules>
+</lexer>

lexers/embedded/actionscript_3.xml

@@ -0,0 +1,163 @@
+<lexer>
+  <config>
+    <name>ActionScript 3</name>
+    <alias>as3</alias>
+    <alias>actionscript3</alias>
+    <filename>*.as</filename>
+    <mime_type>application/x-actionscript3</mime_type>
+    <mime_type>text/x-actionscript3</mime_type>
+    <mime_type>text/actionscript3</mime_type>
+    <dot_all>true</dot_all>
+  </config>
+  <rules>
+    <state name="funcparams">
+      <rule pattern="\s+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="(\s*)(\.\.\.)?([$a-zA-Z_]\w*)(\s*)(:)(\s*)([$a-zA-Z_]\w*(?:\.&lt;\w+&gt;)?|\*)(\s*)">
+        <bygroups>
+          <token type="Text"/>
+          <token type="Punctuation"/>
+          <token type="Name"/>
+          <token type="Text"/>
+          <token type="Operator"/>
+          <token type="Text"/>
+          <token type="KeywordType"/>
+          <token type="Text"/>
+        </bygroups>
+        <push state="defval"/>
+      </rule>
+      <rule pattern="\)">
+        <token type="Operator"/>
+        <push state="type"/>
+      </rule>
+    </state>
+    <state name="type">
+      <rule pattern="(\s*)(:)(\s*)([$a-zA-Z_]\w*(?:\.&lt;\w+&gt;)?|\*)">
+        <bygroups>
+          <token type="Text"/>
+          <token type="Operator"/>
+          <token type="Text"/>
+          <token type="KeywordType"/>
+        </bygroups>
+        <pop depth="2"/>
+      </rule>
+      <rule pattern="\s+">
+        <token type="Text"/>
+        <pop depth="2"/>
+      </rule>
+      <rule>
+        <pop depth="2"/>
+      </rule>
+    </state>
+    <state name="defval">
+      <rule pattern="(=)(\s*)([^(),]+)(\s*)(,?)">
+        <bygroups>
+          <token type="Operator"/>
+          <token type="Text"/>
+          <usingself state="root"/>
+          <token type="Text"/>
+          <token type="Operator"/>
+        </bygroups>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern=",">
+        <token type="Operator"/>
+        <pop depth="1"/>
+      </rule>
+      <rule>
+        <pop depth="1"/>
+      </rule>
+    </state>
+    <state name="root">
+      <rule pattern="\s+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="(function\s+)([$a-zA-Z_]\w*)(\s*)(\()">
+        <bygroups>
+          <token type="KeywordDeclaration"/>
+          <token type="NameFunction"/>
+          <token type="Text"/>
+          <token type="Operator"/>
+        </bygroups>
+        <push state="funcparams"/>
+      </rule>
+      <rule pattern="(var|const)(\s+)([$a-zA-Z_]\w*)(\s*)(:)(\s*)([$a-zA-Z_]\w*(?:\.&lt;\w+&gt;)?)">
+        <bygroups>
+          <token type="KeywordDeclaration"/>
+          <token type="Text"/>
+          <token type="Name"/>
+          <token type="Text"/>
+          <token type="Punctuation"/>
+          <token type="Text"/>
+          <token type="KeywordType"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(import|package)(\s+)((?:[$a-zA-Z_]\w*|\.)+)(\s*)">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="Text"/>
+          <token type="NameNamespace"/>
+          <token type="Text"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(new)(\s+)([$a-zA-Z_]\w*(?:\.&lt;\w+&gt;)?)(\s*)(\()">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="Text"/>
+          <token type="KeywordType"/>
+          <token type="Text"/>
+          <token type="Operator"/>
+        </bygroups>
+      </rule>
+      <rule pattern="//.*?\n">
+        <token type="CommentSingle"/>
+      </rule>
+      <rule pattern="/\*.*?\*/">
+        <token type="CommentMultiline"/>
+      </rule>
+      <rule pattern="/(\\\\|\\/|[^\n])*/[gisx]*">
+        <token type="LiteralStringRegex"/>
+      </rule>
+      <rule pattern="(\.)([$a-zA-Z_]\w*)">
+        <bygroups>
+          <token type="Operator"/>
+          <token type="NameAttribute"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(case|default|for|each|in|while|do|break|return|continue|if|else|throw|try|catch|with|new|typeof|arguments|instanceof|this|switch|import|include|as|is)\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="(class|public|final|internal|native|override|private|protected|static|import|extends|implements|interface|intrinsic|return|super|dynamic|function|const|get|namespace|package|set)\b">
+        <token type="KeywordDeclaration"/>
+      </rule>
+      <rule pattern="(true|false|null|NaN|Infinity|-Infinity|undefined|void)\b">
+        <token type="KeywordConstant"/>
+      </rule>
+      <rule pattern="(decodeURI|decodeURIComponent|encodeURI|escape|eval|isFinite|isNaN|isXMLName|clearInterval|fscommand|getTimer|getURL|getVersion|isFinite|parseFloat|parseInt|setInterval|trace|updateAfterEvent|unescape)\b">
+        <token type="NameFunction"/>
+      </rule>
+      <rule pattern="[$a-zA-Z_]\w*">
+        <token type="Name"/>
+      </rule>
+      <rule pattern="[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?">
+        <token type="LiteralNumberFloat"/>
+      </rule>
+      <rule pattern="0x[0-9a-f]+">
+        <token type="LiteralNumberHex"/>
+      </rule>
+      <rule pattern="[0-9]+">
+        <token type="LiteralNumberInteger"/>
+      </rule>
+      <rule pattern="&#34;(\\\\|\\&#34;|[^&#34;])*&#34;">
+        <token type="LiteralStringDouble"/>
+      </rule>
+      <rule pattern="&#39;(\\\\|\\&#39;|[^&#39;])*&#39;">
+        <token type="LiteralStringSingle"/>
+      </rule>
+      <rule pattern="[~^*!%&amp;&lt;&gt;|+=:;,/?\\{}\[\]().-]+">
+        <token type="Operator"/>
+      </rule>
+    </state>
+  </rules>
+</lexer>

lexers/embedded/ada.xml

@@ -0,0 +1,321 @@
+<lexer>
+  <config>
+    <name>Ada</name>
+    <alias>ada</alias>
+    <alias>ada95</alias>
+    <alias>ada2005</alias>
+    <filename>*.adb</filename>
+    <filename>*.ads</filename>
+    <filename>*.ada</filename>
+    <mime_type>text/x-ada</mime_type>
+    <case_insensitive>true</case_insensitive>
+  </config>
+  <rules>
+    <state name="end">
+      <rule pattern="(if|case|record|loop|select)">
+        <token type="KeywordReserved"/>
+      </rule>
+      <rule pattern="&#34;[^&#34;]+&#34;|[\w.]+">
+        <token type="NameFunction"/>
+      </rule>
+      <rule pattern="\s+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern=";">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+    </state>
+    <state name="array_def">
+      <rule pattern=";">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="(\w+)(\s+)(range)">
+        <bygroups>
+          <token type="KeywordType"/>
+          <token type="Text"/>
+          <token type="KeywordReserved"/>
+        </bygroups>
+      </rule>
+      <rule>
+        <include state="root"/>
+      </rule>
+    </state>
+    <state name="package_instantiation">
+      <rule pattern="(&#34;[^&#34;]+&#34;|\w+)(\s+)(=&gt;)">
+        <bygroups>
+          <token type="NameVariable"/>
+          <token type="Text"/>
+          <token type="Punctuation"/>
+        </bygroups>
+      </rule>
+      <rule pattern="[\w.\&#39;&#34;]">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="\)">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+      <rule>
+        <include state="root"/>
+      </rule>
+    </state>
+    <state name="subprogram">
+      <rule pattern="\(">
+        <token type="Punctuation"/>
+        <push state="#pop" state="formal_part"/>
+      </rule>
+      <rule pattern=";">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="is\b">
+        <token type="KeywordReserved"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="&#34;[^&#34;]+&#34;|\w+">
+        <token type="NameFunction"/>
+      </rule>
+      <rule>
+        <include state="root"/>
+      </rule>
+    </state>
+    <state name="type_def">
+      <rule pattern=";">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="\(">
+        <token type="Punctuation"/>
+        <push state="formal_part"/>
+      </rule>
+      <rule pattern="with|and|use">
+        <token type="KeywordReserved"/>
+      </rule>
+      <rule pattern="array\b">
+        <token type="KeywordReserved"/>
+        <push state="#pop" state="array_def"/>
+      </rule>
+      <rule pattern="record\b">
+        <token type="KeywordReserved"/>
+        <push state="record_def"/>
+      </rule>
+      <rule pattern="(null record)(;)">
+        <bygroups>
+          <token type="KeywordReserved"/>
+          <token type="Punctuation"/>
+        </bygroups>
+        <pop depth="1"/>
+      </rule>
+      <rule>
+        <include state="root"/>
+      </rule>
+    </state>
+    <state name="import">
+      <rule pattern="[\w.]+">
+        <token type="NameNamespace"/>
+        <pop depth="1"/>
+      </rule>
+      <rule>
+        <pop depth="1"/>
+      </rule>
+    </state>
+    <state name="formal_part">
+      <rule pattern="\)">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="\w+">
+        <token type="NameVariable"/>
+      </rule>
+      <rule pattern=",|:[^=]">
+        <token type="Punctuation"/>
+      </rule>
+      <rule pattern="(in|not|null|out|access)\b">
+        <token type="KeywordReserved"/>
+      </rule>
+      <rule>
+        <include state="root"/>
+      </rule>
+    </state>
+    <state name="package">
+      <rule pattern="body">
+        <token type="KeywordDeclaration"/>
+      </rule>
+      <rule pattern="is\s+new|renames">
+        <token type="KeywordReserved"/>
+      </rule>
+      <rule pattern="is">
+        <token type="KeywordReserved"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern=";">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="\(">
+        <token type="Punctuation"/>
+        <push state="package_instantiation"/>
+      </rule>
+      <rule pattern="([\w.]+)">
+        <token type="NameClass"/>
+      </rule>
+      <rule>
+        <include state="root"/>
+      </rule>
+    </state>
+    <state name="attribute">
+      <rule pattern="(&#39;)(\w+)">
+        <bygroups>
+          <token type="Punctuation"/>
+          <token type="NameAttribute"/>
+        </bygroups>
+      </rule>
+    </state>
+    <state name="record_def">
+      <rule pattern="end record">
+        <token type="KeywordReserved"/>
+        <pop depth="1"/>
+      </rule>
+      <rule>
+        <include state="root"/>
+      </rule>
+    </state>
+    <state name="root">
+      <rule pattern="[^\S\n]+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="--.*?\n">
+        <token type="CommentSingle"/>
+      </rule>
+      <rule pattern="[^\S\n]+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="function|procedure|entry">
+        <token type="KeywordDeclaration"/>
+        <push state="subprogram"/>
+      </rule>
+      <rule pattern="(subtype|type)(\s+)(\w+)">
+        <bygroups>
+          <token type="KeywordDeclaration"/>
+          <token type="Text"/>
+          <token type="KeywordType"/>
+        </bygroups>
+        <push state="type_def"/>
+      </rule>
+      <rule pattern="task|protected">
+        <token type="KeywordDeclaration"/>
+      </rule>
+      <rule pattern="(subtype)(\s+)">
+        <bygroups>
+          <token type="KeywordDeclaration"/>
+          <token type="Text"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(end)(\s+)">
+        <bygroups>
+          <token type="KeywordReserved"/>
+          <token type="Text"/>
+        </bygroups>
+        <push state="end"/>
+      </rule>
+      <rule pattern="(pragma)(\s+)(\w+)">
+        <bygroups>
+          <token type="KeywordReserved"/>
+          <token type="Text"/>
+          <token type="CommentPreproc"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(true|false|null)\b">
+        <token type="KeywordConstant"/>
+      </rule>
+      <rule pattern="(Short_Short_Integer|Short_Short_Float|Long_Long_Integer|Long_Long_Float|Wide_Character|Reference_Type|Short_Integer|Long_Integer|Wide_String|Short_Float|Controlled|Long_Float|Character|Generator|File_Type|File_Mode|Positive|Duration|Boolean|Natural|Integer|Address|Cursor|String|Count|Float|Byte)\b">
+        <token type="KeywordType"/>
+      </rule>
+      <rule pattern="(and(\s+then)?|in|mod|not|or(\s+else)|rem)\b">
+        <token type="OperatorWord"/>
+      </rule>
+      <rule pattern="generic|private">
+        <token type="KeywordDeclaration"/>
+      </rule>
+      <rule pattern="package">
+        <token type="KeywordDeclaration"/>
+        <push state="package"/>
+      </rule>
+      <rule pattern="array\b">
+        <token type="KeywordReserved"/>
+        <push state="array_def"/>
+      </rule>
+      <rule pattern="(with|use)(\s+)">
+        <bygroups>
+          <token type="KeywordNamespace"/>
+          <token type="Text"/>
+        </bygroups>
+        <push state="import"/>
+      </rule>
+      <rule pattern="(\w+)(\s*)(:)(\s*)(constant)">
+        <bygroups>
+          <token type="NameConstant"/>
+          <token type="Text"/>
+          <token type="Punctuation"/>
+          <token type="Text"/>
+          <token type="KeywordReserved"/>
+        </bygroups>
+      </rule>
+      <rule pattern="&lt;&lt;\w+&gt;&gt;">
+        <token type="NameLabel"/>
+      </rule>
+      <rule pattern="(\w+)(\s*)(:)(\s*)(declare|begin|loop|for|while)">
+        <bygroups>
+          <token type="NameLabel"/>
+          <token type="Text"/>
+          <token type="Punctuation"/>
+          <token type="Text"/>
+          <token type="KeywordReserved"/>
+        </bygroups>
+      </rule>
+      <rule pattern="\b(synchronized|overriding|terminate|interface|exception|protected|separate|constant|abstract|renames|reverse|subtype|aliased|declare|requeue|limited|return|tagged|access|record|select|accept|digits|others|pragma|entry|elsif|delta|delay|array|until|range|raise|while|begin|abort|else|loop|when|type|null|then|body|task|goto|case|exit|end|for|abs|xor|all|new|out|is|of|if|or|do|at)\b">
+        <token type="KeywordReserved"/>
+      </rule>
+      <rule pattern="&#34;[^&#34;]*&#34;">
+        <token type="LiteralString"/>
+      </rule>
+      <rule>
+        <include state="attribute"/>
+      </rule>
+      <rule>
+        <include state="numbers"/>
+      </rule>
+      <rule pattern="&#39;[^&#39;]&#39;">
+        <token type="LiteralStringChar"/>
+      </rule>
+      <rule pattern="(\w+)(\s*|[(,])">
+        <bygroups>
+          <token type="Name"/>
+          <usingself state="root"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(&lt;&gt;|=&gt;|:=|[()|:;,.&#39;])">
+        <token type="Punctuation"/>
+      </rule>
+      <rule pattern="[*&lt;&gt;+=/&amp;-]">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="\n+">
+        <token type="Text"/>
+      </rule>
+    </state>
+    <state name="numbers">
+      <rule pattern="[0-9_]+#[0-9a-f]+#">
+        <token type="LiteralNumberHex"/>
+      </rule>
+      <rule pattern="[0-9_]+\.[0-9_]*">
+        <token type="LiteralNumberFloat"/>
+      </rule>
+      <rule pattern="[0-9_]+">
+        <token type="LiteralNumberInteger"/>
+      </rule>
+    </state>
+  </rules>
+</lexer>

lexers/embedded/al.xml

@@ -0,0 +1,75 @@
+<lexer>
+  <config>
+    <name>AL</name>
+    <alias>al</alias>
+    <filename>*.al</filename>
+    <filename>*.dal</filename>
+    <mime_type>text/x-al</mime_type>
+    <case_insensitive>true</case_insensitive>
+    <dot_all>true</dot_all>
+  </config>
+  <rules>
+    <state name="root">
+      <rule pattern="\s+">
+        <token type="TextWhitespace"/>
+      </rule>
+      <rule pattern="(?s)\/\*.*?\\*\*\/">
+        <token type="CommentMultiline"/>
+      </rule>
+      <rule pattern="(?s)//.*?\n">
+        <token type="CommentSingle"/>
+      </rule>
+      <rule pattern="\&#34;([^\&#34;])*\&#34;">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="&#39;([^&#39;])*&#39;">
+        <token type="LiteralString"/>
+      </rule>
+      <rule pattern="\b(?i:(ARRAY|ASSERTERROR|BEGIN|BREAK|CASE|DO|DOWNTO|ELSE|END|EVENT|EXIT|FOR|FOREACH|FUNCTION|IF|IMPLEMENTS|IN|INDATASET|INTERFACE|INTERNAL|LOCAL|OF|PROCEDURE|PROGRAM|PROTECTED|REPEAT|RUNONCLIENT|SECURITYFILTERING|SUPPRESSDISPOSE|TEMPORARY|THEN|TO|TRIGGER|UNTIL|VAR|WHILE|WITH|WITHEVENTS))\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="\b(?i:(AND|DIV|MOD|NOT|OR|XOR))\b">
+        <token type="OperatorWord"/>
+      </rule>
+      <rule pattern="\b(?i:(AVERAGE|CONST|COUNT|EXIST|FIELD|FILTER|LOOKUP|MAX|MIN|ORDER|SORTING|SUM|TABLEDATA|UPPERLIMIT|WHERE|ASCENDING|DESCENDING))\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="\b(?i:(CODEUNIT|PAGE|PAGEEXTENSION|PAGECUSTOMIZATION|DOTNET|ENUM|ENUMEXTENSION|VALUE|QUERY|REPORT|TABLE|TABLEEXTENSION|XMLPORT|PROFILE|CONTROLADDIN|REPORTEXTENSION|INTERFACE|PERMISSIONSET|PERMISSIONSETEXTENSION|ENTITLEMENT))\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="\b(?i:(Action|Array|Automation|BigInteger|BigText|Blob|Boolean|Byte|Char|ClientType|Code|Codeunit|CompletionTriggerErrorLevel|ConnectionType|Database|DataClassification|DataScope|Date|DateFormula|DateTime|Decimal|DefaultLayout|Dialog|Dictionary|DotNet|DotNetAssembly|DotNetTypeDeclaration|Duration|Enum|ErrorInfo|ErrorType|ExecutionContext|ExecutionMode|FieldClass|FieldRef|FieldType|File|FilterPageBuilder|Guid|InStream|Integer|Joker|KeyRef|List|ModuleDependencyInfo|ModuleInfo|None|Notification|NotificationScope|ObjectType|Option|OutStream|Page|PageResult|Query|Record|RecordId|RecordRef|Report|ReportFormat|SecurityFilter|SecurityFiltering|Table|TableConnectionType|TableFilter|TestAction|TestField|TestFilterField|TestPage|TestPermissions|TestRequestPage|Text|TextBuilder|TextConst|TextEncoding|Time|TransactionModel|TransactionType|Variant|Verbosity|Version|XmlPort|HttpContent|HttpHeaders|HttpClient|HttpRequestMessage|HttpResponseMessage|JsonToken|JsonValue|JsonArray|JsonObject|View|Views|XmlAttribute|XmlAttributeCollection|XmlComment|XmlCData|XmlDeclaration|XmlDocument|XmlDocumentType|XmlElement|XmlNamespaceManager|XmlNameTable|XmlNode|XmlNodeList|XmlProcessingInstruction|XmlReadOptions|XmlText|XmlWriteOptions|WebServiceActionContext|WebServiceActionResultCode|SessionSettings))\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="\b([&lt;&gt;]=|&lt;&gt;|&lt;|&gt;)\b?">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="\b(\-|\+|\/|\*)\b">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="\s*(\:=|\+=|-=|\/=|\*=)\s*?">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="\b(?i:(ADD|ADDFIRST|ADDLAST|ADDAFTER|ADDBEFORE|ACTION|ACTIONS|AREA|ASSEMBLY|CHARTPART|CUEGROUP|CUSTOMIZES|COLUMN|DATAITEM|DATASET|ELEMENTS|EXTENDS|FIELD|FIELDGROUP|FIELDATTRIBUTE|FIELDELEMENT|FIELDGROUPS|FIELDS|FILTER|FIXED|GRID|GROUP|MOVEAFTER|MOVEBEFORE|KEY|KEYS|LABEL|LABELS|LAYOUT|MODIFY|MOVEFIRST|MOVELAST|MOVEBEFORE|MOVEAFTER|PART|REPEATER|USERCONTROL|REQUESTPAGE|SCHEMA|SEPARATOR|SYSTEMPART|TABLEELEMENT|TEXTATTRIBUTE|TEXTELEMENT|TYPE))\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="\s*[(\.\.)&amp;\|]\s*">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern="\b((0(x|X)[0-9a-fA-F]*)|(([0-9]+\.?[0-9]*)|(\.[0-9]+))((e|E)(\+|-)?[0-9]+)?)(L|l|UL|ul|u|U|F|f|ll|LL|ull|ULL)?\b">
+        <token type="LiteralNumber"/>
+      </rule>
+      <rule pattern="[;:,]">
+        <token type="Punctuation"/>
+      </rule>
+      <rule pattern="#[ \t]*(if|else|elif|endif|define|undef|region|endregion|pragma)\b.*?\n">
+        <token type="CommentPreproc"/>
+      </rule>
+      <rule pattern="\w+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern=".">
+        <token type="Text"/>
+      </rule>
+    </state>
+  </rules>
+</lexer>

lexers/embedded/angular2.xml

@@ -0,0 +1,108 @@
+<lexer>
+  <config>
+    <name>Angular2</name>
+    <alias>ng2</alias>
+  </config>
+  <rules>
+    <state name="attr">
+      <rule pattern="&#34;.*?&#34;">
+        <token type="LiteralString"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="&#39;.*?&#39;">
+        <token type="LiteralString"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="[^\s&gt;]+">
+        <token type="LiteralString"/>
+        <pop depth="1"/>
+      </rule>
+    </state>
+    <state name="root">
+      <rule pattern="[^{([*#]+">
+        <token type="Other"/>
+      </rule>
+      <rule pattern="(\{\{)(\s*)">
+        <bygroups>
+          <token type="CommentPreproc"/>
+          <token type="Text"/>
+        </bygroups>
+        <push state="ngExpression"/>
+      </rule>
+      <rule pattern="([([]+)([\w:.-]+)([\])]+)(\s*)(=)(\s*)">
+        <bygroups>
+          <token type="Punctuation"/>
+          <token type="NameAttribute"/>
+          <token type="Punctuation"/>
+          <token type="Text"/>
+          <token type="Operator"/>
+          <token type="Text"/>
+        </bygroups>
+        <push state="attr"/>
+      </rule>
+      <rule pattern="([([]+)([\w:.-]+)([\])]+)(\s*)">
+        <bygroups>
+          <token type="Punctuation"/>
+          <token type="NameAttribute"/>
+          <token type="Punctuation"/>
+          <token type="Text"/>
+        </bygroups>
+      </rule>
+      <rule pattern="([*#])([\w:.-]+)(\s*)(=)(\s*)">
+        <bygroups>
+          <token type="Punctuation"/>
+          <token type="NameAttribute"/>
+          <token type="Punctuation"/>
+          <token type="Operator"/>
+        </bygroups>
+        <push state="attr"/>
+      </rule>
+      <rule pattern="([*#])([\w:.-]+)(\s*)">
+        <bygroups>
+          <token type="Punctuation"/>
+          <token type="NameAttribute"/>
+          <token type="Punctuation"/>
+        </bygroups>
+      </rule>
+    </state>
+    <state name="ngExpression">
+      <rule pattern="\s+(\|\s+)?">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="\}\}">
+        <token type="CommentPreproc"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern=":?(true|false)">
+        <token type="LiteralStringBoolean"/>
+      </rule>
+      <rule pattern=":?&#34;(\\\\|\\&#34;|[^&#34;])*&#34;">
+        <token type="LiteralStringDouble"/>
+      </rule>
+      <rule pattern=":?&#39;(\\\\|\\&#39;|[^&#39;])*&#39;">
+        <token type="LiteralStringSingle"/>
+      </rule>
+      <rule pattern="[0-9](\.[0-9]*)?(eE[+-][0-9])?[flFLdD]?|0[xX][0-9a-fA-F]+[Ll]?">
+        <token type="LiteralNumber"/>
+      </rule>
+      <rule pattern="[a-zA-Z][\w-]*(\(.*\))?">
+        <token type="NameVariable"/>
+      </rule>
+      <rule pattern="\.[\w-]+(\(.*\))?">
+        <token type="NameVariable"/>
+      </rule>
+      <rule pattern="(\?)(\s*)([^}\s]+)(\s*)(:)(\s*)([^}\s]+)(\s*)">
+        <bygroups>
+          <token type="Operator"/>
+          <token type="Text"/>
+          <token type="LiteralString"/>
+          <token type="Text"/>
+          <token type="Operator"/>
+          <token type="Text"/>
+          <token type="LiteralString"/>
+          <token type="Text"/>
+        </bygroups>
+      </rule>
+    </state>
+  </rules>
+</lexer>

lexers/embedded/antlr.xml

@@ -0,0 +1,317 @@
+<lexer>
+  <config>
+    <name>ANTLR</name>
+    <alias>antlr</alias>
+  </config>
+  <rules>
+    <state name="nested-arg-action">
+      <rule pattern="([^$\[\]\&#39;&#34;/]+|&#34;(\\\\|\\&#34;|[^&#34;])*&#34;|&#39;(\\\\|\\&#39;|[^&#39;])*&#39;|//.*$\n?|/\*(.|\n)*?\*/|/(?!\*)(\\\\|\\/|[^/])*/|/)+">
+        <token type="Other"/>
+      </rule>
+      <rule pattern="\[">
+        <token type="Punctuation"/>
+        <push/>
+      </rule>
+      <rule pattern="\]">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="(\$[a-zA-Z]+)(\.?)(text|value)?">
+        <bygroups>
+          <token type="NameVariable"/>
+          <token type="Punctuation"/>
+          <token type="NameProperty"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(\\\\|\\\]|\\\[|[^\[\]])+">
+        <token type="Other"/>
+      </rule>
+    </state>
+    <state name="exception">
+      <rule pattern="\n">
+        <token type="TextWhitespace"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="\s">
+        <token type="TextWhitespace"/>
+      </rule>
+      <rule>
+        <include state="comments"/>
+      </rule>
+      <rule pattern="\[">
+        <token type="Punctuation"/>
+        <push state="nested-arg-action"/>
+      </rule>
+      <rule pattern="\{">
+        <token type="Punctuation"/>
+        <push state="action"/>
+      </rule>
+    </state>
+    <state name="whitespace">
+      <rule pattern="\s+">
+        <token type="TextWhitespace"/>
+      </rule>
+    </state>
+    <state name="root">
+      <rule>
+        <include state="whitespace"/>
+      </rule>
+      <rule>
+        <include state="comments"/>
+      </rule>
+      <rule pattern="(lexer|parser|tree)?(\s*)(grammar\b)(\s*)([A-Za-z]\w*)(;)">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="TextWhitespace"/>
+          <token type="Keyword"/>
+          <token type="TextWhitespace"/>
+          <token type="NameClass"/>
+          <token type="Punctuation"/>
+        </bygroups>
+      </rule>
+      <rule pattern="options\b">
+        <token type="Keyword"/>
+        <push state="options"/>
+      </rule>
+      <rule pattern="tokens\b">
+        <token type="Keyword"/>
+        <push state="tokens"/>
+      </rule>
+      <rule pattern="(scope)(\s*)([A-Za-z]\w*)(\s*)(\{)">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="TextWhitespace"/>
+          <token type="NameVariable"/>
+          <token type="TextWhitespace"/>
+          <token type="Punctuation"/>
+        </bygroups>
+        <push state="action"/>
+      </rule>
+      <rule pattern="(catch|finally)\b">
+        <token type="Keyword"/>
+        <push state="exception"/>
+      </rule>
+      <rule pattern="(@[A-Za-z]\w*)(\s*)(::)?(\s*)([A-Za-z]\w*)(\s*)(\{)">
+        <bygroups>
+          <token type="NameLabel"/>
+          <token type="TextWhitespace"/>
+          <token type="Punctuation"/>
+          <token type="TextWhitespace"/>
+          <token type="NameLabel"/>
+          <token type="TextWhitespace"/>
+          <token type="Punctuation"/>
+        </bygroups>
+        <push state="action"/>
+      </rule>
+      <rule pattern="((?:protected|private|public|fragment)\b)?(\s*)([A-Za-z]\w*)(!)?">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="TextWhitespace"/>
+          <token type="NameLabel"/>
+          <token type="Punctuation"/>
+        </bygroups>
+        <push state="rule-alts" state="rule-prelims"/>
+      </rule>
+    </state>
+    <state name="tokens">
+      <rule>
+        <include state="whitespace"/>
+      </rule>
+      <rule>
+        <include state="comments"/>
+      </rule>
+      <rule pattern="\{">
+        <token type="Punctuation"/>
+      </rule>
+      <rule pattern="([A-Z]\w*)(\s*)(=)?(\s*)(\&#39;(?:\\\\|\\\&#39;|[^\&#39;]*)\&#39;)?(\s*)(;)">
+        <bygroups>
+          <token type="NameLabel"/>
+          <token type="TextWhitespace"/>
+          <token type="Punctuation"/>
+          <token type="TextWhitespace"/>
+          <token type="LiteralString"/>
+          <token type="TextWhitespace"/>
+          <token type="Punctuation"/>
+        </bygroups>
+      </rule>
+      <rule pattern="\}">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+    </state>
+    <state name="options">
+      <rule>
+        <include state="whitespace"/>
+      </rule>
+      <rule>
+        <include state="comments"/>
+      </rule>
+      <rule pattern="\{">
+        <token type="Punctuation"/>
+      </rule>
+      <rule pattern="([A-Za-z]\w*)(\s*)(=)(\s*)([A-Za-z]\w*|\&#39;(?:\\\\|\\\&#39;|[^\&#39;]*)\&#39;|[0-9]+|\*)(\s*)(;)">
+        <bygroups>
+          <token type="NameVariable"/>
+          <token type="TextWhitespace"/>
+          <token type="Punctuation"/>
+          <token type="TextWhitespace"/>
+          <token type="Text"/>
+          <token type="TextWhitespace"/>
+          <token type="Punctuation"/>
+        </bygroups>
+      </rule>
+      <rule pattern="\}">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+    </state>
+    <state name="rule-alts">
+      <rule>
+        <include state="whitespace"/>
+      </rule>
+      <rule>
+        <include state="comments"/>
+      </rule>
+      <rule pattern="options\b">
+        <token type="Keyword"/>
+        <push state="options"/>
+      </rule>
+      <rule pattern=":">
+        <token type="Punctuation"/>
+      </rule>
+      <rule pattern="&#39;(\\\\|\\&#39;|[^&#39;])*&#39;">
+        <token type="LiteralString"/>
+      </rule>
+      <rule pattern="&#34;(\\\\|\\&#34;|[^&#34;])*&#34;">
+        <token type="LiteralString"/>
+      </rule>
+      <rule pattern="&lt;&lt;([^&gt;]|&gt;[^&gt;])&gt;&gt;">
+        <token type="LiteralString"/>
+      </rule>
+      <rule pattern="\$?[A-Z_]\w*">
+        <token type="NameConstant"/>
+      </rule>
+      <rule pattern="\$?[a-z_]\w*">
+        <token type="NameVariable"/>
+      </rule>
+      <rule pattern="(\+|\||-&gt;|=&gt;|=|\(|\)|\.\.|\.|\?|\*|\^|!|\#|~)">
+        <token type="Operator"/>
+      </rule>
+      <rule pattern=",">
+        <token type="Punctuation"/>
+      </rule>
+      <rule pattern="\[">
+        <token type="Punctuation"/>
+        <push state="nested-arg-action"/>
+      </rule>
+      <rule pattern="\{">
+        <token type="Punctuation"/>
+        <push state="action"/>
+      </rule>
+      <rule pattern=";">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+    </state>
+    <state name="rule-prelims">
+      <rule>
+        <include state="whitespace"/>
+      </rule>
+      <rule>
+        <include state="comments"/>
+      </rule>
+      <rule pattern="returns\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="\[">
+        <token type="Punctuation"/>
+        <push state="nested-arg-action"/>
+      </rule>
+      <rule pattern="\{">
+        <token type="Punctuation"/>
+        <push state="action"/>
+      </rule>
+      <rule pattern="(throws)(\s+)([A-Za-z]\w*)">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="TextWhitespace"/>
+          <token type="NameLabel"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(,)(\s*)([A-Za-z]\w*)">
+        <bygroups>
+          <token type="Punctuation"/>
+          <token type="TextWhitespace"/>
+          <token type="NameLabel"/>
+        </bygroups>
+      </rule>
+      <rule pattern="options\b">
+        <token type="Keyword"/>
+        <push state="options"/>
+      </rule>
+      <rule pattern="(scope)(\s+)(\{)">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="TextWhitespace"/>
+          <token type="Punctuation"/>
+        </bygroups>
+        <push state="action"/>
+      </rule>
+      <rule pattern="(scope)(\s+)([A-Za-z]\w*)(\s*)(;)">
+        <bygroups>
+          <token type="Keyword"/>
+          <token type="TextWhitespace"/>
+          <token type="NameLabel"/>
+          <token type="TextWhitespace"/>
+          <token type="Punctuation"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(@[A-Za-z]\w*)(\s*)(\{)">
+        <bygroups>
+          <token type="NameLabel"/>
+          <token type="TextWhitespace"/>
+          <token type="Punctuation"/>
+        </bygroups>
+        <push state="action"/>
+      </rule>
+      <rule pattern=":">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+    </state>
+    <state name="action">
+      <rule pattern="([^${}\&#39;&#34;/\\]+|&#34;(\\\\|\\&#34;|[^&#34;])*&#34;|&#39;(\\\\|\\&#39;|[^&#39;])*&#39;|//.*$\n?|/\*(.|\n)*?\*/|/(?!\*)(\\\\|\\/|[^/])*/|\\(?!%)|/)+">
+        <token type="Other"/>
+      </rule>
+      <rule pattern="(\\)(%)">
+        <bygroups>
+          <token type="Punctuation"/>
+          <token type="Other"/>
+        </bygroups>
+      </rule>
+      <rule pattern="(\$[a-zA-Z]+)(\.?)(text|value)?">
+        <bygroups>
+          <token type="NameVariable"/>
+          <token type="Punctuation"/>
+          <token type="NameProperty"/>
+        </bygroups>
+      </rule>
+      <rule pattern="\{">
+        <token type="Punctuation"/>
+        <push/>
+      </rule>
+      <rule pattern="\}">
+        <token type="Punctuation"/>
+        <pop depth="1"/>
+      </rule>
+    </state>
+    <state name="comments">
+      <rule pattern="//.*$">
+        <token type="Comment"/>
+      </rule>
+      <rule pattern="/\*(.|\n)*?\*/">
+        <token type="Comment"/>
+      </rule>
+    </state>
+  </rules>
+</lexer>

lexers/embedded/apacheconf.xml

@@ -0,0 +1,74 @@
+<lexer>
+  <config>
+    <name>ApacheConf</name>
+    <alias>apacheconf</alias>
+    <alias>aconf</alias>
+    <alias>apache</alias>
+    <filename>.htaccess</filename>
+    <filename>apache.conf</filename>
+    <filename>apache2.conf</filename>
+    <mime_type>text/x-apacheconf</mime_type>
+    <case_insensitive>true</case_insensitive>
+  </config>
+  <rules>
+    <state name="root">
+      <rule pattern="\s+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="(#.*?)$">
+        <token type="Comment"/>
+      </rule>
+      <rule pattern="(&lt;[^\s&gt;]+)(?:(\s+)(.*?))?(&gt;)">
+        <bygroups>
+          <token type="NameTag"/>
+          <token type="Text"/>
+          <token type="LiteralString"/>
+          <token type="NameTag"/>
+        </bygroups>
+      </rule>
+      <rule pattern="([a-z]\w*)(\s+)">
+        <bygroups>
+          <token type="NameBuiltin"/>
+          <token type="Text"/>
+        </bygroups>
+        <push state="value"/>
+      </rule>
+      <rule pattern="\.+">
+        <token type="Text"/>
+      </rule>
+    </state>
+    <state name="value">
+      <rule pattern="\\\n">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="$">
+        <token type="Text"/>
+        <pop depth="1"/>
+      </rule>
+      <rule pattern="\\">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="[^\S\n]+">
+        <token type="Text"/>
+      </rule>
+      <rule pattern="\d+\.\d+\.\d+\.\d+(?:/\d+)?">
+        <token type="LiteralNumber"/>
+      </rule>
+      <rule pattern="\d+">
+        <token type="LiteralNumber"/>
+      </rule>
+      <rule pattern="/([a-z0-9][\w./-]+)">
+        <token type="LiteralStringOther"/>
+      </rule>
+      <rule pattern="(on|off|none|any|all|double|email|dns|min|minimal|os|productonly|full|emerg|alert|crit|error|warn|notice|info|debug|registry|script|inetd|standalone|user|group)\b">
+        <token type="Keyword"/>
+      </rule>
+      <rule pattern="&#34;([^&#34;\\]*(?:\\.[^&#34;\\]*)*)&#34;">
+        <token type="LiteralStringDouble"/>
+      </rule>
+      <rule pattern="[^\s&#34;\\]+">
+        <token type="Text"/>
+      </rule>
+    </state>
+  </rules>
+</lexer>