Bump the go-modules-updates group in /backend with 8 updates

Bumps the go-modules-updates group in /backend with 8 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/PuerkitoBio/goquery](https://github.com/PuerkitoBio/goquery) | `1.10.1` | `1.10.2` |
| [github.com/alecthomas/chroma/v2](https://github.com/alecthomas/chroma) | `2.14.0` | `2.15.0` |
| [github.com/go-chi/chi/v5](https://github.com/go-chi/chi) | `5.2.0` | `5.2.1` |
| [github.com/go-pkgz/jrpc](https://github.com/go-pkgz/jrpc) | `0.3.0` | `0.3.1` |
| [go.etcd.io/bbolt](https://github.com/etcd-io/bbolt) | `1.3.11` | `1.4.0` |
| [golang.org/x/crypto](https://github.com/golang/crypto) | `0.31.0` | `0.33.0` |
| [golang.org/x/image](https://github.com/golang/image) | `0.23.0` | `0.25.0` |
| [golang.org/x/net](https://github.com/golang/net) | `0.33.0` | `0.35.0` |

Updates `github.com/PuerkitoBio/goquery` from 1.10.1 to 1.10.2
- [Release notes](https://github.com/PuerkitoBio/goquery/releases)
- [Commits](https://github.com/PuerkitoBio/goquery/compare/v1.10.1...v1.10.2)

Updates `github.com/alecthomas/chroma/v2` from 2.14.0 to 2.15.0
- [Release notes](https://github.com/alecthomas/chroma/releases)
- [Changelog](https://github.com/alecthomas/chroma/blob/master/.goreleaser.yml)
- [Commits](https://github.com/alecthomas/chroma/compare/v2.14.0...v2.15.0)

Updates `github.com/go-chi/chi/v5` from 5.2.0 to 5.2.1
- [Release notes](https://github.com/go-chi/chi/releases)
- [Changelog](https://github.com/go-chi/chi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/go-chi/chi/compare/v5.2.0...v5.2.1)

Updates `github.com/go-pkgz/jrpc` from 0.3.0 to 0.3.1
- [Release notes](https://github.com/go-pkgz/jrpc/releases)
- [Commits](https://github.com/go-pkgz/jrpc/compare/v0.3.0...v0.3.1)

Updates `go.etcd.io/bbolt` from 1.3.11 to 1.4.0
- [Release notes](https://github.com/etcd-io/bbolt/releases)
- [Commits](https://github.com/etcd-io/bbolt/compare/v1.3.11...v1.4.0)

Updates `golang.org/x/crypto` from 0.31.0 to 0.33.0
- [Commits](https://github.com/golang/crypto/compare/v0.31.0...v0.33.0)

Updates `golang.org/x/image` from 0.23.0 to 0.25.0
- [Commits](https://github.com/golang/image/compare/v0.23.0...v0.25.0)

Updates `golang.org/x/net` from 0.33.0 to 0.35.0
- [Commits](https://github.com/golang/net/compare/v0.33.0...v0.35.0)

---
updated-dependencies:
- dependency-name: github.com/PuerkitoBio/goquery
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-modules-updates
- dependency-name: github.com/alecthomas/chroma/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-modules-updates
- dependency-name: github.com/go-chi/chi/v5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-modules-updates
- dependency-name: github.com/go-pkgz/jrpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-modules-updates
- dependency-name: go.etcd.io/bbolt
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-modules-updates
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-modules-updates
- dependency-name: golang.org/x/image
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-modules-updates
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-modules-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
This commit is contained in:
dependabot[bot]
2025-03-24 23:13:46 +01:00
committed by Dmitry V
parent e6afc58b34
commit 89221ff2bc
100 changed files with 7038 additions and 2499 deletions
+4 -3
View File
@@ -24,9 +24,9 @@ Syntax-wise, it is as close as possible to jQuery, with the same function names
Required Go version:
* Starting with version `v1.10.0` of goquery, Go 1.23+ is required due to the use of function-based iterators.
* For `v1.9.0` of goquery, Go 1.18+ is required due to the use of generics.
* For previous goquery versions, a Go version of 1.1+ was required because of the `net/html` dependency.
* Starting with version `v1.10.0` of goquery, Go 1.23+ is required due to the use of function-based iterators.
* For `v1.9.0` of goquery, Go 1.18+ is required due to the use of generics.
* For previous goquery versions, a Go version of 1.1+ was required because of the `net/html` dependency.
Ongoing goquery development is tested on the latest 2 versions of Go.
@@ -46,6 +46,7 @@ Ongoing goquery development is tested on the latest 2 versions of Go.
**Note that goquery's API is now stable, and will not break.**
* **2025-02-13 (v1.10.2)** : Update `go.mod` dependencies, add go1.24 to the test matrix.
* **2024-12-26 (v1.10.1)** : Update `go.mod` dependencies.
* **2024-09-06 (v1.10.0)** : Add `EachIter` which provides an iterator that can be used in `for..range` loops on the `*Selection` object. **goquery now requires Go version 1.23+** (thanks [@amikai](https://github.com/amikai)).
* **2024-09-06 (v1.9.3)** : Update `go.mod` dependencies.
+3
View File
@@ -51,6 +51,9 @@ linters:
- musttag
- depguard
- goconst
- perfsprint
- mnd
- predeclared
linters-settings:
govet:
+7 -7
View File
@@ -39,20 +39,20 @@ translators for Pygments lexers and styles.
| D | D, Dart, Dax, Desktop Entry, Diff, Django/Jinja, dns, Docker, DTD, Dylan |
| E | EBNF, Elixir, Elm, EmacsLisp, Erlang |
| F | Factor, Fennel, Fish, Forth, Fortran, FortranFixed, FSharp |
| G | GAS, GDScript, Genshi, Genshi HTML, Genshi Text, Gherkin, GLSL, Gnuplot, Go, Go HTML Template, Go Text Template, GraphQL, Groff, Groovy |
| G | GAS, GDScript, Genshi, Genshi HTML, Genshi Text, Gherkin, Gleam, GLSL, Gnuplot, Go, Go HTML Template, Go Text Template, GraphQL, Groff, Groovy |
| H | Handlebars, Hare, Haskell, Haxe, HCL, Hexdump, HLB, HLSL, HolyC, HTML, HTTP, Hy |
| I | Idris, Igor, INI, Io, ISCdhcpd |
| J | J, Java, JavaScript, JSON, Julia, Jungle |
| J | J, Java, JavaScript, JSON, Jsonnet, Julia, Jungle |
| K | Kotlin |
| L | Lighttpd configuration file, LLVM, Lua |
| M | Makefile, Mako, markdown, Mason, Materialize SQL dialect, Mathematica, Matlab, mcfunction, Meson, Metal, MiniZinc, MLIR, Modula-2, MonkeyC, MorrowindScript, Myghty, MySQL |
| N | NASM, Natural, Newspeak, Nginx configuration file, Nim, Nix |
| M | Makefile, Mako, markdown, Mason, Materialize SQL dialect, Mathematica, Matlab, MCFunction, Meson, Metal, MiniZinc, MLIR, Modula-2, MonkeyC, MorrowindScript, Myghty, MySQL |
| N | NASM, Natural, Newspeak, Nginx configuration file, Nim, Nix, NSIS |
| O | Objective-C, OCaml, Octave, Odin, OnesEnterprise, OpenEdge ABL, OpenSCAD, Org Mode |
| P | PacmanConf, Perl, PHP, PHTML, Pig, PkgConfig, PL/pgSQL, plaintext, Plutus Core, Pony, PostgreSQL SQL dialect, PostScript, POVRay, PowerQuery, PowerShell, Prolog, PromQL, Promela, properties, Protocol Buffer, PRQL, PSL, Puppet, Python, Python 2 |
| Q | QBasic, QML |
| R | R, Racket, Ragel, Raku, react, ReasonML, reg, Rego, reStructuredText, Rexx, RPMSpec, Ruby, Rust |
| S | SAS, Sass, Scala, Scheme, Scilab, SCSS, Sed, Sieve, Smali, Smalltalk, Smarty, Snobol, Solidity, SourcePawn, SPARQL, SQL, SquidConf, Standard ML, stas, Stylus, Svelte, Swift, SYSTEMD, systemverilog |
| T | TableGen, Tal, TASM, Tcl, Tcsh, Termcap, Terminfo, Terraform, TeX, Thrift, TOML, TradingView, Transact-SQL, Turing, Turtle, Twig, TypeScript, TypoScript, TypoScriptCssData, TypoScriptHtmlData |
| S | SAS, Sass, Scala, Scheme, Scilab, SCSS, Sed, Sieve, Smali, Smalltalk, Smarty, SNBT, Snobol, Solidity, SourcePawn, SPARQL, SQL, SquidConf, Standard ML, stas, Stylus, Svelte, Swift, SYSTEMD, systemverilog |
| T | TableGen, Tal, TASM, Tcl, Tcsh, Termcap, Terminfo, Terraform, TeX, Thrift, TOML, TradingView, Transact-SQL, Turing, Turtle, Twig, TypeScript, TypoScript, TypoScriptCssData, TypoScriptHtmlData, Typst |
| V | V, V shell, Vala, VB.net, verilog, VHDL, VHS, VimL, vue |
| W | WDTE, WebGPU Shading Language, Whiley |
| X | XML, Xorg |
@@ -272,7 +272,7 @@ for that setup the `chroma` executable can be just symlinked to `~/.lessfilter`.
If you edit some lexers and want to try it, open a shell in `cmd/chromad` and run:
```shell
go run .
go run . --csrf-key=securekey
```
A Link will be printed. Open it in your Browser. Now you can test on the Playground with your local changes.
+4 -4
View File
@@ -141,7 +141,7 @@ func ParseColour(colour string) Colour {
if err != nil {
return 0
}
return Colour(n + 1)
return Colour(n + 1) //nolint:gosec
}
// MustParseColour is like ParseColour except it panics if the colour is invalid.
@@ -162,13 +162,13 @@ func (c Colour) String() string { return fmt.Sprintf("#%06x", int(c-1)) }
func (c Colour) GoString() string { return fmt.Sprintf("Colour(0x%06x)", int(c-1)) }
// Red component of colour.
func (c Colour) Red() uint8 { return uint8(((c - 1) >> 16) & 0xff) }
func (c Colour) Red() uint8 { return uint8(((c - 1) >> 16) & 0xff) } //nolint:gosec
// Green component of colour.
func (c Colour) Green() uint8 { return uint8(((c - 1) >> 8) & 0xff) }
func (c Colour) Green() uint8 { return uint8(((c - 1) >> 8) & 0xff) } //nolint:gosec
// Blue component of colour.
func (c Colour) Blue() uint8 { return uint8((c - 1) & 0xff) }
func (c Colour) Blue() uint8 { return uint8((c - 1) & 0xff) } //nolint:gosec
// Colours is an orderable set of colours.
type Colours []Colour
@@ -152,7 +152,7 @@
<rule pattern="(?-i)(?:CURRENT|NEW|OLD)\b">
<token type="NameBuiltinPseudo"/>
</rule>
<rule pattern="(?:to_bool|to_number|to_string|to_array|to_list|is_null|is_bool|is_number|is_string|is_array|is_list|is_object|is_document|is_datestring|typename|json_stringify|json_parse|concat|concat_separator|char_length|lower|upper|substring|substring_bytes|left|right|trim|reverse|contains|log|log2|log10|exp|exp2|sin|cos|tan|asin|acos|atan|atan2|radians|degrees|pi|regex_test|regex_replace|like|floor|ceil|round|abs|rand|sqrt|pow|length|count|min|max|average|avg|sum|product|median|variance_population|variance_sample|variance|percentile|bit_and|bit_or|bit_xor|bit_negate|bit_test|bit_popcount|bit_shift_left|bit_shift_right|bit_construct|bit_deconstruct|bit_to_string|bit_from_string|first|last|unique|outersection|interleave|in_range|jaccard|matches|merge|merge_recursive|has|attributes|keys|values|unset|unset_recursive|keep|keep_recursive|near|within|within_rectangle|is_in_polygon|distance|fulltext|stddev_sample|stddev_population|stddev|slice|nth|position|contains_array|translate|zip|call|apply|push|append|pop|shift|unshift|remove_value|remove_values|remove_nth|replace_nth|date_now|date_timestamp|date_iso8601|date_dayofweek|date_year|date_month|date_day|date_hour|date_minute|date_second|date_millisecond|date_dayofyear|date_isoweek|date_isoweekyear|date_leapyear|date_quarter|date_days_in_month|date_trunc|date_round|date_add|date_subtract|date_diff|date_compare|date_format|date_utctolocal|date_localtoutc|date_timezone|date_timezones|fail|passthru|v8|sleep|schema_get|schema_validate|shard_id|version|noopt|noeval|not_null|first_list|first_document|parse_identifier|current_user|current_database|collection_count|pregel_result|collections|document|decode_rev|range|union|union_distinct|minus|intersection|flatten|is_same_collection|check_document|ltrim|rtrim|find_first|find_last|split|substitute|ipv4_to_number|ipv4_from_number|is_ipv4|md5|sha1|sha256|sha512|crc32|fnv64|hash|random_token|to_base64|to_hex|encode_uri_component|soundex|assert|warn|is_key|sorted|sorted_unique|count_distinct|count_unique|levenshtein_distance|levenshtein_match|regex_matches|regex_split|ngram_match|ngram_similarity|ngram_positional_similarity|uuid|tokens|exists|starts_with|phrase|min_match|bm25|tfidf|boost|analyzer|offset_info|value|cosine_similarity|decay_exp|decay_gauss|decay_linear|l1_distance|l2_distance|minhash|minhash_count|minhash_error|minhash_match|geo_point|geo_multipoint|geo_polygon|geo_multipolygon|geo_linestring|geo_multilinestring|geo_contains|geo_intersects|geo_equals|geo_distance|geo_area|geo_in_range)(?=\s*\()">
<rule pattern="(?:to_bool|to_number|to_char|to_string|to_array|to_list|is_null|is_bool|is_number|is_string|is_array|is_list|is_object|is_document|is_datestring|typename|json_stringify|json_parse|concat|concat_separator|char_length|lower|upper|substring|substring_bytes|left|right|trim|reverse|repeat|contains|log|log2|log10|exp|exp2|sin|cos|tan|asin|acos|atan|atan2|radians|degrees|pi|regex_test|regex_replace|like|floor|ceil|round|abs|rand|random|sqrt|pow|length|count|min|max|average|avg|sum|product|median|variance_population|variance_sample|variance|percentile|bit_and|bit_or|bit_xor|bit_negate|bit_test|bit_popcount|bit_shift_left|bit_shift_right|bit_construct|bit_deconstruct|bit_to_string|bit_from_string|first|last|unique|outersection|interleave|in_range|jaccard|matches|merge|merge_recursive|has|attributes|keys|values|entries|unset|unset_recursive|keep|keep_recursive|near|within|within_rectangle|is_in_polygon|distance|fulltext|stddev_sample|stddev_population|stddev|slice|nth|position|contains_array|translate|zip|call|apply|push|append|pop|shift|unshift|remove_value|remove_values|remove_nth|replace_nth|date_now|date_timestamp|date_iso8601|date_dayofweek|date_year|date_month|date_day|date_hour|date_minute|date_second|date_millisecond|date_dayofyear|date_isoweek|date_isoweekyear|date_leapyear|date_quarter|date_days_in_month|date_trunc|date_round|date_add|date_subtract|date_diff|date_compare|date_format|date_utctolocal|date_localtoutc|date_timezone|date_timezones|fail|passthru|v8|sleep|schema_get|schema_validate|shard_id|version|noopt|noeval|not_null|first_list|first_document|parse_identifier|parse_collection|parse_key|current_user|current_database|collection_count|pregel_result|collections|document|decode_rev|range|union|union_distinct|minus|intersection|flatten|is_same_collection|check_document|ltrim|rtrim|find_first|find_last|split|substitute|ipv4_to_number|ipv4_from_number|is_ipv4|md5|sha1|sha256|sha512|crc32|fnv64|hash|random_token|to_base64|to_hex|encode_uri_component|soundex|assert|warn|is_key|sorted|sorted_unique|count_distinct|count_unique|levenshtein_distance|levenshtein_match|regex_matches|regex_split|ngram_match|ngram_similarity|ngram_positional_similarity|uuid|tokens|exists|starts_with|phrase|min_match|bm25|tfidf|boost|analyzer|offset_info|value|cosine_similarity|decay_exp|decay_gauss|decay_linear|l1_distance|l2_distance|minhash|minhash_count|minhash_error|minhash_match|geo_point|geo_multipoint|geo_polygon|geo_multipolygon|geo_linestring|geo_multilinestring|geo_contains|geo_intersects|geo_equals|geo_distance|geo_area|geo_in_range)(?=\s*\()">
<token type="NameFunction"/>
</rule>
<rule pattern="&#34;">
+165
View File
@@ -0,0 +1,165 @@
<lexer>
<config>
<name>ATL</name>
<alias>atl</alias>
<filename>*.atl</filename>
<mime_type>text/x-atl</mime_type>
<dot_all>true</dot_all>
</config>
<rules>
<state name="root">
<rule pattern="(--.*?)(\n)">
<bygroups>
<token type="CommentSingle" />
<token type="TextWhitespace" />
</bygroups>
</rule>
<rule pattern="(and|distinct|endif|else|for|foreach|if|implies|in|let|not|or|self|super|then|thisModule|xor)\b">
<token type="Keyword" />
</rule>
<rule pattern="(OclUndefined|true|false|#\w+)\b">
<token type="KeywordConstant" />
</rule>
<rule pattern="(module|query|library|create|from|to|uses)\b">
<token type="KeywordNamespace" />
</rule>
<rule pattern="(do)(\s*)({)">
<bygroups>
<token type="KeywordNamespace" />
<token type="TextWhitespace" />
<token type="Punctuation" />
</bygroups>
</rule>
<rule pattern="(abstract|endpoint|entrypoint|lazy|unique)(\s+)">
<bygroups>
<token type="KeywordDeclaration" />
<token type="TextWhitespace" />
</bygroups>
</rule>
<rule pattern="(rule)(\s+)">
<bygroups>
<token type="KeywordNamespace" />
<token type="TextWhitespace" />
</bygroups>
</rule>
<rule pattern="(helper)(\s+)">
<bygroups>
<token type="KeywordNamespace" />
<token type="TextWhitespace" />
</bygroups>
</rule>
<rule pattern="(context)(\s+)">
<bygroups>
<token type="KeywordNamespace" />
<token type="TextWhitespace" />
</bygroups>
</rule>
<rule pattern="(def)(\s*)(:)(\s*)">
<bygroups>
<token type="KeywordNamespace" />
<token type="TextWhitespace" />
<token type="Punctuation" />
<token type="TextWhitespace" />
</bygroups>
</rule>
<rule pattern="(Bag|Boolean|Integer|OrderedSet|Real|Sequence|Set|String|Tuple)">
<token type="KeywordType" />
</rule>
<rule pattern="(\w+)(\s*)(&lt;-|&lt;:=)">
<bygroups>
<token type="NameNamespace" />
<token type="TextWhitespace" />
<token type="Punctuation" />
</bygroups>
</rule>
<rule pattern="#&quot;">
<token type="KeywordConstant" />
<push state="quotedenumliteral" />
</rule>
<rule pattern="&quot;">
<token type="NameNamespace" />
<push state="quotedname" />
</rule>
<rule pattern="[^\S\n]+">
<token type="TextWhitespace" />
</rule>
<rule pattern="&#x27;">
<token type="LiteralString" />
<push state="string" />
</rule>
<rule
pattern="[0-9]*\.[0-9]+">
<token type="LiteralNumberFloat" />
</rule>
<rule pattern="0|[1-9][0-9]*">
<token type="LiteralNumberInteger" />
</rule>
<rule pattern="[*&lt;&gt;+=/-]">
<token type="Operator" />
</rule>
<rule pattern="([{}();:.,!|]|-&gt;)">
<token type="Punctuation" />
</rule>
<rule pattern="\n">
<token type="TextWhitespace" />
</rule>
<rule pattern="\w+">
<token type="NameNamespace" />
</rule>
</state>
<state name="string">
<rule pattern="[^\\&#x27;]+">
<token type="LiteralString" />
</rule>
<rule pattern="\\\\">
<token type="LiteralString" />
</rule>
<rule pattern="\\&#x27;">
<token type="LiteralString" />
</rule>
<rule pattern="\\">
<token type="LiteralString" />
</rule>
<rule pattern="&#x27;">
<token type="LiteralString" />
<pop depth="1" />
</rule>
</state>
<state name="quotedname">
<rule pattern="[^\\&quot;]+">
<token type="NameNamespace" />
</rule>
<rule pattern="\\\\">
<token type="NameNamespace" />
</rule>
<rule pattern="\\&quot;">
<token type="NameNamespace" />
</rule>
<rule pattern="\\">
<token type="NameNamespace" />
</rule>
<rule pattern="&quot;">
<token type="NameNamespace" />
<pop depth="1" />
</rule>
</state>
<state name="quotedenumliteral">
<rule pattern="[^\\&quot;]+">
<token type="KeywordConstant" />
</rule>
<rule pattern="\\\\">
<token type="KeywordConstant" />
</rule>
<rule pattern="\\&quot;">
<token type="KeywordConstant" />
</rule>
<rule pattern="\\">
<token type="KeywordConstant" />
</rule>
<rule pattern="&quot;">
<token type="KeywordConstant" />
<pop depth="1" />
</rule>
</state>
</rules>
</lexer>
+120
View File
@@ -0,0 +1,120 @@
<lexer>
<config>
<name>Beef</name>
<alias>beef</alias>
<filename>*.bf</filename>
<mime_type>text/x-beef</mime_type>
<dot_all>true</dot_all>
<ensure_nl>true</ensure_nl>
</config>
<rules>
<state name="root">
<rule pattern="^\s*\[.*?\]">
<token type="NameAttribute"/>
</rule>
<rule pattern="[^\S\n]+">
<token type="Text"/>
</rule>
<rule pattern="\\\n">
<token type="Text"/>
</rule>
<rule pattern="///[^\n\r]*">
<token type="CommentSpecial"/>
</rule>
<rule pattern="//[^\n\r]*">
<token type="CommentSingle"/>
</rule>
<rule pattern="/[*].*?[*]/">
<token type="CommentMultiline"/>
</rule>
<rule pattern="\n">
<token type="Text"/>
</rule>
<rule pattern="[~!%^&amp;*()+=|\[\]:;,.&lt;&gt;/?-]">
<token type="Punctuation"/>
</rule>
<rule pattern="[{}]">
<token type="Punctuation"/>
</rule>
<rule pattern="@&#34;(&#34;&#34;|[^&#34;])*&#34;">
<token type="LiteralString"/>
</rule>
<rule pattern="\$@?&#34;(&#34;&#34;|[^&#34;])*&#34;">
<token type="LiteralString"/>
</rule>
<rule pattern="&#34;(\\\\|\\&#34;|[^&#34;\n])*[&#34;\n]">
<token type="LiteralString"/>
</rule>
<rule pattern="&#39;\\.&#39;|&#39;[^\\]&#39;">
<token type="LiteralStringChar"/>
</rule>
<rule pattern="0[xX][0-9a-fA-F]+[Ll]?|\d[_\d]*(\.\d*)?([eE][+-]?\d+)?[flFLdD]?">
<token type="LiteralNumber"/>
</rule>
<rule pattern="#[ \t]*(if|endif|else|elif|define|undef|line|error|warning|region|endregion|pragma|nullable)\b">
<token type="CommentPreproc"/>
</rule>
<rule pattern="\b(extern)(\s+)(alias)\b">
<bygroups>
<token type="Keyword"/>
<token type="Text"/>
<token type="Keyword"/>
</bygroups>
</rule>
<rule pattern="(as|await|base|break|by|case|catch|checked|continue|default|delegate|else|event|finally|fixed|for|repeat|goto|if|in|init|is|let|lock|new|scope|on|out|params|readonly|ref|return|sizeof|stackalloc|switch|this|throw|try|typeof|unchecked|virtual|void|while|get|set|new|yield|add|remove|value|alias|ascending|descending|from|group|into|orderby|select|thenby|where|join|equals)\b">
<token type="Keyword"/>
</rule>
<rule pattern="(global)(::)">
<bygroups>
<token type="Keyword"/>
<token type="Punctuation"/>
</bygroups>
</rule>
<rule pattern="(abstract|async|const|enum|explicit|extern|implicit|internal|operator|override|partial|extension|private|protected|public|static|sealed|unsafe|volatile)\b">
<token type="KeywordDeclaration"/>
</rule>
<rule pattern="(bool|byte|char8|char16|char32|decimal|double|float|int|int8|int16|int32|int64|long|object|sbyte|short|string|uint|uint8|uint16|uint32|uint64|uint|let|var)\b\??">
<token type="KeywordType"/>
</rule>
<rule pattern="(true|false|null)\b">
<token type="KeywordConstant"/>
</rule>
<rule pattern="(class|struct|record|interface)(\s+)">
<bygroups>
<token type="Keyword"/>
<token type="Text"/>
</bygroups>
<push state="class"/>
</rule>
<rule pattern="(namespace|using)(\s+)">
<bygroups>
<token type="Keyword"/>
<token type="Text"/>
</bygroups>
<push state="namespace"/>
</rule>
<rule pattern="@?[_a-zA-Z]\w*">
<token type="Name"/>
</rule>
</state>
<state name="class">
<rule pattern="@?[_a-zA-Z]\w*">
<token type="NameClass"/>
<pop depth="1"/>
</rule>
<rule>
<pop depth="1"/>
</rule>
</state>
<state name="namespace">
<rule pattern="(?=\()">
<token type="Text"/>
<pop depth="1"/>
</rule>
<rule pattern="(@?[_a-zA-Z]\w*|\.)+">
<token type="NameNamespace"/>
<pop depth="1"/>
</rule>
</state>
</rules>
</lexer>
+53
View File
@@ -0,0 +1,53 @@
<!--
Lexer for RFC-4180 compliant CSV subject to the following additions:
- UTF-8 encoding is accepted (the RFC requires 7-bit ASCII)
- The line terminator character can be LF or CRLF (the RFC allows CRLF only)
Link to the RFC-4180 specification: https://tools.ietf.org/html/rfc4180
Additions inspired by:
https://github.com/frictionlessdata/datapackage/issues/204#issuecomment-193242077
Future improvements:
- Identify non-quoted numbers as LiteralNumber
- Identify y as an error in "x"y. Currently it's identified as another string
literal.
-->
<lexer>
<config>
<name>CSV</name>
<alias>csv</alias>
<filename>*.csv</filename>
<mime_type>text/csv</mime_type>
</config>
<rules>
<state name="root">
<rule pattern="\r?\n">
<token type="Punctuation" />
</rule>
<rule pattern=",">
<token type="Punctuation" />
</rule>
<rule pattern="&quot;">
<token type="LiteralStringDouble" />
<push state="escaped" />
</rule>
<rule pattern="[^\r\n,]+">
<token type="LiteralString" />
</rule>
</state>
<state name="escaped">
<rule pattern="&quot;&quot;">
<token type="LiteralStringEscape"/>
</rule>
<rule pattern="&quot;">
<token type="LiteralStringDouble" />
<pop depth="1"/>
</rule>
<rule pattern="[^&quot;]+">
<token type="LiteralStringDouble" />
</rule>
</state>
</rules>
</lexer>
+2 -2
View File
@@ -1,7 +1,7 @@
<lexer>
<config>
<name>Gleam</name>
<alias>gleam></alias>
<alias>gleam</alias>
<filename>*.gleam</filename>
<mime_type>text/x-gleam</mime_type>
</config>
@@ -114,4 +114,4 @@
</rule>
</state>
</rules>
</lexer>
</lexer>
+1 -1
View File
@@ -84,7 +84,7 @@
<token type="Punctuation" />
</bygroups>
</rule>
<rule pattern="(str|size|rune|bool|int|uint|uintptr|u8|u16|u32|u64|i8|i16|i32|i64|f32|f64|null|void|nullable|valist|opaque|never)\b">
<rule pattern="(str|size|rune|bool|int|uint|uintptr|u8|u16|u32|u64|i8|i16|i32|i64|f32|f64|null|void|done|nullable|valist|opaque|never)\b">
<token type="KeywordType"/>
</rule>
<rule pattern="(true|false)\b">
@@ -95,19 +95,22 @@
<rule pattern="[:!#$%&amp;*+.\\/&lt;=&gt;?@^|~-]+">
<token type="Operator"/>
</rule>
<rule pattern="\d+[eE][+-]?\d+">
<rule pattern="\d+_*[eE][+-]?\d+">
<token type="LiteralNumberFloat"/>
</rule>
<rule pattern="\d+\.\d+([eE][+-]?\d+)?">
<rule pattern="\d+(_+[\d]+)*\.\d+(_+[\d]+)*([eE][+-]?\d+)?">
<token type="LiteralNumberFloat"/>
</rule>
<rule pattern="0[oO][0-7]+">
<rule pattern="0[oO](_*[0-7])+">
<token type="LiteralNumberOct"/>
</rule>
<rule pattern="0[xX][\da-fA-F]+">
<rule pattern="0[xX](_*[\da-fA-F])+">
<token type="LiteralNumberHex"/>
</rule>
<rule pattern="\d+">
<rule pattern="0[bB](_*[01])+">
<token type="LiteralNumberBin"/>
</rule>
<rule pattern="\d+(_*[\d])*">
<token type="LiteralNumberInteger"/>
</rule>
<rule pattern="&#39;">
+2 -1
View File
@@ -3,6 +3,7 @@
<name>JSON</name>
<alias>json</alias>
<filename>*.json</filename>
<filename>*.jsonc</filename>
<filename>*.avsc</filename>
<mime_type>application/json</mime_type>
<dot_all>true</dot_all>
@@ -108,4 +109,4 @@
</rule>
</state>
</rules>
</lexer>
</lexer>
@@ -0,0 +1,83 @@
<lexer>
<config>
<name>JSONata</name>
<alias>jsonata</alias>
<filename>*.jsonata</filename>
<dot_all>true</dot_all>
</config>
<rules>
<state name="root">
<rule pattern="/\*.*?\*/">
<token type="CommentMultiline"/>
</rule>
<rule pattern="[{}()\[\]:;,\.=]">
<token type="Punctuation"/>
</rule>
<rule pattern="\.\."> // Spread operator
<token type="Operator"/>
</rule>
<rule pattern="\^(?=\()"> // Sort operator
<token type="Operator"/>
</rule>
<rule pattern="\*\*|\*(?=\.)|\*"> // Descendant | Wildcard | Multiplication
<token type="Operator"/>
</rule>
<rule pattern="\/(?!\*)"> // Division
<token type="Operator"/>
</rule>
<rule pattern="[&lt;&gt;!]=?"> // Comparison operators
<token type="Operator"/>
</rule>
<rule pattern="~>">
<token type="Operator"/>
</rule>
<rule pattern="\b(and|or|in)\b">
<token type="Operator"/>
</rule>
<rule pattern="[%@#&amp;?]|\+(?!\d)|\-(?!\d)">
<token type="Operator"/>
</rule>
<rule pattern="\$[a-zA-Z0-9_]*(?![\w\(])">
<token type="NameVariable"/>
</rule>
<rule pattern="\$\w*(?=\()">
<token type="NameFunction"/>
</rule>
<rule pattern="\s+">
<token type="Text"/>
</rule>
<rule pattern="(true|false)\b">
<token type="KeywordConstant"/>
</rule>
<rule pattern="\b(function)\b">
<token type="Keyword"/>
</rule>
<rule pattern="(\+|-)?(0|[1-9]\d*)(\.\d+[eE](\+|-)?\d+|[eE](\+|-)?\d+|\.\d+)">
<token type="LiteralNumberFloat"/>
</rule>
<rule pattern="(\+|-)?(0|[1-9]\d*)">
<token type="LiteralNumberInteger"/>
</rule>
<!-- NOTE: This expression matches all object keys (NameTags), which are essentially strings with double quotes
that should only be captured on the left side of a colon (:) within a JSON-like object.
Therefore, this expression must preceed the one for all LiteralStringDouble -->
<rule pattern="&#34;(\\.|[^\\&#34;\r\n])*&#34;(?=\s*:)">
<token type="NameTag"/>
</rule>
<rule pattern="&#34;(\\\\|\\&#34;|[^&#34;])*&#34;">
<token type="LiteralStringDouble"/>
</rule>
<rule pattern="&#39;(\\|\\&#39;|[^&#39;])*&#39;">
<token type="LiteralStringSingle"/>
</rule>
<rule pattern="`.*`">
<token type="LiteralStringBacktick"/>
</rule>
<!-- NOTE: This expression matches everything remaining, which should be only JSONata names.
Therefore, it has been left as last intentionally -->
<rule pattern="[a-zA-Z0-9_]*">
<token type="Name"/>
</rule>
</state>
</rules>
</lexer>
@@ -0,0 +1,138 @@
<lexer>
<config>
<name>Jsonnet</name>
<alias>jsonnet</alias>
<filename>*.jsonnet</filename>
<filename>*.libsonnet</filename>
</config>
<rules>
<state name="_comments">
<rule pattern="(//|#).*\n"><token type="CommentSingle"/></rule>
<rule pattern="/\*\*([^/]|/(?!\*))*\*/"><token type="LiteralStringDoc"/></rule>
<rule pattern="/\*([^/]|/(?!\*))*\*/"><token type="Comment"/></rule>
</state>
<state name="root">
<rule><include state="_comments"/></rule>
<rule pattern="@&#x27;.*&#x27;"><token type="LiteralString"/></rule>
<rule pattern="@&quot;.*&quot;"><token type="LiteralString"/></rule>
<rule pattern="&#x27;"><token type="LiteralString"/><push state="singlestring"/></rule>
<rule pattern="&quot;"><token type="LiteralString"/><push state="doublestring"/></rule>
<rule pattern="\|\|\|(.|\n)*\|\|\|"><token type="LiteralString"/></rule>
<rule pattern="[+-]?[0-9]+(.[0-9])?"><token type="LiteralNumberFloat"/></rule>
<rule pattern="[!$~+\-&amp;|^=&lt;&gt;*/%]"><token type="Operator"/></rule>
<rule pattern="\{"><token type="Punctuation"/><push state="object"/></rule>
<rule pattern="\["><token type="Punctuation"/><push state="array"/></rule>
<rule pattern="local\b"><token type="Keyword"/><push state="local_name"/></rule>
<rule pattern="assert\b"><token type="Keyword"/><push state="assert"/></rule>
<rule pattern="(assert|else|error|false|for|if|import|importstr|in|null|tailstrict|then|self|super|true)\b"><token type="Keyword"/></rule>
<rule pattern="\s+"><token type="TextWhitespace"/></rule>
<rule pattern="function(?=\()"><token type="Keyword"/><push state="function_params"/></rule>
<rule pattern="std\.[^\W\d]\w*(?=\()"><token type="NameBuiltin"/><push state="function_args"/></rule>
<rule pattern="[^\W\d]\w*(?=\()"><token type="NameFunction"/><push state="function_args"/></rule>
<rule pattern="[^\W\d]\w*"><token type="NameVariable"/></rule>
<rule pattern="[\.()]"><token type="Punctuation"/></rule>
</state>
<state name="singlestring">
<rule pattern="[^&#x27;\\]"><token type="LiteralString"/></rule>
<rule pattern="\\."><token type="LiteralStringEscape"/></rule>
<rule pattern="&#x27;"><token type="LiteralString"/><pop depth="1"/></rule>
</state>
<state name="doublestring">
<rule pattern="[^&quot;\\]"><token type="LiteralString"/></rule>
<rule pattern="\\."><token type="LiteralStringEscape"/></rule>
<rule pattern="&quot;"><token type="LiteralString"/><pop depth="1"/></rule>
</state>
<state name="array">
<rule pattern=","><token type="Punctuation"/></rule>
<rule pattern="\]"><token type="Punctuation"/><pop depth="1"/></rule>
<rule><include state="root"/></rule>
</state>
<state name="local_name">
<rule pattern="[^\W\d]\w*(?=\()"><token type="NameFunction"/><push state="function_params"/></rule>
<rule pattern="[^\W\d]\w*"><token type="NameVariable"/></rule>
<rule pattern="\s+"><token type="TextWhitespace"/></rule>
<rule pattern="(?==)"><token type="TextWhitespace"/><push state="#pop" state="local_value"/></rule>
</state>
<state name="local_value">
<rule pattern="="><token type="Operator"/></rule>
<rule pattern=";"><token type="Punctuation"/><pop depth="1"/></rule>
<rule><include state="root"/></rule>
</state>
<state name="assert">
<rule pattern=":"><token type="Punctuation"/></rule>
<rule pattern=";"><token type="Punctuation"/><pop depth="1"/></rule>
<rule><include state="root"/></rule>
</state>
<state name="function_params">
<rule pattern="[^\W\d]\w*"><token type="NameVariable"/></rule>
<rule pattern="\("><token type="Punctuation"/></rule>
<rule pattern="\)"><token type="Punctuation"/><pop depth="1"/></rule>
<rule pattern=","><token type="Punctuation"/></rule>
<rule pattern="\s+"><token type="TextWhitespace"/></rule>
<rule pattern="="><token type="Operator"/><push state="function_param_default"/></rule>
</state>
<state name="function_args">
<rule pattern="\("><token type="Punctuation"/></rule>
<rule pattern="\)"><token type="Punctuation"/><pop depth="1"/></rule>
<rule pattern=","><token type="Punctuation"/></rule>
<rule pattern="\s+"><token type="TextWhitespace"/></rule>
<rule><include state="root"/></rule>
</state>
<state name="object">
<rule pattern="\s+"><token type="TextWhitespace"/></rule>
<rule pattern="local\b"><token type="Keyword"/><push state="object_local_name"/></rule>
<rule pattern="assert\b"><token type="Keyword"/><push state="object_assert"/></rule>
<rule pattern="\["><token type="Operator"/><push state="field_name_expr"/></rule>
<rule pattern="(?=[^\W\d]\w*)"><token type="Text"/><push state="field_name"/></rule>
<rule pattern="\}"><token type="Punctuation"/><pop depth="1"/></rule>
<rule pattern="&quot;"><token type="NameVariable"/><push state="double_field_name"/></rule>
<rule pattern="&#x27;"><token type="NameVariable"/><push state="single_field_name"/></rule>
<rule><include state="_comments"/></rule>
</state>
<state name="field_name">
<rule pattern="[^\W\d]\w*(?=\()"><token type="NameFunction"/><push state="field_separator" state="function_params"/></rule>
<rule pattern="[^\W\d]\w*"><token type="NameVariable"/><push state="field_separator"/></rule>
</state>
<state name="double_field_name">
<rule pattern="([^&quot;\\]|\\.)*&quot;"><token type="NameVariable"/><push state="field_separator"/></rule>
</state>
<state name="single_field_name">
<rule pattern="([^&#x27;\\]|\\.)*&#x27;"><token type="NameVariable"/><push state="field_separator"/></rule>
</state>
<state name="field_name_expr">
<rule pattern="\]"><token type="Operator"/><push state="field_separator"/></rule>
<rule><include state="root"/></rule>
</state>
<state name="function_param_default">
<rule pattern="(?=[,\)])"><token type="TextWhitespace"/><pop depth="1"/></rule>
<rule><include state="root"/></rule>
</state>
<state name="field_separator">
<rule pattern="\s+"><token type="TextWhitespace"/></rule>
<rule pattern="\+?::?:?"><token type="Punctuation"/><push state="#pop" state="#pop" state="field_value"/></rule>
<rule><include state="_comments"/></rule>
</state>
<state name="field_value">
<rule pattern=","><token type="Punctuation"/><pop depth="1"/></rule>
<rule pattern="\}"><token type="Punctuation"/><pop depth="2"/></rule>
<rule><include state="root"/></rule>
</state>
<state name="object_assert">
<rule pattern=":"><token type="Punctuation"/></rule>
<rule pattern=","><token type="Punctuation"/><pop depth="1"/></rule>
<rule><include state="root"/></rule>
</state>
<state name="object_local_name">
<rule pattern="[^\W\d]\w*"><token type="NameVariable"/><push state="#pop" state="object_local_value"/></rule>
<rule pattern="\s+"><token type="TextWhitespace"/></rule>
</state>
<state name="object_local_value">
<rule pattern="="><token type="Operator"/></rule>
<rule pattern=","><token type="Punctuation"/><pop depth="1"/></rule>
<rule pattern="\}"><token type="Punctuation"/><pop depth="2"/></rule>
<rule><include state="root"/></rule>
</state>
</rules>
</lexer>
@@ -1,154 +1,154 @@
<lexer>
<config>
<name>Materialize SQL dialect</name>
<alias>materialize</alias>
<alias>mzsql</alias>
<mime_type>text/x-materializesql</mime_type>
<case_insensitive>true</case_insensitive>
<not_multiline>true</not_multiline>
<alias>materialize</alias>
<alias>mzsql</alias>
</config>
<rules>
<state name="root">
<rule pattern="\s+">
<token type="Text"/>
<token type="Text" />
</rule>
<rule pattern="--.*\n?">
<token type="CommentSingle"/>
<token type="CommentSingle" />
</rule>
<rule pattern="/\*">
<token type="CommentMultiline"/>
<push state="multiline-comments"/>
<token type="CommentMultiline" />
<push state="multiline-comments" />
</rule>
<rule pattern="(bigint|bigserial|bit|bit\s+varying|bool|boolean|box|bytea|char|character|character\s+varying|cidr|circle|date|decimal|double\s+precision|float4|float8|inet|int|int2|int4|int8|integer|interval|json|jsonb|line|lseg|macaddr|money|numeric|path|pg_lsn|point|polygon|real|serial|serial2|serial4|serial8|smallint|smallserial|text|time|timestamp|timestamptz|timetz|tsquery|tsvector|txid_snapshot|uuid|varbit|varchar|with\s+time\s+zone|without\s+time\s+zone|xml|anyarray|anyelement|anyenum|anynonarray|anyrange|cstring|fdw_handler|internal|language_handler|opaque|record|void)\b">
<token type="NameBuiltin"/>
<token type="NameBuiltin" />
</rule>
<rule pattern="(?s)(DO)(\s+)(?:(LANGUAGE)?(\s+)(&#39;?)(\w+)?(&#39;?)(\s+))?(\$)([^$]*)(\$)(.*?)(\$)(\10)(\$)">
<rule pattern="(?s)(DO)(\s+)(?:(LANGUAGE)?(\s+)('?)(\w+)?('?)(\s+))?(\$)([^$]*)(\$)(.*?)(\$)(\10)(\$)">
<usingbygroup>
<sublexer_name_group>6</sublexer_name_group>
<code_group>12</code_group>
<emitters>
<token type="Keyword"/>
<token type="Text"/>
<token type="Keyword"/>
<token type="Text"/>
<token type="LiteralStringSingle"/>
<token type="LiteralStringSingle"/>
<token type="LiteralStringSingle"/>
<token type="Text"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="Keyword" />
<token type="Text" />
<token type="Keyword" />
<token type="Text" />
<token type="LiteralStringSingle" />
<token type="LiteralStringSingle" />
<token type="LiteralStringSingle" />
<token type="Text" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
</emitters>
</usingbygroup>
</rule>
<rule pattern="(ACCESS|ACKS|ADD|ADDRESSES|AGGREGATE|ALL|ALTER|AND|ANY|ARN|ARRANGEMENT|ARRAY|AS|ASC|ASSERT|AT|AUCTION|AUTHORITY|AVAILABILITY|AVRO|AWS|BEGIN|BETWEEN|BIGINT|BILLED|BODY|BOOLEAN|BOTH|BPCHAR|BROKEN|BROKER|BROKERS|BY|BYTES|CARDINALITY|CASCADE|CASE|CAST|CERTIFICATE|CHAIN|CHAR|CHARACTER|CHARACTERISTICS|CHECK|CLIENT|CLOSE|CLUSTER|CLUSTERS|COALESCE|COLLATE|COLUMN|COLUMNS|COMMENT|COMMIT|COMMITTED|COMPACTION|COMPRESSION|COMPUTE|COMPUTECTL|CONFLUENT|CONNECTION|CONNECTIONS|CONSTRAINT|COPY|COUNT|COUNTER|CREATE|CREATECLUSTER|CREATEDB|CREATEROLE|CROSS|CSV|CURRENT|CURSOR|DATABASE|DATABASES|DATUMS|DAY|DAYS|DEALLOCATE|DEBEZIUM|DEBUG|DEBUGGING|DEC|DECIMAL|DECLARE|DECORRELATED|DEFAULT|DEFAULTS|DELETE|DELIMITED|DELIMITER|DESC|DETAILS|DISCARD|DISK|DISTINCT|DOC|DOT|DOUBLE|DROP|EFFORT|ELEMENT|ELSE|ENABLE|END|ENDPOINT|ENFORCED|ENVELOPE|ERROR|ESCAPE|EXCEPT|EXECUTE|EXISTS|EXPECTED|EXPLAIN|EXPOSE|EXTRACT|FACTOR|FALSE|FETCH|FIELDS|FILTER|FIRST|FLOAT|FOLLOWING|FOR|FOREIGN|FORMAT|FORWARD|FROM|FULL|FULLNAME|FUNCTION|GENERATOR|GRANT|GREATEST|GROUP|GROUPS|HAVING|HEADER|HEADERS|HOLD|HOST|HOUR|HOURS|ID|IDEMPOTENCE|IDLE|IF|IGNORE|ILIKE|IN|INCLUDE|INDEX|INDEXES|INFO|INHERIT|INLINE|INNER|INPUT|INSERT|INSPECT|INT|INTEGER|INTERNAL|INTERSECT|INTERVAL|INTO|INTROSPECTION|IS|ISNULL|ISOLATION|JOIN|JSON|KAFKA|KEY|KEYS|LAST|LATERAL|LATEST|LEADING|LEAST|LEFT|LEVEL|LIKE|LIMIT|LIST|LOAD|LOCAL|LOG|LOGICAL|LOGIN|MANAGED|MAP|MARKETING|MATERIALIZE|MATERIALIZED|MAX|MECHANISMS|MEMBERSHIP|MERGE|MESSAGE|METADATA|MINUTE|MINUTES|MODE|MONTH|MONTHS|MS|MUTUALLY|NAME|NAMES|NATURAL|NEXT|NO|NOCREATECLUSTER|NOCREATEDB|NOCREATEROLE|NOINHERIT|NOLOGIN|NONE|NOSUPERUSER|NOT|NOTICE|NULL|NULLIF|NULLS|OBJECTS|OF|OFFSET|ON|ONLY|OPERATOR|OPTIMIZED|OPTIMIZER|OPTIONS|OR|ORDER|ORDINALITY|OUTER|OVER|OWNED|OWNER|PARTITION|PASSWORD|PHYSICAL|PLAN|PLANS|PORT|POSITION|POSTGRES|PRECEDING|PRECISION|PREFIX|PREPARE|PRIMARY|PRIVATELINK|PRIVILEGES|PROGRESS|PROTOBUF|PROTOCOL|PUBLICATION|QUERY|QUOTE|RAISE|RANGE|RAW|READ|REAL|REASSIGN|RECURSION|RECURSIVE|REFERENCES|REFRESH|REGEX|REGION|REGISTRY|RENAME|REPEATABLE|REPLACE|REPLICA|REPLICAS|REPLICATION|RESET|RESPECT|RESTRICT|RETENTION|RETURN|RETURNING|REVOKE|RIGHT|ROLE|ROLES|ROLLBACK|ROTATE|ROW|ROWS|SASL|SCALE|SCHEMA|SCHEMAS|SCRIPT|SECOND|SECONDS|SECRET|SECRETS|SECURITY|SEED|SELECT|SEQUENCES|SERIALIZABLE|SERVICE|SESSION|SET|SHARD|SHOW|SINK|SINKS|SIZE|SMALLINT|SNAPSHOT|SOME|SOURCE|SOURCES|SSH|SSL|START|STDIN|STDOUT|STORAGE|STORAGECTL|STRATEGY|STRICT|STRING|SUBSCRIBE|SUBSOURCE|SUBSOURCES|SUBSTRING|SUPERUSER|SWAP|SYSTEM|TABLE|TABLES|TAIL|TEMP|TEMPORARY|TEST|TEXT|THEN|TICK|TIES|TIME|TIMELINE|TIMEOUT|TIMESTAMP|TIMESTAMPTZ|TO|TOKEN|TOPIC|TPCH|TRACE|TRAILING|TRANSACTION|TRIM|TRUE|TUNNEL|TYPE|TYPES|UNBOUNDED|UNCOMMITTED|UNION|UNIQUE|UNKNOWN|UP|UPDATE|UPSERT|URL|USAGE|USER|USERNAME|USERS|USING|VALIDATE|VALUE|VALUES|VARCHAR|VARYING|VIEW|VIEWS|WARNING|WEBHOOK|WHEN|WHERE|WINDOW|WIRE|WITH|WITHIN|WITHOUT|WORK|WORKERS|WRITE|YEAR|YEARS|ZONE|ZONES)\b">
<token type="Keyword"/>
<rule pattern="(ACCESS|ADD|ADDRESSES|AGGREGATE|ALIGNED|ALL|ALTER|ANALYSIS|AND|ANY|ARITY|ARN|ARRANGEMENT|ARRAY|AS|ASC|ASSERT|ASSUME|AT|AUCTION|AUTHORITY|AVAILABILITY|AVRO|AWS|BATCH|BEGIN|BETWEEN|BIGINT|BILLED|BODY|BOOLEAN|BOTH|BPCHAR|BROKEN|BROKER|BROKERS|BY|BYTES|CARDINALITY|CASCADE|CASE|CAST|CERTIFICATE|CHAIN|CHAINS|CHAR|CHARACTER|CHARACTERISTICS|CHECK|CLASS|CLIENT|CLOCK|CLOSE|CLUSTER|CLUSTERS|COALESCE|COLLATE|COLUMN|COLUMNS|COMMENT|COMMIT|COMMITTED|COMPACTION|COMPATIBILITY|COMPRESSION|COMPUTE|COMPUTECTL|CONFIG|CONFLUENT|CONNECTION|CONNECTIONS|CONSTRAINT|CONTINUAL|COPY|COUNT|COUNTER|CREATE|CREATECLUSTER|CREATEDB|CREATEROLE|CREATION|CROSS|CSV|CURRENT|CURSOR|DATABASE|DATABASES|DATUMS|DAY|DAYS|DEALLOCATE|DEBEZIUM|DEBUG|DEBUGGING|DEC|DECIMAL|DECLARE|DECODING|DECORRELATED|DEFAULT|DEFAULTS|DELETE|DELIMITED|DELIMITER|DELTA|DESC|DETAILS|DISCARD|DISK|DISTINCT|DOC|DOT|DOUBLE|DROP|EAGER|ELEMENT|ELSE|ENABLE|END|ENDPOINT|ENFORCED|ENVELOPE|ERROR|ERRORS|ESCAPE|ESTIMATE|EVERY|EXCEPT|EXCLUDE|EXECUTE|EXISTS|EXPECTED|EXPLAIN|EXPOSE|EXPRESSIONS|EXTERNAL|EXTRACT|FACTOR|FALSE|FAST|FEATURES|FETCH|FIELDS|FILE|FILTER|FIRST|FIXPOINT|FLOAT|FOLLOWING|FOR|FOREIGN|FORMAT|FORWARD|FROM|FULL|FULLNAME|FUNCTION|FUSION|GENERATOR|GRANT|GREATEST|GROUP|GROUPS|HAVING|HEADER|HEADERS|HISTORY|HOLD|HOST|HOUR|HOURS|HUMANIZED|HYDRATION|ID|IDENTIFIERS|IDS|IF|IGNORE|ILIKE|IMPLEMENTATIONS|IMPORTED|IN|INCLUDE|INDEX|INDEXES|INFO|INHERIT|INLINE|INNER|INPUT|INSERT|INSIGHTS|INSPECT|INT|INTEGER|INTERNAL|INTERSECT|INTERVAL|INTO|INTROSPECTION|IS|ISNULL|ISOLATION|JOIN|JOINS|JSON|KAFKA|KEY|KEYS|LAST|LATERAL|LATEST|LEADING|LEAST|LEFT|LEGACY|LETREC|LEVEL|LIKE|LIMIT|LINEAR|LIST|LOAD|LOCAL|LOCALLY|LOG|LOGICAL|LOGIN|LOWERING|MANAGED|MANUAL|MAP|MARKETING|MATERIALIZE|MATERIALIZED|MAX|MECHANISMS|MEMBERSHIP|MESSAGE|METADATA|MINUTE|MINUTES|MODE|MONTH|MONTHS|MUTUALLY|MYSQL|NAME|NAMES|NATURAL|NEGATIVE|NEW|NEXT|NO|NOCREATECLUSTER|NOCREATEDB|NOCREATEROLE|NODE|NOINHERIT|NOLOGIN|NON|NONE|NOSUPERUSER|NOT|NOTICE|NOTICES|NULL|NULLIF|NULLS|OBJECTS|OF|OFFSET|ON|ONLY|OPERATOR|OPTIMIZED|OPTIMIZER|OPTIONS|OR|ORDER|ORDINALITY|OUTER|OVER|OWNED|OWNER|PARTITION|PARTITIONS|PASSWORD|PATH|PHYSICAL|PLAN|PLANS|PORT|POSITION|POSTGRES|PRECEDING|PRECISION|PREFIX|PREPARE|PRIMARY|PRIVATELINK|PRIVILEGES|PROGRESS|PROTOBUF|PROTOCOL|PUBLIC|PUBLICATION|PUSHDOWN|QUERY|QUOTE|RAISE|RANGE|RATE|RAW|READ|READY|REAL|REASSIGN|RECURSION|RECURSIVE|REDACTED|REDUCE|REFERENCE|REFERENCES|REFRESH|REGEX|REGION|REGISTRY|RENAME|REOPTIMIZE|REPEATABLE|REPLACE|REPLAN|REPLICA|REPLICAS|REPLICATION|RESET|RESPECT|RESTRICT|RETAIN|RETURN|RETURNING|REVOKE|RIGHT|ROLE|ROLES|ROLLBACK|ROTATE|ROUNDS|ROW|ROWS|SASL|SCALE|SCHEDULE|SCHEMA|SCHEMAS|SECOND|SECONDS|SECRET|SECRETS|SECURITY|SEED|SELECT|SEQUENCES|SERIALIZABLE|SERVICE|SESSION|SET|SHARD|SHOW|SINK|SINKS|SIZE|SMALLINT|SNAPSHOT|SOME|SOURCE|SOURCES|SSH|SSL|START|STDIN|STDOUT|STORAGE|STORAGECTL|STRATEGY|STRICT|STRING|STRONG|SUBSCRIBE|SUBSOURCE|SUBSOURCES|SUBSTRING|SUBTREE|SUPERUSER|SWAP|SYNTAX|SYSTEM|TABLE|TABLES|TAIL|TASK|TEMP|TEMPORARY|TEXT|THEN|TICK|TIES|TIME|TIMELINE|TIMEOUT|TIMESTAMP|TIMESTAMPTZ|TIMING|TO|TOKEN|TOPIC|TPCH|TRACE|TRAILING|TRANSACTION|TRANSACTIONAL|TRIM|TRUE|TUNNEL|TYPE|TYPES|UNBOUNDED|UNCOMMITTED|UNION|UNIQUE|UNKNOWN|UNNEST|UNTIL|UP|UPDATE|UPSERT|URL|USAGE|USER|USERNAME|USERS|USING|VALIDATE|VALUE|VALUES|VARCHAR|VARIADIC|VARYING|VERSION|VIEW|VIEWS|WAIT|WARNING|WEBHOOK|WHEN|WHERE|WINDOW|WIRE|WITH|WITHIN|WITHOUT|WORK|WORKERS|WORKLOAD|WRITE|YEAR|YEARS|YUGABYTE|ZONE|ZONES)\b">
<token type="Keyword" />
</rule>
<rule pattern="[+*/&lt;&gt;=~!@#%^&amp;|`?-]+">
<token type="Operator"/>
<token type="Operator" />
</rule>
<rule pattern="::">
<token type="Operator"/>
<token type="Operator" />
</rule>
<rule pattern="\$\d+">
<token type="NameVariable"/>
<token type="NameVariable" />
</rule>
<rule pattern="([0-9]*\.[0-9]*|[0-9]+)(e[+-]?[0-9]+)?">
<token type="LiteralNumberFloat"/>
<token type="LiteralNumberFloat" />
</rule>
<rule pattern="[0-9]+">
<token type="LiteralNumberInteger"/>
<token type="LiteralNumberInteger" />
</rule>
<rule pattern="((?:E|U&amp;)?)(&#39;)">
<rule pattern="((?:E|U&amp;)?)(')">
<bygroups>
<token type="LiteralStringAffix"/>
<token type="LiteralStringSingle"/>
<token type="LiteralStringAffix" />
<token type="LiteralStringSingle" />
</bygroups>
<push state="string"/>
<push state="string" />
</rule>
<rule pattern="((?:U&amp;)?)(&#34;)">
<rule pattern="((?:U&amp;)?)(&quot;)">
<bygroups>
<token type="LiteralStringAffix"/>
<token type="LiteralStringName"/>
<token type="LiteralStringAffix" />
<token type="LiteralStringName" />
</bygroups>
<push state="quoted-ident"/>
<push state="quoted-ident" />
</rule>
<rule pattern="(?s)(\$)([^$]*)(\$)(.*?)(\$)(\2)(\$)(\s+)(LANGUAGE)?(\s+)(&#39;?)(\w+)?(&#39;?)">
<rule pattern="(?s)(\$)([^$]*)(\$)(.*?)(\$)(\2)(\$)(\s+)(LANGUAGE)?(\s+)('?)(\w+)?('?)">
<usingbygroup>
<sublexer_name_group>12</sublexer_name_group>
<code_group>4</code_group>
<emitters>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc"/>
<token type="Text"/>
<token type="Keyword"/>
<token type="Text"/>
<token type="LiteralStringSingle"/>
<token type="LiteralStringSingle"/>
<token type="LiteralStringSingle"/>
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="LiteralStringHeredoc" />
<token type="Text" />
<token type="Keyword" />
<token type="Text" />
<token type="LiteralStringSingle" />
<token type="LiteralStringSingle" />
<token type="LiteralStringSingle" />
</emitters>
</usingbygroup>
</rule>
<rule pattern="(?s)(\$)([^$]*)(\$)(.*?)(\$)(\2)(\$)">
<token type="LiteralStringHeredoc"/>
<token type="LiteralStringHeredoc" />
</rule>
<rule pattern="[a-z_]\w*">
<token type="Name"/>
<token type="Name" />
</rule>
<rule pattern=":([&#39;&#34;]?)[a-z]\w*\b\1">
<token type="NameVariable"/>
<rule pattern=":(['&quot;]?)[a-z]\w*\b\1">
<token type="NameVariable" />
</rule>
<rule pattern="[;:()\[\]{},.]">
<token type="Punctuation"/>
<token type="Punctuation" />
</rule>
</state>
<state name="multiline-comments">
<rule pattern="/\*">
<token type="CommentMultiline"/>
<push state="multiline-comments"/>
<token type="CommentMultiline" />
<push state="multiline-comments" />
</rule>
<rule pattern="\*/">
<token type="CommentMultiline"/>
<pop depth="1"/>
<token type="CommentMultiline" />
<pop depth="1" />
</rule>
<rule pattern="[^/*]+">
<token type="CommentMultiline"/>
<token type="CommentMultiline" />
</rule>
<rule pattern="[/*]">
<token type="CommentMultiline"/>
<token type="CommentMultiline" />
</rule>
</state>
<state name="string">
<rule pattern="[^&#39;]+">
<token type="LiteralStringSingle"/>
<rule pattern="[^']+">
<token type="LiteralStringSingle" />
</rule>
<rule pattern="&#39;&#39;">
<token type="LiteralStringSingle"/>
<rule pattern="''">
<token type="LiteralStringSingle" />
</rule>
<rule pattern="&#39;">
<token type="LiteralStringSingle"/>
<pop depth="1"/>
<rule pattern="'">
<token type="LiteralStringSingle" />
<pop depth="1" />
</rule>
</state>
<state name="quoted-ident">
<rule pattern="[^&#34;]+">
<token type="LiteralStringName"/>
<rule pattern="[^&quot;]+">
<token type="LiteralStringName" />
</rule>
<rule pattern="&#34;&#34;">
<token type="LiteralStringName"/>
<rule pattern="&quot;&quot;">
<token type="LiteralStringName" />
</rule>
<rule pattern="&#34;">
<token type="LiteralStringName"/>
<pop depth="1"/>
<rule pattern="&quot;">
<token type="LiteralStringName" />
<pop depth="1" />
</rule>
</state>
</rules>
+126 -170
View File
@@ -1,182 +1,138 @@
<lexer>
<config>
<name>mcfunction</name>
<name>MCFunction</name>
<alias>mcfunction</alias>
<alias>mcf</alias>
<filename>*.mcfunction</filename>
<dot_all>true</dot_all>
<not_multiline>true</not_multiline>
<mime_type>text/mcfunction</mime_type>
</config>
<rules>
<state name="nbtobjectvalue">
<rule pattern="(&#34;(\\\\|\\&#34;|[^&#34;])*&#34;|[a-zA-Z0-9_]+)">
<token type="NameTag"/>
<push state="nbtobjectattribute"/>
</rule>
<rule pattern="\}">
<token type="Punctuation"/>
<pop depth="1"/>
</rule>
</state>
<state name="nbtarrayvalue">
<rule>
<include state="nbtvalue"/>
</rule>
<rule pattern=",">
<token type="Punctuation"/>
</rule>
<rule pattern="\]">
<token type="Punctuation"/>
<pop depth="1"/>
</rule>
</state>
<state name="nbtvalue">
<rule>
<include state="simplevalue"/>
</rule>
<rule pattern="\{">
<token type="Punctuation"/>
<push state="nbtobjectvalue"/>
</rule>
<rule pattern="\[">
<token type="Punctuation"/>
<push state="nbtarrayvalue"/>
</rule>
</state>
<state name="argumentvalue">
<rule>
<include state="simplevalue"/>
</rule>
<rule pattern=",">
<token type="Punctuation"/>
<pop depth="1"/>
</rule>
<rule pattern="[}\]]">
<token type="Punctuation"/>
<pop depth="2"/>
</rule>
</state>
<state name="argumentlist">
<rule pattern="(nbt)(={)">
<bygroups>
<token type="NameAttribute"/>
<token type="Punctuation"/>
</bygroups>
<push state="nbtobjectvalue"/>
</rule>
<rule pattern="([A-Za-z0-9/_!]+)(={)">
<bygroups>
<token type="NameAttribute"/>
<token type="Punctuation"/>
</bygroups>
<push state="argumentlist"/>
</rule>
<rule pattern="([A-Za-z0-9/_!]+)(=)">
<bygroups>
<token type="NameAttribute"/>
<token type="Punctuation"/>
</bygroups>
<push state="argumentvalue"/>
</rule>
<rule>
<include state="simplevalue"/>
</rule>
<rule pattern=",">
<token type="Punctuation"/>
</rule>
<rule pattern="[}\]]">
<token type="Punctuation"/>
<pop depth="1"/>
</rule>
</state>
<state name="root">
<rule pattern="#.*?\n">
<token type="CommentSingle"/>
</rule>
<rule pattern="/?(geteduclientinfo|clearspawnpoint|defaultgamemode|transferserver|toggledownfall|immutableworld|detectredstone|setidletimeout|playanimation|classroommode|spreadplayers|testforblocks|setmaxplayers|setworldspawn|testforblock|worldbuilder|createagent|worldborder|camerashake|advancement|raytracefog|locatebiome|tickingarea|replaceitem|attributes|spawnpoint|difficulty|experience|scoreboard|whitelist|structure|playsound|stopsound|forceload|spectate|gamerule|function|schedule|wsserver|teleport|position|save-off|particle|setblock|datapack|mobevent|transfer|gamemode|save-all|bossbar|enchant|trigger|collect|execute|weather|teammsg|tpagent|banlist|dropall|publish|tellraw|testfor|save-on|destroy|ability|locate|summon|remove|effect|reload|ban-ip|recipe|pardon|detect|music|clear|clone|event|mixer|debug|title|ride|stop|list|turn|data|team|kick|loot|tell|help|give|flog|fill|move|time|seed|kill|save|item|deop|code|tag|ban|msg|say|tp|me|op|xp|w|place)\b">
<token type="KeywordReserved"/>
</rule>
<rule pattern="(@p|@r|@a|@e|@s|@c|@v)">
<token type="KeywordConstant"/>
</rule>
<rule pattern="\[">
<token type="Punctuation"/>
<push state="argumentlist"/>
</rule>
<rule pattern="{">
<token type="Punctuation"/>
<push state="nbtobjectvalue"/>
</rule>
<rule pattern="~">
<token type="NameBuiltin"/>
</rule>
<rule pattern="([a-zA-Z_]+:)?[a-zA-Z_]+\b">
<token type="Text"/>
</rule>
<rule pattern="([a-z]+)(\.)([0-9]+)\b">
<bygroups>
<token type="Text"/>
<token type="Punctuation"/>
<token type="LiteralNumber"/>
</bygroups>
</rule>
<rule pattern="([&lt;&gt;=]|&lt;=|&gt;=)">
<token type="Punctuation"/>
</rule>
<rule>
<include state="simplevalue"/>
</rule>
<rule pattern="\s+">
<token type="TextWhitespace"/>
</rule>
<rule><include state="names"/></rule>
<rule><include state="comments"/></rule>
<rule><include state="literals"/></rule>
<rule><include state="whitespace"/></rule>
<rule><include state="property"/></rule>
<rule><include state="operators"/></rule>
<rule><include state="selectors"/></rule>
</state>
<state name="simplevalue">
<rule pattern="(true|false)">
<token type="KeywordConstant"/>
</rule>
<rule pattern="[01]b">
<token type="LiteralNumber"/>
</rule>
<rule pattern="-?(0|[1-9]\d*)(\.\d+[eE](\+|-)?\d+|[eE](\+|-)?\d+|\.\d+)">
<token type="LiteralNumberFloat"/>
</rule>
<rule pattern="(-?\d+)(\.\.)(-?\d+)">
<bygroups>
<token type="LiteralNumberInteger"/>
<token type="Punctuation"/>
<token type="LiteralNumberInteger"/>
</bygroups>
</rule>
<rule pattern="-?(0|[1-9]\d*)">
<token type="LiteralNumberInteger"/>
</rule>
<rule pattern="&#34;(\\\\|\\&#34;|[^&#34;])*&#34;">
<token type="LiteralStringDouble"/>
</rule>
<rule pattern="&#39;[^&#39;]+&#39;">
<token type="LiteralStringSingle"/>
</rule>
<rule pattern="([!#]?)(\w+)">
<bygroups>
<token type="Punctuation"/>
<token type="Text"/>
</bygroups>
</rule>
<state name="names">
<rule pattern="^(\s*)([a-z_]+)"><bygroups><token type="TextWhitespace"/><token type="NameBuiltin"/></bygroups></rule>
<rule pattern="(?&lt;=run)\s+[a-z_]+"><token type="NameBuiltin"/></rule>
<rule pattern="\b[0-9a-fA-F]+(?:-[0-9a-fA-F]+){4}\b"><token type="NameVariable"/></rule>
<rule><include state="resource-name"/></rule>
<rule pattern="[A-Za-z_][\w.#%$]+"><token type="KeywordConstant"/></rule>
<rule pattern="[#%$][\w.#%$]+"><token type="NameVariableMagic"/></rule>
</state>
<state name="nbtobjectattribute">
<rule>
<include state="nbtvalue"/>
</rule>
<rule pattern=":">
<token type="Punctuation"/>
</rule>
<rule pattern=",">
<token type="Punctuation"/>
<pop depth="1"/>
</rule>
<rule pattern="\}">
<token type="Punctuation"/>
<pop depth="2"/>
</rule>
<state name="resource-name">
<rule pattern="#?[a-z_][a-z_.-]*:[a-z0-9_./-]+"><token type="NameFunction"/></rule>
<rule pattern="#?[a-z0-9_\.\-]+\/[a-z0-9_\.\-\/]+"><token type="NameFunction"/></rule>
</state>
<state name="whitespace">
<rule pattern="\s+"><token type="TextWhitespace"/></rule>
</state>
<state name="comments">
<rule pattern="^\s*(#[&gt;!])"><token type="CommentMultiline"/><push state="comments.block" state="comments.block.emphasized"/></rule>
<rule pattern="#.*$"><token type="CommentSingle"/></rule>
</state>
<state name="comments.block">
<rule pattern="^\s*#[&gt;!]"><token type="CommentMultiline"/><push state="comments.block.emphasized"/></rule>
<rule pattern="^\s*#"><token type="CommentMultiline"/><push state="comments.block.normal"/></rule>
<rule><pop depth="1"/></rule>
</state>
<state name="comments.block.normal">
<rule><include state="comments.block.special"/></rule>
<rule pattern="\S+"><token type="CommentMultiline"/></rule>
<rule pattern="\n"><token type="Text"/><pop depth="1"/></rule>
<rule><include state="whitespace"/></rule>
</state>
<state name="comments.block.emphasized">
<rule><include state="comments.block.special"/></rule>
<rule pattern="\S+"><token type="LiteralStringDoc"/></rule>
<rule pattern="\n"><token type="Text"/><pop depth="1"/></rule>
<rule><include state="whitespace"/></rule>
</state>
<state name="comments.block.special">
<rule pattern="@\S+"><token type="NameDecorator"/></rule>
<rule><include state="resource-name"/></rule>
<rule pattern="[#%$][\w.#%$]+"><token type="NameVariableMagic"/></rule>
</state>
<state name="operators">
<rule pattern="[\-~%^?!+*&lt;&gt;\\/|&amp;=.]"><token type="Operator"/></rule>
</state>
<state name="literals">
<rule pattern="\.\."><token type="Literal"/></rule>
<rule pattern="(true|false)"><token type="KeywordPseudo"/></rule>
<rule pattern="[A-Za-z_]+"><token type="NameVariableClass"/></rule>
<rule pattern="[0-7]b"><token type="LiteralNumberByte"/></rule>
<rule pattern="[+-]?\d*\.?\d+([eE]?[+-]?\d+)?[df]?\b"><token type="LiteralNumberFloat"/></rule>
<rule pattern="[+-]?\d+\b"><token type="LiteralNumberInteger"/></rule>
<rule pattern="&quot;"><token type="LiteralStringDouble"/><push state="literals.string-double"/></rule>
<rule pattern="&#x27;"><token type="LiteralStringSingle"/><push state="literals.string-single"/></rule>
</state>
<state name="literals.string-double">
<rule pattern="\\."><token type="LiteralStringEscape"/></rule>
<rule pattern="[^\\&quot;\n]+"><token type="LiteralStringDouble"/></rule>
<rule pattern="&quot;"><token type="LiteralStringDouble"/><pop depth="1"/></rule>
</state>
<state name="literals.string-single">
<rule pattern="\\."><token type="LiteralStringEscape"/></rule>
<rule pattern="[^\\&#x27;\n]+"><token type="LiteralStringSingle"/></rule>
<rule pattern="&#x27;"><token type="LiteralStringSingle"/><pop depth="1"/></rule>
</state>
<state name="selectors">
<rule pattern="@[a-z]"><token type="NameVariable"/></rule>
</state>
<state name="property">
<rule pattern="\{"><token type="Punctuation"/><push state="property.curly" state="property.key"/></rule>
<rule pattern="\["><token type="Punctuation"/><push state="property.square" state="property.key"/></rule>
</state>
<state name="property.curly">
<rule><include state="whitespace"/></rule>
<rule><include state="property"/></rule>
<rule pattern="\}"><token type="Punctuation"/><pop depth="1"/></rule>
</state>
<state name="property.square">
<rule><include state="whitespace"/></rule>
<rule><include state="property"/></rule>
<rule pattern="\]"><token type="Punctuation"/><pop depth="1"/></rule>
<rule pattern=","><token type="Punctuation"/></rule>
</state>
<state name="property.key">
<rule><include state="whitespace"/></rule>
<rule pattern="#?[a-z_][a-z_\.\-]*\:[a-z0-9_\.\-/]+(?=\s*\=)"><token type="NameAttribute"/><push state="property.delimiter"/></rule>
<rule pattern="#?[a-z_][a-z0-9_\.\-/]+"><token type="NameAttribute"/><push state="property.delimiter"/></rule>
<rule pattern="[A-Za-z_\-\+]+"><token type="NameAttribute"/><push state="property.delimiter"/></rule>
<rule pattern="&quot;"><token type="NameAttribute"/><push state="property.delimiter"/></rule>
<rule pattern="&#x27;"><token type="NameAttribute"/><push state="property.delimiter"/></rule>
<rule pattern="-?\d+"><token type="LiteralNumberInteger"/><push state="property.delimiter"/></rule>
<rule><pop depth="1"/></rule>
</state>
<state name="property.key.string-double">
<rule pattern="\\."><token type="LiteralStringEscape"/></rule>
<rule pattern="[^\\&quot;\n]+"><token type="NameAttribute"/></rule>
<rule pattern="&quot;"><token type="NameAttribute"/><pop depth="1"/></rule>
</state>
<state name="property.key.string-single">
<rule pattern="\\."><token type="LiteralStringEscape"/></rule>
<rule pattern="[^\\&#x27;\n]+"><token type="NameAttribute"/></rule>
<rule pattern="&#x27;"><token type="NameAttribute"/><pop depth="1"/></rule>
</state>
<state name="property.delimiter">
<rule><include state="whitespace"/></rule>
<rule pattern="[:=]!?"><token type="Punctuation"/><push state="property.value"/></rule>
<rule pattern=","><token type="Punctuation"/></rule>
<rule><pop depth="1"/></rule>
</state>
<state name="property.value">
<rule><include state="whitespace"/></rule>
<rule pattern="#?[a-z_][a-z_\.\-]*\:[a-z0-9_\.\-/]+"><token type="NameTag"/></rule>
<rule pattern="#?[a-z_][a-z0-9_\.\-/]+"><token type="NameTag"/></rule>
<rule><include state="literals"/></rule>
<rule><include state="property"/></rule>
<rule><pop depth="1"/></rule>
</state>
</rules>
</lexer>
+1 -1
View File
@@ -106,7 +106,7 @@
</bygroups>
<push state="interpol"/>
</rule>
<rule pattern="(&amp;&amp;|&gt;=|&lt;=|\+\+|-&gt;|!=|\|\||//|==|@|!|\+|\?|&lt;|\.|&gt;|\*)">
<rule pattern="(&amp;&amp;|&gt;=|&lt;=|\+\+|-&gt;|!=|=|\|\||//|==|@|!|\+|\?|&lt;|\.|&gt;|\*)">
<token type="Operator"/>
</rule>
<rule pattern="[;:]">
@@ -0,0 +1,59 @@
<lexer>
<config>
<name>NSIS</name>
<alias>nsis</alias>
<alias>nsi</alias>
<alias>nsh</alias>
<filename>*.nsi</filename>
<filename>*.nsh</filename>
<mime_type>text/x-nsis</mime_type>
<case_insensitive>true</case_insensitive>
<not_multiline>true</not_multiline>
</config>
<rules>
<state name="root">
<rule pattern="([;#].*)(\n)"><bygroups><token type="Comment"/><token type="TextWhitespace"/></bygroups></rule>
<rule pattern="&#x27;.*?&#x27;"><token type="LiteralStringSingle"/></rule>
<rule pattern="&quot;"><token type="LiteralStringDouble"/><push state="str_double"/></rule>
<rule pattern="`"><token type="LiteralStringBacktick"/><push state="str_backtick"/></rule>
<rule><include state="macro"/></rule>
<rule><include state="interpol"/></rule>
<rule><include state="basic"/></rule>
<rule pattern="\$\{[a-z_|][\w|]*\}"><token type="KeywordPseudo"/></rule>
<rule pattern="/[a-z_]\w*"><token type="NameAttribute"/></rule>
<rule pattern="\s+"><token type="TextWhitespace"/></rule>
<rule pattern="[\w.]+"><token type="Text"/></rule>
</state>
<state name="basic">
<rule pattern="(\n)(Function)(\s+)([._a-z][.\w]*)\b"><bygroups><token type="TextWhitespace"/><token type="Keyword"/><token type="TextWhitespace"/><token type="NameFunction"/></bygroups></rule>
<rule pattern="\b([_a-z]\w*)(::)([a-z][a-z0-9]*)\b"><bygroups><token type="KeywordNamespace"/><token type="Punctuation"/><token type="NameFunction"/></bygroups></rule>
<rule pattern="\b([_a-z]\w*)(:)"><bygroups><token type="NameLabel"/><token type="Punctuation"/></bygroups></rule>
<rule pattern="(\b[ULS]|\B)([!&lt;&gt;=]?=|\&lt;\&gt;?|\&gt;)\B"><token type="Operator"/></rule>
<rule pattern="[|+-]"><token type="Operator"/></rule>
<rule pattern="\\"><token type="Punctuation"/></rule>
<rule pattern="\b(Abort|Add(?:BrandingImage|Size)|Allow(?:RootDirInstall|SkipFiles)|AutoCloseWindow|BG(?:Font|Gradient)|BrandingText|BringToFront|Call(?:InstDLL)?|(?:Sub)?Caption|ChangeUI|CheckBitmap|ClearErrors|CompletedText|ComponentText|CopyFiles|CRCCheck|Create(?:Directory|Font|Shortcut)|Delete(?:INI(?:Sec|Str)|Reg(?:Key|Value))?|DetailPrint|DetailsButtonText|Dir(?:Show|Text|Var|Verify)|(?:Disabled|Enabled)Bitmap|EnableWindow|EnumReg(?:Key|Value)|Exch|Exec(?:Shell|Wait)?|ExpandEnvStrings|File(?:BufSize|Close|ErrorText|Open|Read(?:Byte)?|Seek|Write(?:Byte)?)?|Find(?:Close|First|Next|Window)|FlushINI|Function(?:End)?|Get(?:CurInstType|CurrentAddress|DlgItem|DLLVersion(?:Local)?|ErrorLevel|FileTime(?:Local)?|FullPathName|FunctionAddress|InstDirError|LabelAddress|TempFileName)|Goto|HideWindow|Icon|If(?:Abort|Errors|FileExists|RebootFlag|Silent)|InitPluginsDir|Install(?:ButtonText|Colors|Dir(?:RegKey)?)|Inst(?:ProgressFlags|Type(?:[GS]etText)?)|Int(?:CmpU?|Fmt|Op)|IsWindow|LangString(?:UP)?|License(?:BkColor|Data|ForceSelection|LangString|Text)|LoadLanguageFile|LockWindow|Log(?:Set|Text)|MessageBox|MiscButtonText|Name|Nop|OutFile|(?:Uninst)?Page(?:Ex(?:End)?)?|PluginDir|Pop|Push|Quit|Read(?:(?:Env|INI|Reg)Str|RegDWORD)|Reboot|(?:Un)?RegDLL|Rename|RequestExecutionLevel|ReserveFile|Return|RMDir|SearchPath|Section(?:Divider|End|(?:(?:Get|Set)(?:Flags|InstTypes|Size|Text))|Group(?:End)?|In)?|SendMessage|Set(?:AutoClose|BrandingImage|Compress(?:ionLevel|or(?:DictSize)?)?|CtlColors|CurInstType|DatablockOptimize|DateSave|Details(?:Print|View)|Error(?:s|Level)|FileAttributes|Font|OutPath|Overwrite|PluginUnload|RebootFlag|ShellVarContext|Silent|StaticBkColor)|Show(?:(?:I|Uni)nstDetails|Window)|Silent(?:Un)?Install|Sleep|SpaceTexts|Str(?:CmpS?|Cpy|Len)|SubSection(?:End)?|Uninstall(?:ButtonText|(?:Sub)?Caption|EXEName|Icon|Text)|UninstPage|Var|VI(?:AddVersionKey|ProductVersion)|WindowIcon|Write(?:INIStr|Reg(:?Bin|DWORD|(?:Expand)?Str)|Uninstaller)|XPStyle)\b"><token type="Keyword"/></rule>
<rule pattern="\b(CUR|END|(?:FILE_ATTRIBUTE_)?(?:ARCHIVE|HIDDEN|NORMAL|OFFLINE|READONLY|SYSTEM|TEMPORARY)|HK(CC|CR|CU|DD|LM|PD|U)|HKEY_(?:CLASSES_ROOT|CURRENT_(?:CONFIG|USER)|DYN_DATA|LOCAL_MACHINE|PERFORMANCE_DATA|USERS)|ID(?:ABORT|CANCEL|IGNORE|NO|OK|RETRY|YES)|MB_(?:ABORTRETRYIGNORE|DEFBUTTON[1-4]|ICON(?:EXCLAMATION|INFORMATION|QUESTION|STOP)|OK(?:CANCEL)?|RETRYCANCEL|RIGHT|SETFOREGROUND|TOPMOST|USERICON|YESNO(?:CANCEL)?)|SET|SHCTX|SW_(?:HIDE|SHOW(?:MAXIMIZED|MINIMIZED|NORMAL))|admin|all|auto|both|bottom|bzip2|checkbox|colored|current|false|force|hide|highest|if(?:diff|newer)|lastused|leave|left|listonly|lzma|nevershow|none|normal|off|on|pop|push|radiobuttons|right|show|silent|silentlog|smooth|textonly|top|true|try|user|zlib)\b"><token type="NameConstant"/></rule>
</state>
<state name="macro">
<rule pattern="\!(addincludedir(?:dir)?|addplugindir|appendfile|cd|define|delfilefile|echo(?:message)?|else|endif|error|execute|if(?:macro)?n?(?:def)?|include|insertmacro|macro(?:end)?|packhdr|search(?:parse|replace)|system|tempfilesymbol|undef|verbose|warning)\b"><token type="CommentPreproc"/></rule>
</state>
<state name="interpol">
<rule pattern="\$(R?[0-9])"><token type="NameBuiltinPseudo"/></rule>
<rule pattern="\$(ADMINTOOLS|APPDATA|CDBURN_AREA|COOKIES|COMMONFILES(?:32|64)|DESKTOP|DOCUMENTS|EXE(?:DIR|FILE|PATH)|FAVORITES|FONTS|HISTORY|HWNDPARENT|INTERNET_CACHE|LOCALAPPDATA|MUSIC|NETHOOD|PICTURES|PLUGINSDIR|PRINTHOOD|PROFILE|PROGRAMFILES(?:32|64)|QUICKLAUNCH|RECENT|RESOURCES(?:_LOCALIZED)?|SENDTO|SM(?:PROGRAMS|STARTUP)|STARTMENU|SYSDIR|TEMP(?:LATES)?|VIDEOS|WINDIR|\{NSISDIR\})"><token type="NameBuiltin"/></rule>
<rule pattern="\$(CMDLINE|INSTDIR|OUTDIR|LANGUAGE)"><token type="NameVariableGlobal"/></rule>
<rule pattern="\$[a-z_]\w*"><token type="NameVariable"/></rule>
</state>
<state name="str_double">
<rule pattern="&quot;"><token type="LiteralStringDouble"/><pop depth="1"/></rule>
<rule pattern="\$(\\[nrt&quot;]|\$)"><token type="LiteralStringEscape"/></rule>
<rule><include state="interpol"/></rule>
<rule pattern="[^&quot;]+"><token type="LiteralStringDouble"/></rule>
</state>
<state name="str_backtick">
<rule pattern="`"><token type="LiteralStringDouble"/><pop depth="1"/></rule>
<rule pattern="\$(\\[nrt&quot;]|\$)"><token type="LiteralStringEscape"/></rule>
<rule><include state="interpol"/></rule>
<rule pattern="[^`]+"><token type="LiteralStringDouble"/></rule>
</state>
</rules>
</lexer>
@@ -0,0 +1,58 @@
<lexer>
<config>
<name>SNBT</name>
<alias>snbt</alias>
<filename>*.snbt</filename>
<mime_type>text/snbt</mime_type>
</config>
<rules>
<state name="root">
<rule pattern="\{"><token type="Punctuation"/><push state="compound"/></rule>
<rule pattern="[^\{]+"><token type="Text"/></rule>
</state>
<state name="whitespace">
<rule pattern="\s+"><token type="TextWhitespace"/></rule>
</state>
<state name="operators">
<rule pattern="[,:;]"><token type="Punctuation"/></rule>
</state>
<state name="literals">
<rule pattern="(true|false)"><token type="KeywordConstant"/></rule>
<rule pattern="-?\d+[eE]-?\d+"><token type="LiteralNumberFloat"/></rule>
<rule pattern="-?\d*\.\d+[fFdD]?"><token type="LiteralNumberFloat"/></rule>
<rule pattern="-?\d+[bBsSlLfFdD]?"><token type="LiteralNumberInteger"/></rule>
<rule pattern="&quot;"><token type="LiteralStringDouble"/><push state="literals.string_double"/></rule>
<rule pattern="&#x27;"><token type="LiteralStringSingle"/><push state="literals.string_single"/></rule>
</state>
<state name="literals.string_double">
<rule pattern="\\."><token type="LiteralStringEscape"/></rule>
<rule pattern="[^\\&quot;\n]+"><token type="LiteralStringDouble"/></rule>
<rule pattern="&quot;"><token type="LiteralStringDouble"/><pop depth="1"/></rule>
</state>
<state name="literals.string_single">
<rule pattern="\\."><token type="LiteralStringEscape"/></rule>
<rule pattern="[^\\&#x27;\n]+"><token type="LiteralStringSingle"/></rule>
<rule pattern="&#x27;"><token type="LiteralStringSingle"/><pop depth="1"/></rule>
</state>
<state name="compound">
<rule pattern="[A-Z_a-z]+"><token type="NameAttribute"/></rule>
<rule><include state="operators"/></rule>
<rule><include state="whitespace"/></rule>
<rule><include state="literals"/></rule>
<rule pattern="\{"><token type="Punctuation"/><push/></rule>
<rule pattern="\["><token type="Punctuation"/><push state="list"/></rule>
<rule pattern="\}"><token type="Punctuation"/><pop depth="1"/></rule>
</state>
<state name="list">
<rule pattern="[A-Z_a-z]+"><token type="NameAttribute"/></rule>
<rule><include state="literals"/></rule>
<rule><include state="operators"/></rule>
<rule><include state="whitespace"/></rule>
<rule pattern="\["><token type="Punctuation"/><push/></rule>
<rule pattern="\{"><token type="Punctuation"/><push state="compound"/></rule>
<rule pattern="\]"><token type="Punctuation"/><pop depth="1"/></rule>
</state>
</rules>
</lexer>
@@ -51,6 +51,22 @@
</rule>
</state>
<state name="tag">
<rule>
<include state="jsx"/>
</rule>
<rule pattern=",">
<token type="Punctuation"/>
</rule>
<rule pattern="&#34;(\\\\|\\&#34;|[^&#34;])*&#34;">
<token type="LiteralStringDouble"/>
</rule>
<rule pattern="&#39;(\\\\|\\&#39;|[^&#39;])*&#39;">
<token type="LiteralStringSingle"/>
</rule>
<rule pattern="`">
<token type="LiteralStringBacktick"/>
<push state="interp"/>
</rule>
<rule>
<include state="commentsandwhitespace"/>
</rule>
@@ -171,7 +187,7 @@
</rule>
<rule pattern="(?=/)">
<token type="Text"/>
<push state="#pop" state="badregex"/>
<push state="badregex"/>
</rule>
<rule>
<pop depth="1"/>
@@ -276,4 +292,4 @@
</rule>
</state>
</rules>
</lexer>
</lexer>
@@ -0,0 +1,108 @@
<lexer>
<config>
<name>Typst</name>
<alias>typst</alias>
<filename>*.typ</filename>
<mime_type>text/x-typst</mime_type>
</config>
<rules>
<state name="root">
<rule><include state="markup"/></rule>
</state>
<state name="into_code">
<rule pattern="(\#let|\#set|\#show)\b"><token type="KeywordDeclaration"/><push state="inline_code"/></rule>
<rule pattern="(\#import|\#include)\b"><token type="KeywordNamespace"/><push state="inline_code"/></rule>
<rule pattern="(\#if|\#for|\#while|\#export)\b"><token type="KeywordReserved"/><push state="inline_code"/></rule>
<rule pattern="#\{"><token type="Punctuation"/><push state="code"/></rule>
<rule pattern="#\("><token type="Punctuation"/><push state="code"/></rule>
<rule pattern="(#[a-zA-Z_][a-zA-Z0-9_-]*)(\[)"><bygroups><token type="NameFunction"/><token type="Punctuation"/></bygroups><push state="markup"/></rule>
<rule pattern="(#[a-zA-Z_][a-zA-Z0-9_-]*)(\()"><bygroups><token type="NameFunction"/><token type="Punctuation"/></bygroups><push state="code"/></rule>
<rule pattern="(\#true|\#false|\#none|\#auto)\b"><token type="KeywordConstant"/></rule>
<rule pattern="#[a-zA-Z_][a-zA-Z0-9_]*"><token type="NameVariable"/></rule>
<rule pattern="#0x[0-9a-fA-F]+"><token type="LiteralNumberHex"/></rule>
<rule pattern="#0b[01]+"><token type="LiteralNumberBin"/></rule>
<rule pattern="#0o[0-7]+"><token type="LiteralNumberOct"/></rule>
<rule pattern="#[0-9]+[\.e][0-9]+"><token type="LiteralNumberFloat"/></rule>
<rule pattern="#[0-9]+"><token type="LiteralNumberInteger"/></rule>
</state>
<state name="markup">
<rule><include state="comment"/></rule>
<rule pattern="^\s*=+.*$"><token type="GenericHeading"/></rule>
<rule pattern="[*][^*]*[*]"><token type="GenericStrong"/></rule>
<rule pattern="_[^_]*_"><token type="GenericEmph"/></rule>
<rule pattern="\$"><token type="Punctuation"/><push state="math"/></rule>
<rule pattern="`[^`]*`"><token type="LiteralStringBacktick"/></rule>
<rule pattern="^(\s*)(-)(\s+)"><bygroups><token type="TextWhitespace"/><token type="Punctuation"/><token type="TextWhitespace"/></bygroups></rule>
<rule pattern="^(\s*)(\+)(\s+)"><bygroups><token type="TextWhitespace"/><token type="Punctuation"/><token type="TextWhitespace"/></bygroups></rule>
<rule pattern="^(\s*)([0-9]+\.)"><bygroups><token type="TextWhitespace"/><token type="Punctuation"/></bygroups></rule>
<rule pattern="^(\s*)(/)(\s+)([^:]+)(:)"><bygroups><token type="TextWhitespace"/><token type="Punctuation"/><token type="TextWhitespace"/><token type="NameVariable"/><token type="Punctuation"/></bygroups></rule>
<rule pattern="&lt;[a-zA-Z_][a-zA-Z0-9_-]*&gt;"><token type="NameLabel"/></rule>
<rule pattern="@[a-zA-Z_][a-zA-Z0-9_-]*"><token type="NameLabel"/></rule>
<rule pattern="\\#"><token type="Text"/></rule>
<rule><include state="into_code"/></rule>
<rule pattern="```(?:.|\n)*?```"><token type="LiteralStringBacktick"/></rule>
<rule pattern="https?://[0-9a-zA-Z~/%#&amp;=\&#x27;,;.+?]*"><token type="GenericEmph"/></rule>
<rule pattern="(\-\-\-|\\|\~|\-\-|\.\.\.)\B"><token type="Punctuation"/></rule>
<rule pattern="\\\["><token type="Punctuation"/></rule>
<rule pattern="\\\]"><token type="Punctuation"/></rule>
<rule pattern="\["><token type="Punctuation"/><push/></rule>
<rule pattern="\]"><token type="Punctuation"/><pop depth="1"/></rule>
<rule pattern="[ \t]+\n?|\n"><token type="TextWhitespace"/></rule>
<rule pattern="((?![*_$`&lt;@\\#\] ]|https?://).)+"><token type="Text"/></rule>
</state>
<state name="math">
<rule><include state="comment"/></rule>
<rule pattern="(\\_|\\\^|\\\&amp;)"><token type="Text"/></rule>
<rule pattern="(_|\^|\&amp;|;)"><token type="Punctuation"/></rule>
<rule pattern="(\+|/|=|\[\||\|\]|\|\||\*|:=|::=|\.\.\.|&#x27;|\-|=:|!=|&gt;&gt;|&gt;=|&gt;&gt;&gt;|&lt;&lt;|&lt;=|&lt;&lt;&lt;|\-&gt;|\|\-&gt;|=&gt;|\|=&gt;|==&gt;|\-\-&gt;|\~\~&gt;|\~&gt;|&gt;\-&gt;|\-&gt;&gt;|&lt;\-|&lt;==|&lt;\-\-|&lt;\~\~|&lt;\~|&lt;\-&lt;|&lt;&lt;\-|&lt;\-&gt;|&lt;=&gt;|&lt;==&gt;|&lt;\-\-&gt;|&gt;|&lt;|\~|:|\|)"><token type="Operator"/></rule>
<rule pattern="\\"><token type="Punctuation"/></rule>
<rule pattern="\\\$"><token type="Punctuation"/></rule>
<rule pattern="\$"><token type="Punctuation"/><pop depth="1"/></rule>
<rule><include state="into_code"/></rule>
<rule pattern="([a-zA-Z][a-zA-Z0-9-]*)(\s*)(\()"><bygroups><token type="NameFunction"/><token type="TextWhitespace"/><token type="Punctuation"/></bygroups></rule>
<rule pattern="([a-zA-Z][a-zA-Z0-9-]*)(:)"><bygroups><token type="NameVariable"/><token type="Punctuation"/></bygroups></rule>
<rule pattern="([a-zA-Z][a-zA-Z0-9-]*)"><token type="NameVariable"/></rule>
<rule pattern="[0-9]+(\.[0-9]+)?"><token type="LiteralNumber"/></rule>
<rule pattern="\.{1,3}|\(|\)|,|\{|\}"><token type="Punctuation"/></rule>
<rule pattern="&quot;[^&quot;]*&quot;"><token type="LiteralStringDouble"/></rule>
<rule pattern="[ \t\n]+"><token type="TextWhitespace"/></rule>
</state>
<state name="comment">
<rule pattern="//.*$"><token type="CommentSingle"/></rule>
<rule pattern="/[*](.|\n)*?[*]/"><token type="CommentMultiline"/></rule>
</state>
<state name="code">
<rule><include state="comment"/></rule>
<rule pattern="\["><token type="Punctuation"/><push state="markup"/></rule>
<rule pattern="\(|\{"><token type="Punctuation"/><push state="code"/></rule>
<rule pattern="\)|\}"><token type="Punctuation"/><pop depth="1"/></rule>
<rule pattern="&quot;[^&quot;]*&quot;"><token type="LiteralStringDouble"/></rule>
<rule pattern=",|\.{1,2}"><token type="Punctuation"/></rule>
<rule pattern="="><token type="Operator"/></rule>
<rule pattern="(and|or|not)\b"><token type="OperatorWord"/></rule>
<rule pattern="=&gt;|&lt;=|==|!=|&gt;|&lt;|-=|\+=|\*=|/=|\+|-|\\|\*"><token type="Operator"/></rule>
<rule pattern="([a-zA-Z_][a-zA-Z0-9_-]*)(:)"><bygroups><token type="NameVariable"/><token type="Punctuation"/></bygroups></rule>
<rule pattern="([a-zA-Z_][a-zA-Z0-9_-]*)(\()"><bygroups><token type="NameFunction"/><token type="Punctuation"/></bygroups><push state="code"/></rule>
<rule pattern="(as|break|export|continue|else|for|if|in|return|while)\b"><token type="KeywordReserved"/></rule>
<rule pattern="(import|include)\b"><token type="KeywordNamespace"/></rule>
<rule pattern="(auto|none|true|false)\b"><token type="KeywordConstant"/></rule>
<rule pattern="([0-9.]+)(mm|pt|cm|in|em|fr|%)"><bygroups><token type="LiteralNumber"/><token type="KeywordReserved"/></bygroups></rule>
<rule pattern="0x[0-9a-fA-F]+"><token type="LiteralNumberHex"/></rule>
<rule pattern="0b[01]+"><token type="LiteralNumberBin"/></rule>
<rule pattern="0o[0-7]+"><token type="LiteralNumberOct"/></rule>
<rule pattern="[0-9]+[\.e][0-9]+"><token type="LiteralNumberFloat"/></rule>
<rule pattern="[0-9]+"><token type="LiteralNumberInteger"/></rule>
<rule pattern="(let|set|show)\b"><token type="KeywordDeclaration"/></rule>
<rule pattern="([a-zA-Z_][a-zA-Z0-9_-]*)"><token type="NameVariable"/></rule>
<rule pattern="[ \t\n]+"><token type="TextWhitespace"/></rule>
<rule pattern=":"><token type="Punctuation"/></rule>
</state>
<state name="inline_code">
<rule pattern=";\b"><token type="Punctuation"/><pop depth="1"/></rule>
<rule pattern="\n"><token type="TextWhitespace"/><pop depth="1"/></rule>
<rule><include state="code"/></rule>
</state>
</rules>
</lexer>
@@ -0,0 +1,283 @@
<lexer>
<config>
<name>WebVTT</name>
<alias>vtt</alias>
<filename>*.vtt</filename>
<mime_type>text/vtt</mime_type>
</config>
<!--
The WebVTT spec refers to a WebVTT line terminator as either CRLF, CR or LF.
(https://www.w3.org/TR/webvtt1/#webvtt-line-terminator) However, with this
definition it is unclear whether CRLF is one line terminator (CRLF) or two
line terminators (CR and LF).
To work around this ambiguity, only CRLF and LF are considered as line terminators.
To my knowledge only classic Mac OS uses CR as line terminators, so the lexer should
still work for most files.
-->
<rules>
<!-- https://www.w3.org/TR/webvtt1/#webvtt-file-body -->
<state name="root">
<rule pattern="(\AWEBVTT)((?:[ \t][^\r\n]*)?(?:\r?\n){2,})">
<bygroups>
<token type="Keyword" />
<token type="Text" />
</bygroups>
</rule>
<rule pattern="(^REGION)([ \t]*$)">
<bygroups>
<token type="Keyword" />
<token type="Text" />
</bygroups>
<push state="region-settings-list" />
</rule>
<rule
pattern="(^STYLE)([ \t]*$)((?:(?!&#45;&#45;&gt;)[\s\S])*?)((?:\r?\n){2})">
<bygroups>
<token type="Keyword" />
<token type="Text" />
<using lexer="CSS" />
<token type="Text" />
</bygroups>
</rule>
<rule>
<include state="comment" />
</rule>
<rule
pattern="(?=((?![^\r\n]*&#45;&#45;&gt;)[^\r\n]*\r?\n)?(\d{2}:)?(?:[0-5][0-9]):(?:[0-5][0-9])\.\d{3}[ \t]+&#45;&#45;&gt;[ \t]+(\d{2}:)?(?:[0-5][0-9]):(?:[0-5][0-9])\.\d{3})"
>
<push state="cues" />
</rule>
</state>
<!-- https://www.w3.org/TR/webvtt1/#webvtt-region-settings-list -->
<state name="region-settings-list">
<rule pattern="(?: |\t|\r?\n(?!\r?\n))+">
<token type="Text" />
</rule>
<rule pattern="(?:\r?\n){2}">
<token type="Text" />
<pop depth="1" />
</rule>
<rule pattern="(id)(:)(?!&#45;&#45;&gt;)(\S+)">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="Literal" />
</bygroups>
</rule>
<rule pattern="(width)(:)((?:[1-9]?\d|100)(?:\.\d+)?)(%)">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="Literal" />
<token type="KeywordType" />
</bygroups>
</rule>
<rule pattern="(lines)(:)(\d+)">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="Literal" />
</bygroups>
</rule>
<rule
pattern="(regionanchor|viewportanchor)(:)((?:[1-9]?\d|100)(?:\.\d+)?)(%)(,)((?:[1-9]?\d|100)(?:\.\d+)?)(%)">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="Literal" />
<token type="KeywordType" />
<token type="Punctuation" />
<token type="Literal" />
<token type="KeywordType" />
</bygroups>
</rule>
<rule pattern="(scroll)(:)(up)">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="KeywordConstant" />
</bygroups>
</rule>
</state>
<!-- https://www.w3.org/TR/webvtt1/#webvtt-comment-block -->
<state name="comment">
<rule
pattern="^NOTE( |\t|\r?\n)((?!&#45;&#45;&gt;)[\s\S])*?(?:(\r?\n){2}|\Z)">
<token type="Comment" />
</rule>
</state>
<!--
"Zero or more WebVTT cue blocks and WebVTT comment blocks separated from each other by one or more
WebVTT line terminators." (https://www.w3.org/TR/webvtt1/#file-structure)
-->
<state name="cues">
<rule
pattern="(?:((?!&#45;&#45;&gt;)[^\r\n]+)?(\r?\n))?((?:\d{2}:)?(?:[0-5][0-9]):(?:[0-5][0-9])\.\d{3})([ \t]+)(&#45;&#45;&gt;)([ \t]+)((?:\d{2}:)?(?:[0-5][0-9]):(?:[0-5][0-9])\.\d{3})([ \t]*)">
<bygroups>
<token type="Name" />
<token type="Text" />
<token type="LiteralDate" />
<token type="Text" />
<token type="Operator" />
<token type="Text" />
<token type="LiteralDate" />
<token type="Text" />
</bygroups>
<push state="cue-settings-list" />
</rule>
<rule>
<include state="comment" />
</rule>
</state>
<!-- https://www.w3.org/TR/webvtt1/#webvtt-cue-settings-list -->
<state name="cue-settings-list">
<rule pattern="[ \t]+">
<token type="Text" />
</rule>
<rule pattern="(vertical)(:)?(rl|lr)?">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="KeywordConstant" />
</bygroups>
</rule>
<rule
pattern="(line)(:)?(?:(?:((?:[1-9]?\d|100)(?:\.\d+)?)(%)|(-?\d+))(?:(,)(start|center|end))?)?">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="Literal" />
<token type="KeywordType" />
<token type="Literal" />
<token type="Punctuation" />
<token type="KeywordConstant" />
</bygroups>
</rule>
<rule
pattern="(position)(:)?(?:(?:((?:[1-9]?\d|100)(?:\.\d+)?)(%)|(-?\d+))(?:(,)(line-left|center|line-right))?)?">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="Literal" />
<token type="KeywordType" />
<token type="Literal" />
<token type="Punctuation" />
<token type="KeywordConstant" />
</bygroups>
</rule>
<rule pattern="(size)(:)?(?:((?:[1-9]?\d|100)(?:\.\d+)?)(%))?">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="Literal" />
<token type="KeywordType" />
</bygroups>
</rule>
<rule pattern="(align)(:)?(start|center|end|left|right)?">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="KeywordConstant" />
</bygroups>
</rule>
<rule pattern="(region)(:)?((?![^\r\n]*&#45;&#45;&gt;(?=[ \t]+?))[^ \t\r\n]+)?">
<bygroups>
<token type="Keyword" />
<token type="Punctuation" />
<token type="Literal" />
</bygroups>
</rule>
<rule
pattern="(?=\r?\n)">
<push state="cue-payload" />
</rule>
</state>
<!-- https://www.w3.org/TR/webvtt1/#cue-payload -->
<state name="cue-payload">
<rule pattern="(\r?\n){2,}">
<token type="Text" />
<pop depth="2" />
</rule>
<rule pattern="[^&lt;&amp;]+?">
<token type="Text" />
</rule>
<rule pattern="&amp;(#\d+|#x[0-9A-Fa-f]+|[a-zA-Z0-9]+);">
<token type="Text" />
</rule>
<rule pattern="(?=&lt;)">
<token type="Text" />
<push state="cue-span-tag" />
</rule>
</state>
<state name="cue-span-tag">
<rule
pattern="&lt;(?=c|i|b|u|ruby|rt|v|lang|(?:\d{2}:)?(?:[0-5][0-9]):(?:[0-5][0-9])\.\d{3})">
<token type="Punctuation" />
<push state="cue-span-start-tag-name" />
</rule>
<rule pattern="(&lt;/)(c|i|b|u|ruby|rt|v|lang)">
<bygroups>
<token type="Punctuation" />
<token type="NameTag" />
</bygroups>
</rule>
<rule pattern="&gt;">
<token type="Punctuation" />
<pop depth="1" />
</rule>
</state>
<state name="cue-span-start-tag-name">
<rule pattern="(c|i|b|u|ruby|rt)|((?:\d{2}:)?(?:[0-5][0-9]):(?:[0-5][0-9])\.\d{3})">
<bygroups>
<token type="NameTag" />
<token type="LiteralDate" />
</bygroups>
<push state="cue-span-classes-without-annotations" />
</rule>
<rule pattern="v|lang">
<token type="NameTag" />
<push state="cue-span-classes-with-annotations" />
</rule>
</state>
<state name="cue-span-classes-without-annotations">
<rule>
<include state="cue-span-classes" />
</rule>
<rule pattern="(?=&gt;)">
<pop depth="2" />
</rule>
</state>
<state name="cue-span-classes-with-annotations">
<rule>
<include state="cue-span-classes" />
</rule>
<rule pattern="(?=[ \t])">
<push state="cue-span-start-tag-annotations" />
</rule>
</state>
<state name="cue-span-classes">
<rule pattern="(\.)([^ \t\n\r&amp;&lt;&gt;\.]+)">
<bygroups>
<token type="Punctuation" />
<token type="NameTag" />
</bygroups>
</rule>
</state>
<state name="cue-span-start-tag-annotations">
<rule
pattern="[ \t](?:[^\n\r&amp;&gt;]|&amp;(?:#\d+|#x[0-9A-Fa-f]+|[a-zA-Z0-9]+);)+">
<token type="Text" />
</rule>
<rule pattern="(?=&gt;)">
<token type="Text" />
<pop depth="3" />
</rule>
</state>
</rules>
</lexer>
+1 -1
View File
@@ -53,7 +53,7 @@
<bygroups>
<token type="Punctuation"/>
<token type="LiteralStringDoc"/>
<token type="TextWhitespace"/>
<token type="Ignore"/>
</bygroups>
</rule>
<rule pattern="(false|False|FALSE|true|True|TRUE|null|Off|off|yes|Yes|YES|OFF|On|ON|no|No|on|NO|n|N|Y|y)\b">
+2 -3
View File
@@ -13,7 +13,6 @@ var Go = Register(MustNewLexer(
Aliases: []string{"go", "golang"},
Filenames: []string{"*.go"},
MimeTypes: []string{"text/x-gosrc"},
EnsureNL: true,
},
goRules,
).SetAnalyser(func(text string) float32 {
@@ -32,14 +31,14 @@ func goRules() Rules {
{`\n`, Text, nil},
{`\s+`, Text, nil},
{`\\\n`, Text, nil},
{`//(.*?)\n`, CommentSingle, nil},
{`//[^\n\r]*`, CommentSingle, nil},
{`/(\\\n)?[*](.|\n)*?[*](\\\n)?/`, CommentMultiline, nil},
{`(import|package)\b`, KeywordNamespace, nil},
{`(var|func|struct|map|chan|type|interface|const)\b`, KeywordDeclaration, nil},
{Words(``, `\b`, `break`, `default`, `select`, `case`, `defer`, `go`, `else`, `goto`, `switch`, `fallthrough`, `if`, `range`, `continue`, `for`, `return`), Keyword, nil},
{`(true|false|iota|nil)\b`, KeywordConstant, nil},
{Words(``, `\b(\()`, `uint`, `uint8`, `uint16`, `uint32`, `uint64`, `int`, `int8`, `int16`, `int32`, `int64`, `float`, `float32`, `float64`, `complex64`, `complex128`, `byte`, `rune`, `string`, `bool`, `error`, `uintptr`, `print`, `println`, `panic`, `recover`, `close`, `complex`, `real`, `imag`, `len`, `cap`, `append`, `copy`, `delete`, `new`, `make`, `clear`, `min`, `max`), ByGroups(NameBuiltin, Punctuation), nil},
{Words(``, `\b`, `uint`, `uint8`, `uint16`, `uint32`, `uint64`, `int`, `int8`, `int16`, `int32`, `int64`, `float`, `float32`, `float64`, `complex64`, `complex128`, `byte`, `rune`, `string`, `bool`, `error`, `uintptr`), KeywordType, nil},
{Words(``, `\b`, `uint`, `uint8`, `uint16`, `uint32`, `uint64`, `int`, `int8`, `int16`, `int32`, `int64`, `float`, `float32`, `float64`, `complex64`, `complex128`, `byte`, `rune`, `string`, `bool`, `error`, `uintptr`, `any`), KeywordType, nil},
{`\d+i`, LiteralNumber, nil},
{`\d+\.\d*([Ee][-+]\d+)?i`, LiteralNumber, nil},
{`\.\d+([Ee][-+]\d+)?i`, LiteralNumber, nil},
-1
View File
@@ -40,7 +40,6 @@ func markdownRules() Rules {
{"`[^`]+`", LiteralStringBacktick, nil},
{`[@#][\w/:]+`, NameEntity, nil},
{`(!?\[)([^]]+)(\])(\()([^)]+)(\))`, ByGroups(Text, NameTag, Text, Text, NameAttribute, Text), nil},
{`[^\\\s]+`, Other, nil},
{`.|\n`, Other, nil},
},
}
+6
View File
@@ -194,6 +194,9 @@ func (l *LexerState) Iterator() Token { // nolint: gocognit
for len(l.iteratorStack) > 0 {
n := len(l.iteratorStack) - 1
t := l.iteratorStack[n]()
if t.Type == Ignore {
continue
}
if t == EOF {
l.iteratorStack = l.iteratorStack[:n]
continue
@@ -243,6 +246,9 @@ func (l *LexerState) Iterator() Token { // nolint: gocognit
for len(l.iteratorStack) > 0 {
n := len(l.iteratorStack) - 1
t := l.iteratorStack[n]()
if t.Type == Ignore {
continue
}
if t == EOF {
l.iteratorStack = l.iteratorStack[:n]
continue
+33
View File
@@ -0,0 +1,33 @@
<style name="evergarden">
<entry type="Background" style="noinherit #D6CBB4 bg:#252B2E"/>
<entry type="Keyword" style="noinherit #E67E80"/>
<entry type="KeywordType" style="noinherit #DBBC7F"/>
<entry type="Name" style="#D6CBB4"/>
<entry type="NameAttribute" style="bold #D699B6"/>
<entry type="NameBuiltin" style="#D699B6"/>
<entry type="NameConstant" style="noinherit #D699B6"/>
<entry type="NameEntity" style="noinherit #DBBC7F"/>
<entry type="NameException" style="noinherit #E67E80"/>
<entry type="NameFunction" style="#B2C98F"/>
<entry type="NameLabel" style="noinherit #E67E80"/>
<entry type="NameTag" style="noinherit #7a8478"/>
<entry type="NameVariable" style="noinherit #D6CBB4"/>
<entry type="LiteralString" style="noinherit #B2C98F"/>
<entry type="LiteralStringSymbol" style="#E69875"/>
<entry type="LiteralNumber" style="noinherit #D699B6"/>
<entry type="LiteralNumberFloat" style="noinherit #D699B6"/>
<entry type="Operator" style="#7a8478"/>
<entry type="Comment" style="italic #859289"/>
<entry type="CommentPreproc" style="noinherit #E67E80"/>
<entry type="Generic" style="#D6CBB4"/>
<entry type="GenericDeleted" style="noinherit #252B2E bg:#E67E80"/>
<entry type="GenericEmph" style="#6E8585"/>
<entry type="GenericError" style="bold bg:#E67E80"/>
<entry type="GenericHeading" style="bold #D699B6"/>
<entry type="GenericInserted" style="noinherit #252B2E bg:#B2C98F"/>
<entry type="GenericOutput" style="noinherit #6E8585"/>
<entry type="GenericPrompt" style="#D6CBB4"/>
<entry type="GenericStrong" style="#D6CBB4"/>
<entry type="GenericSubheading" style="bold #B2C98F"/>
<entry type="GenericTraceback" style="bold bg:#E67E80"/>
</style>
+34 -39
View File
@@ -1,44 +1,39 @@
<style name="github">
<entry type="Error" style="#a61717 bg:#e3d2d2"/>
<entry type="Error" style="#f6f8fa bg:#82071e"/>
<entry type="Background" style="bg:#ffffff"/>
<entry type="Keyword" style="bold #000000"/>
<entry type="KeywordType" style="bold #445588"/>
<entry type="NameAttribute" style="#008080"/>
<entry type="NameBuiltin" style="#0086b3"/>
<entry type="NameBuiltinPseudo" style="#999999"/>
<entry type="NameClass" style="bold #445588"/>
<entry type="NameConstant" style="#008080"/>
<entry type="NameDecorator" style="bold #3c5d5d"/>
<entry type="NameEntity" style="#800080"/>
<entry type="NameException" style="bold #990000"/>
<entry type="NameFunction" style="bold #990000"/>
<entry type="Keyword" style="#cf222e"/>
<entry type="KeywordType" style="#cf222e"/>
<entry type="NameAttribute" style="#1f2328"/>
<entry type="NameBuiltin" style="#6639ba"/>
<entry type="NameBuiltinPseudo" style="#6a737d"/>
<entry type="NameClass" style="#1f2328"/>
<entry type="NameConstant" style="#0550ae"/>
<entry type="NameDecorator" style="#0550ae"/>
<entry type="NameEntity" style="#6639ba"/>
<entry type="NameFunction" style="#6639ba"/>
<entry type="NameLabel" style="bold #990000"/>
<entry type="NameNamespace" style="#555555"/>
<entry type="NameTag" style="#000080"/>
<entry type="NameVariable" style="#008080"/>
<entry type="NameVariableClass" style="#008080"/>
<entry type="NameVariableGlobal" style="#008080"/>
<entry type="NameVariableInstance" style="#008080"/>
<entry type="LiteralString" style="#dd1144"/>
<entry type="LiteralStringRegex" style="#009926"/>
<entry type="LiteralStringSymbol" style="#990073"/>
<entry type="LiteralNumber" style="#009999"/>
<entry type="Operator" style="bold #000000"/>
<entry type="Comment" style="italic #999988"/>
<entry type="CommentMultiline" style="italic #999988"/>
<entry type="CommentSingle" style="italic #999988"/>
<entry type="CommentSpecial" style="bold italic #999999"/>
<entry type="CommentPreproc" style="bold #999999"/>
<entry type="GenericDeleted" style="#000000 bg:#ffdddd"/>
<entry type="GenericEmph" style="italic #000000"/>
<entry type="GenericError" style="#aa0000"/>
<entry type="GenericHeading" style="#999999"/>
<entry type="GenericInserted" style="#000000 bg:#ddffdd"/>
<entry type="GenericOutput" style="#888888"/>
<entry type="GenericPrompt" style="#555555"/>
<entry type="GenericStrong" style="bold"/>
<entry type="GenericSubheading" style="#aaaaaa"/>
<entry type="GenericTraceback" style="#aa0000"/>
<entry type="NameNamespace" style="#24292e"/>
<entry type="NameOther" style="#1f2328"/>
<entry type="NameTag" style="#0550ae"/>
<entry type="NameVariable" style="#953800"/>
<entry type="NameVariableClass" style="#953800"/>
<entry type="NameVariableGlobal" style="#953800"/>
<entry type="NameVariableInstance" style="#953800"/>
<entry type="LiteralString" style="#0a3069"/>
<entry type="LiteralStringRegex" style="#0a3069"/>
<entry type="LiteralStringSymbol" style="#032f62"/>
<entry type="LiteralNumber" style="#0550ae"/>
<entry type="Operator" style="#0550ae"/>
<entry type="Comment" style="#57606a"/>
<entry type="CommentMultiline" style="#57606a"/>
<entry type="CommentSingle" style="#57606a"/>
<entry type="CommentSpecial" style="#57606a"/>
<entry type="CommentPreproc" style="#57606a"/>
<entry type="GenericDeleted" style="#82071e bg:#ffebe9"/>
<entry type="GenericEmph" style="#1f2328"/>
<entry type="GenericInserted" style="#116329 bg:#dafbe1"/>
<entry type="GenericOutput" style="#1f2328"/>
<entry type="GenericUnderline" style="underline"/>
<entry type="TextWhitespace" style="#bbbbbb"/>
<entry type="Punctuation" style="#1f2328"/>
<entry type="TextWhitespace" style="#ffffff"/>
</style>
+46
View File
@@ -0,0 +1,46 @@
<style name="nordic">
<entry type="Error" style="#C5727A"/>
<entry type="Background" style="#BBC3D4 bg:#242933"/>
<entry type="Keyword" style="bold #D08770"/>
<entry type="KeywordPseudo" style="nobold #D08770"/>
<entry type="KeywordType" style="nobold #D08770"/>
<entry type="Name" style="#BBC3D4"/>
<entry type="NameAttribute" style="#8FBCBB"/>
<entry type="NameBuiltin" style="#5E81AC"/>
<entry type="NameClass" style="#8FBCBB"/>
<entry type="NameConstant" style="#8FBCBB"/>
<entry type="NameDecorator" style="#D08770"/>
<entry type="NameEntity" style="#D08770"/>
<entry type="NameException" style="#C5727A"/>
<entry type="NameFunction" style="#88C0D0"/>
<entry type="NameLabel" style="#8FBCBB"/>
<entry type="NameNamespace" style="#8FBCBB"/>
<entry type="NameOther" style="#BBC3D4"/>
<entry type="NameTag" style="#5E81AC"/>
<entry type="NameVariable" style="#BBC3D4"/>
<entry type="NameProperty" style="#8FBCBB"/>
<entry type="LiteralString" style="#A3BE8C"/>
<entry type="LiteralStringDoc" style="#4C566A"/>
<entry type="LiteralStringEscape" style="#EBCB8B"/>
<entry type="LiteralStringInterpol" style="#A3BE8C"/>
<entry type="LiteralStringOther" style="#A3BE8C"/>
<entry type="LiteralStringRegex" style="#EBCB8B"/>
<entry type="LiteralStringSymbol" style="#A3BE8C"/>
<entry type="LiteralNumber" style="#B48EAD"/>
<entry type="Operator" style="#5E81AC"/>
<entry type="OperatorWord" style="bold #5E81AC"/>
<entry type="Punctuation" style="#ECEFF4"/>
<entry type="Comment" style="italic #4C566A"/>
<entry type="CommentPreproc" style="#5E81AC"/>
<entry type="GenericDeleted" style="#C5727A"/>
<entry type="GenericEmph" style="italic"/>
<entry type="GenericError" style="#C5727A"/>
<entry type="GenericHeading" style="bold #88C0D0"/>
<entry type="GenericInserted" style="#A3BE8C"/>
<entry type="GenericOutput" style="#BBC3D4"/>
<entry type="GenericPrompt" style="bold #1E222A"/>
<entry type="GenericStrong" style="bold"/>
<entry type="GenericSubheading" style="bold #88C0D0"/>
<entry type="GenericTraceback" style="#C5727A"/>
<entry type="TextWhitespace" style="#BBC3D4"/>
</style>
+409 -399
View File
@@ -7,109 +7,111 @@ import (
"strings"
)
const _TokenTypeName = "NoneOtherErrorCodeLineLineLinkLineTableTDLineTableLineHighlightLineNumbersTableLineNumbersLinePreWrapperBackgroundEOFTypeKeywordKeywordConstantKeywordDeclarationKeywordNamespaceKeywordPseudoKeywordReservedKeywordTypeNameNameAttributeNameBuiltinNameBuiltinPseudoNameClassNameConstantNameDecoratorNameEntityNameExceptionNameFunctionNameFunctionMagicNameKeywordNameLabelNameNamespaceNameOperatorNameOtherNamePseudoNamePropertyNameTagNameVariableNameVariableAnonymousNameVariableClassNameVariableGlobalNameVariableInstanceNameVariableMagicLiteralLiteralDateLiteralOtherLiteralStringLiteralStringAffixLiteralStringAtomLiteralStringBacktickLiteralStringBooleanLiteralStringCharLiteralStringDelimiterLiteralStringDocLiteralStringDoubleLiteralStringEscapeLiteralStringHeredocLiteralStringInterpolLiteralStringNameLiteralStringOtherLiteralStringRegexLiteralStringSingleLiteralStringSymbolLiteralNumberLiteralNumberBinLiteralNumberFloatLiteralNumberHexLiteralNumberIntegerLiteralNumberIntegerLongLiteralNumberOctOperatorOperatorWordPunctuationCommentCommentHashbangCommentMultilineCommentSingleCommentSpecialCommentPreprocCommentPreprocFileGenericGenericDeletedGenericEmphGenericErrorGenericHeadingGenericInsertedGenericOutputGenericPromptGenericStrongGenericSubheadingGenericTracebackGenericUnderlineTextTextWhitespaceTextSymbolTextPunctuation"
const _TokenTypeLowerName = "noneothererrorcodelinelinelinklinetabletdlinetablelinehighlightlinenumberstablelinenumberslineprewrapperbackgroundeoftypekeywordkeywordconstantkeyworddeclarationkeywordnamespacekeywordpseudokeywordreservedkeywordtypenamenameattributenamebuiltinnamebuiltinpseudonameclassnameconstantnamedecoratornameentitynameexceptionnamefunctionnamefunctionmagicnamekeywordnamelabelnamenamespacenameoperatornameothernamepseudonamepropertynametagnamevariablenamevariableanonymousnamevariableclassnamevariableglobalnamevariableinstancenamevariablemagicliteralliteraldateliteralotherliteralstringliteralstringaffixliteralstringatomliteralstringbacktickliteralstringbooleanliteralstringcharliteralstringdelimiterliteralstringdocliteralstringdoubleliteralstringescapeliteralstringheredocliteralstringinterpolliteralstringnameliteralstringotherliteralstringregexliteralstringsingleliteralstringsymbolliteralnumberliteralnumberbinliteralnumberfloatliteralnumberhexliteralnumberintegerliteralnumberintegerlongliteralnumberoctoperatoroperatorwordpunctuationcommentcommenthashbangcommentmultilinecommentsinglecommentspecialcommentpreproccommentpreprocfilegenericgenericdeletedgenericemphgenericerrorgenericheadinggenericinsertedgenericoutputgenericpromptgenericstronggenericsubheadinggenerictracebackgenericunderlinetexttextwhitespacetextsymboltextpunctuation"
const _TokenTypeName = "IgnoreNoneOtherErrorCodeLineLineLinkLineTableTDLineTableLineHighlightLineNumbersTableLineNumbersLinePreWrapperBackgroundEOFTypeKeywordKeywordConstantKeywordDeclarationKeywordNamespaceKeywordPseudoKeywordReservedKeywordTypeNameNameAttributeNameBuiltinNameBuiltinPseudoNameClassNameConstantNameDecoratorNameEntityNameExceptionNameFunctionNameFunctionMagicNameKeywordNameLabelNameNamespaceNameOperatorNameOtherNamePseudoNamePropertyNameTagNameVariableNameVariableAnonymousNameVariableClassNameVariableGlobalNameVariableInstanceNameVariableMagicLiteralLiteralDateLiteralOtherLiteralStringLiteralStringAffixLiteralStringAtomLiteralStringBacktickLiteralStringBooleanLiteralStringCharLiteralStringDelimiterLiteralStringDocLiteralStringDoubleLiteralStringEscapeLiteralStringHeredocLiteralStringInterpolLiteralStringNameLiteralStringOtherLiteralStringRegexLiteralStringSingleLiteralStringSymbolLiteralNumberLiteralNumberBinLiteralNumberFloatLiteralNumberHexLiteralNumberIntegerLiteralNumberIntegerLongLiteralNumberOctLiteralNumberByteOperatorOperatorWordPunctuationCommentCommentHashbangCommentMultilineCommentSingleCommentSpecialCommentPreprocCommentPreprocFileGenericGenericDeletedGenericEmphGenericErrorGenericHeadingGenericInsertedGenericOutputGenericPromptGenericStrongGenericSubheadingGenericTracebackGenericUnderlineTextTextWhitespaceTextSymbolTextPunctuation"
const _TokenTypeLowerName = "ignorenoneothererrorcodelinelinelinklinetabletdlinetablelinehighlightlinenumberstablelinenumberslineprewrapperbackgroundeoftypekeywordkeywordconstantkeyworddeclarationkeywordnamespacekeywordpseudokeywordreservedkeywordtypenamenameattributenamebuiltinnamebuiltinpseudonameclassnameconstantnamedecoratornameentitynameexceptionnamefunctionnamefunctionmagicnamekeywordnamelabelnamenamespacenameoperatornameothernamepseudonamepropertynametagnamevariablenamevariableanonymousnamevariableclassnamevariableglobalnamevariableinstancenamevariablemagicliteralliteraldateliteralotherliteralstringliteralstringaffixliteralstringatomliteralstringbacktickliteralstringbooleanliteralstringcharliteralstringdelimiterliteralstringdocliteralstringdoubleliteralstringescapeliteralstringheredocliteralstringinterpolliteralstringnameliteralstringotherliteralstringregexliteralstringsingleliteralstringsymbolliteralnumberliteralnumberbinliteralnumberfloatliteralnumberhexliteralnumberintegerliteralnumberintegerlongliteralnumberoctliteralnumberbyteoperatoroperatorwordpunctuationcommentcommenthashbangcommentmultilinecommentsinglecommentspecialcommentpreproccommentpreprocfilegenericgenericdeletedgenericemphgenericerrorgenericheadinggenericinsertedgenericoutputgenericpromptgenericstronggenericsubheadinggenerictracebackgenericunderlinetexttextwhitespacetextsymboltextpunctuation"
var _TokenTypeMap = map[TokenType]string{
-13: _TokenTypeName[0:4],
-12: _TokenTypeName[4:9],
-11: _TokenTypeName[9:14],
-10: _TokenTypeName[14:22],
-9: _TokenTypeName[22:30],
-8: _TokenTypeName[30:41],
-7: _TokenTypeName[41:50],
-6: _TokenTypeName[50:63],
-5: _TokenTypeName[63:79],
-4: _TokenTypeName[79:90],
-3: _TokenTypeName[90:94],
-2: _TokenTypeName[94:104],
-1: _TokenTypeName[104:114],
0: _TokenTypeName[114:121],
1000: _TokenTypeName[121:128],
1001: _TokenTypeName[128:143],
1002: _TokenTypeName[143:161],
1003: _TokenTypeName[161:177],
1004: _TokenTypeName[177:190],
1005: _TokenTypeName[190:205],
1006: _TokenTypeName[205:216],
2000: _TokenTypeName[216:220],
2001: _TokenTypeName[220:233],
2002: _TokenTypeName[233:244],
2003: _TokenTypeName[244:261],
2004: _TokenTypeName[261:270],
2005: _TokenTypeName[270:282],
2006: _TokenTypeName[282:295],
2007: _TokenTypeName[295:305],
2008: _TokenTypeName[305:318],
2009: _TokenTypeName[318:330],
2010: _TokenTypeName[330:347],
2011: _TokenTypeName[347:358],
2012: _TokenTypeName[358:367],
2013: _TokenTypeName[367:380],
2014: _TokenTypeName[380:392],
2015: _TokenTypeName[392:401],
2016: _TokenTypeName[401:411],
2017: _TokenTypeName[411:423],
2018: _TokenTypeName[423:430],
2019: _TokenTypeName[430:442],
2020: _TokenTypeName[442:463],
2021: _TokenTypeName[463:480],
2022: _TokenTypeName[480:498],
2023: _TokenTypeName[498:518],
2024: _TokenTypeName[518:535],
3000: _TokenTypeName[535:542],
3001: _TokenTypeName[542:553],
3002: _TokenTypeName[553:565],
3100: _TokenTypeName[565:578],
3101: _TokenTypeName[578:596],
3102: _TokenTypeName[596:613],
3103: _TokenTypeName[613:634],
3104: _TokenTypeName[634:654],
3105: _TokenTypeName[654:671],
3106: _TokenTypeName[671:693],
3107: _TokenTypeName[693:709],
3108: _TokenTypeName[709:728],
3109: _TokenTypeName[728:747],
3110: _TokenTypeName[747:767],
3111: _TokenTypeName[767:788],
3112: _TokenTypeName[788:805],
3113: _TokenTypeName[805:823],
3114: _TokenTypeName[823:841],
3115: _TokenTypeName[841:860],
3116: _TokenTypeName[860:879],
3200: _TokenTypeName[879:892],
3201: _TokenTypeName[892:908],
3202: _TokenTypeName[908:926],
3203: _TokenTypeName[926:942],
3204: _TokenTypeName[942:962],
3205: _TokenTypeName[962:986],
3206: _TokenTypeName[986:1002],
4000: _TokenTypeName[1002:1010],
4001: _TokenTypeName[1010:1022],
5000: _TokenTypeName[1022:1033],
6000: _TokenTypeName[1033:1040],
6001: _TokenTypeName[1040:1055],
6002: _TokenTypeName[1055:1071],
6003: _TokenTypeName[1071:1084],
6004: _TokenTypeName[1084:1098],
6100: _TokenTypeName[1098:1112],
6101: _TokenTypeName[1112:1130],
7000: _TokenTypeName[1130:1137],
7001: _TokenTypeName[1137:1151],
7002: _TokenTypeName[1151:1162],
7003: _TokenTypeName[1162:1174],
7004: _TokenTypeName[1174:1188],
7005: _TokenTypeName[1188:1203],
7006: _TokenTypeName[1203:1216],
7007: _TokenTypeName[1216:1229],
7008: _TokenTypeName[1229:1242],
7009: _TokenTypeName[1242:1259],
7010: _TokenTypeName[1259:1275],
7011: _TokenTypeName[1275:1291],
8000: _TokenTypeName[1291:1295],
8001: _TokenTypeName[1295:1309],
8002: _TokenTypeName[1309:1319],
8003: _TokenTypeName[1319:1334],
-14: _TokenTypeName[0:6],
-13: _TokenTypeName[6:10],
-12: _TokenTypeName[10:15],
-11: _TokenTypeName[15:20],
-10: _TokenTypeName[20:28],
-9: _TokenTypeName[28:36],
-8: _TokenTypeName[36:47],
-7: _TokenTypeName[47:56],
-6: _TokenTypeName[56:69],
-5: _TokenTypeName[69:85],
-4: _TokenTypeName[85:96],
-3: _TokenTypeName[96:100],
-2: _TokenTypeName[100:110],
-1: _TokenTypeName[110:120],
0: _TokenTypeName[120:127],
1000: _TokenTypeName[127:134],
1001: _TokenTypeName[134:149],
1002: _TokenTypeName[149:167],
1003: _TokenTypeName[167:183],
1004: _TokenTypeName[183:196],
1005: _TokenTypeName[196:211],
1006: _TokenTypeName[211:222],
2000: _TokenTypeName[222:226],
2001: _TokenTypeName[226:239],
2002: _TokenTypeName[239:250],
2003: _TokenTypeName[250:267],
2004: _TokenTypeName[267:276],
2005: _TokenTypeName[276:288],
2006: _TokenTypeName[288:301],
2007: _TokenTypeName[301:311],
2008: _TokenTypeName[311:324],
2009: _TokenTypeName[324:336],
2010: _TokenTypeName[336:353],
2011: _TokenTypeName[353:364],
2012: _TokenTypeName[364:373],
2013: _TokenTypeName[373:386],
2014: _TokenTypeName[386:398],
2015: _TokenTypeName[398:407],
2016: _TokenTypeName[407:417],
2017: _TokenTypeName[417:429],
2018: _TokenTypeName[429:436],
2019: _TokenTypeName[436:448],
2020: _TokenTypeName[448:469],
2021: _TokenTypeName[469:486],
2022: _TokenTypeName[486:504],
2023: _TokenTypeName[504:524],
2024: _TokenTypeName[524:541],
3000: _TokenTypeName[541:548],
3001: _TokenTypeName[548:559],
3002: _TokenTypeName[559:571],
3100: _TokenTypeName[571:584],
3101: _TokenTypeName[584:602],
3102: _TokenTypeName[602:619],
3103: _TokenTypeName[619:640],
3104: _TokenTypeName[640:660],
3105: _TokenTypeName[660:677],
3106: _TokenTypeName[677:699],
3107: _TokenTypeName[699:715],
3108: _TokenTypeName[715:734],
3109: _TokenTypeName[734:753],
3110: _TokenTypeName[753:773],
3111: _TokenTypeName[773:794],
3112: _TokenTypeName[794:811],
3113: _TokenTypeName[811:829],
3114: _TokenTypeName[829:847],
3115: _TokenTypeName[847:866],
3116: _TokenTypeName[866:885],
3200: _TokenTypeName[885:898],
3201: _TokenTypeName[898:914],
3202: _TokenTypeName[914:932],
3203: _TokenTypeName[932:948],
3204: _TokenTypeName[948:968],
3205: _TokenTypeName[968:992],
3206: _TokenTypeName[992:1008],
3207: _TokenTypeName[1008:1025],
4000: _TokenTypeName[1025:1033],
4001: _TokenTypeName[1033:1045],
5000: _TokenTypeName[1045:1056],
6000: _TokenTypeName[1056:1063],
6001: _TokenTypeName[1063:1078],
6002: _TokenTypeName[1078:1094],
6003: _TokenTypeName[1094:1107],
6004: _TokenTypeName[1107:1121],
6100: _TokenTypeName[1121:1135],
6101: _TokenTypeName[1135:1153],
7000: _TokenTypeName[1153:1160],
7001: _TokenTypeName[1160:1174],
7002: _TokenTypeName[1174:1185],
7003: _TokenTypeName[1185:1197],
7004: _TokenTypeName[1197:1211],
7005: _TokenTypeName[1211:1226],
7006: _TokenTypeName[1226:1239],
7007: _TokenTypeName[1239:1252],
7008: _TokenTypeName[1252:1265],
7009: _TokenTypeName[1265:1282],
7010: _TokenTypeName[1282:1298],
7011: _TokenTypeName[1298:1314],
8000: _TokenTypeName[1314:1318],
8001: _TokenTypeName[1318:1332],
8002: _TokenTypeName[1332:1342],
8003: _TokenTypeName[1342:1357],
}
func (i TokenType) String() string {
@@ -123,6 +125,7 @@ func (i TokenType) String() string {
// Re-run the stringer command to generate them again.
func _TokenTypeNoOp() {
var x [1]struct{}
_ = x[Ignore-(-14)]
_ = x[None-(-13)]
_ = x[Other-(-12)]
_ = x[Error-(-11)]
@@ -196,6 +199,7 @@ func _TokenTypeNoOp() {
_ = x[LiteralNumberInteger-(3204)]
_ = x[LiteralNumberIntegerLong-(3205)]
_ = x[LiteralNumberOct-(3206)]
_ = x[LiteralNumberByte-(3207)]
_ = x[Operator-(4000)]
_ = x[OperatorWord-(4001)]
_ = x[Punctuation-(5000)]
@@ -224,309 +228,315 @@ func _TokenTypeNoOp() {
_ = x[TextPunctuation-(8003)]
}
var _TokenTypeValues = []TokenType{None, Other, Error, CodeLine, LineLink, LineTableTD, LineTable, LineHighlight, LineNumbersTable, LineNumbers, Line, PreWrapper, Background, EOFType, Keyword, KeywordConstant, KeywordDeclaration, KeywordNamespace, KeywordPseudo, KeywordReserved, KeywordType, Name, NameAttribute, NameBuiltin, NameBuiltinPseudo, NameClass, NameConstant, NameDecorator, NameEntity, NameException, NameFunction, NameFunctionMagic, NameKeyword, NameLabel, NameNamespace, NameOperator, NameOther, NamePseudo, NameProperty, NameTag, NameVariable, NameVariableAnonymous, NameVariableClass, NameVariableGlobal, NameVariableInstance, NameVariableMagic, Literal, LiteralDate, LiteralOther, LiteralString, LiteralStringAffix, LiteralStringAtom, LiteralStringBacktick, LiteralStringBoolean, LiteralStringChar, LiteralStringDelimiter, LiteralStringDoc, LiteralStringDouble, LiteralStringEscape, LiteralStringHeredoc, LiteralStringInterpol, LiteralStringName, LiteralStringOther, LiteralStringRegex, LiteralStringSingle, LiteralStringSymbol, LiteralNumber, LiteralNumberBin, LiteralNumberFloat, LiteralNumberHex, LiteralNumberInteger, LiteralNumberIntegerLong, LiteralNumberOct, Operator, OperatorWord, Punctuation, Comment, CommentHashbang, CommentMultiline, CommentSingle, CommentSpecial, CommentPreproc, CommentPreprocFile, Generic, GenericDeleted, GenericEmph, GenericError, GenericHeading, GenericInserted, GenericOutput, GenericPrompt, GenericStrong, GenericSubheading, GenericTraceback, GenericUnderline, Text, TextWhitespace, TextSymbol, TextPunctuation}
var _TokenTypeValues = []TokenType{Ignore, None, Other, Error, CodeLine, LineLink, LineTableTD, LineTable, LineHighlight, LineNumbersTable, LineNumbers, Line, PreWrapper, Background, EOFType, Keyword, KeywordConstant, KeywordDeclaration, KeywordNamespace, KeywordPseudo, KeywordReserved, KeywordType, Name, NameAttribute, NameBuiltin, NameBuiltinPseudo, NameClass, NameConstant, NameDecorator, NameEntity, NameException, NameFunction, NameFunctionMagic, NameKeyword, NameLabel, NameNamespace, NameOperator, NameOther, NamePseudo, NameProperty, NameTag, NameVariable, NameVariableAnonymous, NameVariableClass, NameVariableGlobal, NameVariableInstance, NameVariableMagic, Literal, LiteralDate, LiteralOther, LiteralString, LiteralStringAffix, LiteralStringAtom, LiteralStringBacktick, LiteralStringBoolean, LiteralStringChar, LiteralStringDelimiter, LiteralStringDoc, LiteralStringDouble, LiteralStringEscape, LiteralStringHeredoc, LiteralStringInterpol, LiteralStringName, LiteralStringOther, LiteralStringRegex, LiteralStringSingle, LiteralStringSymbol, LiteralNumber, LiteralNumberBin, LiteralNumberFloat, LiteralNumberHex, LiteralNumberInteger, LiteralNumberIntegerLong, LiteralNumberOct, LiteralNumberByte, Operator, OperatorWord, Punctuation, Comment, CommentHashbang, CommentMultiline, CommentSingle, CommentSpecial, CommentPreproc, CommentPreprocFile, Generic, GenericDeleted, GenericEmph, GenericError, GenericHeading, GenericInserted, GenericOutput, GenericPrompt, GenericStrong, GenericSubheading, GenericTraceback, GenericUnderline, Text, TextWhitespace, TextSymbol, TextPunctuation}
var _TokenTypeNameToValueMap = map[string]TokenType{
_TokenTypeName[0:4]: None,
_TokenTypeLowerName[0:4]: None,
_TokenTypeName[4:9]: Other,
_TokenTypeLowerName[4:9]: Other,
_TokenTypeName[9:14]: Error,
_TokenTypeLowerName[9:14]: Error,
_TokenTypeName[14:22]: CodeLine,
_TokenTypeLowerName[14:22]: CodeLine,
_TokenTypeName[22:30]: LineLink,
_TokenTypeLowerName[22:30]: LineLink,
_TokenTypeName[30:41]: LineTableTD,
_TokenTypeLowerName[30:41]: LineTableTD,
_TokenTypeName[41:50]: LineTable,
_TokenTypeLowerName[41:50]: LineTable,
_TokenTypeName[50:63]: LineHighlight,
_TokenTypeLowerName[50:63]: LineHighlight,
_TokenTypeName[63:79]: LineNumbersTable,
_TokenTypeLowerName[63:79]: LineNumbersTable,
_TokenTypeName[79:90]: LineNumbers,
_TokenTypeLowerName[79:90]: LineNumbers,
_TokenTypeName[90:94]: Line,
_TokenTypeLowerName[90:94]: Line,
_TokenTypeName[94:104]: PreWrapper,
_TokenTypeLowerName[94:104]: PreWrapper,
_TokenTypeName[104:114]: Background,
_TokenTypeLowerName[104:114]: Background,
_TokenTypeName[114:121]: EOFType,
_TokenTypeLowerName[114:121]: EOFType,
_TokenTypeName[121:128]: Keyword,
_TokenTypeLowerName[121:128]: Keyword,
_TokenTypeName[128:143]: KeywordConstant,
_TokenTypeLowerName[128:143]: KeywordConstant,
_TokenTypeName[143:161]: KeywordDeclaration,
_TokenTypeLowerName[143:161]: KeywordDeclaration,
_TokenTypeName[161:177]: KeywordNamespace,
_TokenTypeLowerName[161:177]: KeywordNamespace,
_TokenTypeName[177:190]: KeywordPseudo,
_TokenTypeLowerName[177:190]: KeywordPseudo,
_TokenTypeName[190:205]: KeywordReserved,
_TokenTypeLowerName[190:205]: KeywordReserved,
_TokenTypeName[205:216]: KeywordType,
_TokenTypeLowerName[205:216]: KeywordType,
_TokenTypeName[216:220]: Name,
_TokenTypeLowerName[216:220]: Name,
_TokenTypeName[220:233]: NameAttribute,
_TokenTypeLowerName[220:233]: NameAttribute,
_TokenTypeName[233:244]: NameBuiltin,
_TokenTypeLowerName[233:244]: NameBuiltin,
_TokenTypeName[244:261]: NameBuiltinPseudo,
_TokenTypeLowerName[244:261]: NameBuiltinPseudo,
_TokenTypeName[261:270]: NameClass,
_TokenTypeLowerName[261:270]: NameClass,
_TokenTypeName[270:282]: NameConstant,
_TokenTypeLowerName[270:282]: NameConstant,
_TokenTypeName[282:295]: NameDecorator,
_TokenTypeLowerName[282:295]: NameDecorator,
_TokenTypeName[295:305]: NameEntity,
_TokenTypeLowerName[295:305]: NameEntity,
_TokenTypeName[305:318]: NameException,
_TokenTypeLowerName[305:318]: NameException,
_TokenTypeName[318:330]: NameFunction,
_TokenTypeLowerName[318:330]: NameFunction,
_TokenTypeName[330:347]: NameFunctionMagic,
_TokenTypeLowerName[330:347]: NameFunctionMagic,
_TokenTypeName[347:358]: NameKeyword,
_TokenTypeLowerName[347:358]: NameKeyword,
_TokenTypeName[358:367]: NameLabel,
_TokenTypeLowerName[358:367]: NameLabel,
_TokenTypeName[367:380]: NameNamespace,
_TokenTypeLowerName[367:380]: NameNamespace,
_TokenTypeName[380:392]: NameOperator,
_TokenTypeLowerName[380:392]: NameOperator,
_TokenTypeName[392:401]: NameOther,
_TokenTypeLowerName[392:401]: NameOther,
_TokenTypeName[401:411]: NamePseudo,
_TokenTypeLowerName[401:411]: NamePseudo,
_TokenTypeName[411:423]: NameProperty,
_TokenTypeLowerName[411:423]: NameProperty,
_TokenTypeName[423:430]: NameTag,
_TokenTypeLowerName[423:430]: NameTag,
_TokenTypeName[430:442]: NameVariable,
_TokenTypeLowerName[430:442]: NameVariable,
_TokenTypeName[442:463]: NameVariableAnonymous,
_TokenTypeLowerName[442:463]: NameVariableAnonymous,
_TokenTypeName[463:480]: NameVariableClass,
_TokenTypeLowerName[463:480]: NameVariableClass,
_TokenTypeName[480:498]: NameVariableGlobal,
_TokenTypeLowerName[480:498]: NameVariableGlobal,
_TokenTypeName[498:518]: NameVariableInstance,
_TokenTypeLowerName[498:518]: NameVariableInstance,
_TokenTypeName[518:535]: NameVariableMagic,
_TokenTypeLowerName[518:535]: NameVariableMagic,
_TokenTypeName[535:542]: Literal,
_TokenTypeLowerName[535:542]: Literal,
_TokenTypeName[542:553]: LiteralDate,
_TokenTypeLowerName[542:553]: LiteralDate,
_TokenTypeName[553:565]: LiteralOther,
_TokenTypeLowerName[553:565]: LiteralOther,
_TokenTypeName[565:578]: LiteralString,
_TokenTypeLowerName[565:578]: LiteralString,
_TokenTypeName[578:596]: LiteralStringAffix,
_TokenTypeLowerName[578:596]: LiteralStringAffix,
_TokenTypeName[596:613]: LiteralStringAtom,
_TokenTypeLowerName[596:613]: LiteralStringAtom,
_TokenTypeName[613:634]: LiteralStringBacktick,
_TokenTypeLowerName[613:634]: LiteralStringBacktick,
_TokenTypeName[634:654]: LiteralStringBoolean,
_TokenTypeLowerName[634:654]: LiteralStringBoolean,
_TokenTypeName[654:671]: LiteralStringChar,
_TokenTypeLowerName[654:671]: LiteralStringChar,
_TokenTypeName[671:693]: LiteralStringDelimiter,
_TokenTypeLowerName[671:693]: LiteralStringDelimiter,
_TokenTypeName[693:709]: LiteralStringDoc,
_TokenTypeLowerName[693:709]: LiteralStringDoc,
_TokenTypeName[709:728]: LiteralStringDouble,
_TokenTypeLowerName[709:728]: LiteralStringDouble,
_TokenTypeName[728:747]: LiteralStringEscape,
_TokenTypeLowerName[728:747]: LiteralStringEscape,
_TokenTypeName[747:767]: LiteralStringHeredoc,
_TokenTypeLowerName[747:767]: LiteralStringHeredoc,
_TokenTypeName[767:788]: LiteralStringInterpol,
_TokenTypeLowerName[767:788]: LiteralStringInterpol,
_TokenTypeName[788:805]: LiteralStringName,
_TokenTypeLowerName[788:805]: LiteralStringName,
_TokenTypeName[805:823]: LiteralStringOther,
_TokenTypeLowerName[805:823]: LiteralStringOther,
_TokenTypeName[823:841]: LiteralStringRegex,
_TokenTypeLowerName[823:841]: LiteralStringRegex,
_TokenTypeName[841:860]: LiteralStringSingle,
_TokenTypeLowerName[841:860]: LiteralStringSingle,
_TokenTypeName[860:879]: LiteralStringSymbol,
_TokenTypeLowerName[860:879]: LiteralStringSymbol,
_TokenTypeName[879:892]: LiteralNumber,
_TokenTypeLowerName[879:892]: LiteralNumber,
_TokenTypeName[892:908]: LiteralNumberBin,
_TokenTypeLowerName[892:908]: LiteralNumberBin,
_TokenTypeName[908:926]: LiteralNumberFloat,
_TokenTypeLowerName[908:926]: LiteralNumberFloat,
_TokenTypeName[926:942]: LiteralNumberHex,
_TokenTypeLowerName[926:942]: LiteralNumberHex,
_TokenTypeName[942:962]: LiteralNumberInteger,
_TokenTypeLowerName[942:962]: LiteralNumberInteger,
_TokenTypeName[962:986]: LiteralNumberIntegerLong,
_TokenTypeLowerName[962:986]: LiteralNumberIntegerLong,
_TokenTypeName[986:1002]: LiteralNumberOct,
_TokenTypeLowerName[986:1002]: LiteralNumberOct,
_TokenTypeName[1002:1010]: Operator,
_TokenTypeLowerName[1002:1010]: Operator,
_TokenTypeName[1010:1022]: OperatorWord,
_TokenTypeLowerName[1010:1022]: OperatorWord,
_TokenTypeName[1022:1033]: Punctuation,
_TokenTypeLowerName[1022:1033]: Punctuation,
_TokenTypeName[1033:1040]: Comment,
_TokenTypeLowerName[1033:1040]: Comment,
_TokenTypeName[1040:1055]: CommentHashbang,
_TokenTypeLowerName[1040:1055]: CommentHashbang,
_TokenTypeName[1055:1071]: CommentMultiline,
_TokenTypeLowerName[1055:1071]: CommentMultiline,
_TokenTypeName[1071:1084]: CommentSingle,
_TokenTypeLowerName[1071:1084]: CommentSingle,
_TokenTypeName[1084:1098]: CommentSpecial,
_TokenTypeLowerName[1084:1098]: CommentSpecial,
_TokenTypeName[1098:1112]: CommentPreproc,
_TokenTypeLowerName[1098:1112]: CommentPreproc,
_TokenTypeName[1112:1130]: CommentPreprocFile,
_TokenTypeLowerName[1112:1130]: CommentPreprocFile,
_TokenTypeName[1130:1137]: Generic,
_TokenTypeLowerName[1130:1137]: Generic,
_TokenTypeName[1137:1151]: GenericDeleted,
_TokenTypeLowerName[1137:1151]: GenericDeleted,
_TokenTypeName[1151:1162]: GenericEmph,
_TokenTypeLowerName[1151:1162]: GenericEmph,
_TokenTypeName[1162:1174]: GenericError,
_TokenTypeLowerName[1162:1174]: GenericError,
_TokenTypeName[1174:1188]: GenericHeading,
_TokenTypeLowerName[1174:1188]: GenericHeading,
_TokenTypeName[1188:1203]: GenericInserted,
_TokenTypeLowerName[1188:1203]: GenericInserted,
_TokenTypeName[1203:1216]: GenericOutput,
_TokenTypeLowerName[1203:1216]: GenericOutput,
_TokenTypeName[1216:1229]: GenericPrompt,
_TokenTypeLowerName[1216:1229]: GenericPrompt,
_TokenTypeName[1229:1242]: GenericStrong,
_TokenTypeLowerName[1229:1242]: GenericStrong,
_TokenTypeName[1242:1259]: GenericSubheading,
_TokenTypeLowerName[1242:1259]: GenericSubheading,
_TokenTypeName[1259:1275]: GenericTraceback,
_TokenTypeLowerName[1259:1275]: GenericTraceback,
_TokenTypeName[1275:1291]: GenericUnderline,
_TokenTypeLowerName[1275:1291]: GenericUnderline,
_TokenTypeName[1291:1295]: Text,
_TokenTypeLowerName[1291:1295]: Text,
_TokenTypeName[1295:1309]: TextWhitespace,
_TokenTypeLowerName[1295:1309]: TextWhitespace,
_TokenTypeName[1309:1319]: TextSymbol,
_TokenTypeLowerName[1309:1319]: TextSymbol,
_TokenTypeName[1319:1334]: TextPunctuation,
_TokenTypeLowerName[1319:1334]: TextPunctuation,
_TokenTypeName[0:6]: Ignore,
_TokenTypeLowerName[0:6]: Ignore,
_TokenTypeName[6:10]: None,
_TokenTypeLowerName[6:10]: None,
_TokenTypeName[10:15]: Other,
_TokenTypeLowerName[10:15]: Other,
_TokenTypeName[15:20]: Error,
_TokenTypeLowerName[15:20]: Error,
_TokenTypeName[20:28]: CodeLine,
_TokenTypeLowerName[20:28]: CodeLine,
_TokenTypeName[28:36]: LineLink,
_TokenTypeLowerName[28:36]: LineLink,
_TokenTypeName[36:47]: LineTableTD,
_TokenTypeLowerName[36:47]: LineTableTD,
_TokenTypeName[47:56]: LineTable,
_TokenTypeLowerName[47:56]: LineTable,
_TokenTypeName[56:69]: LineHighlight,
_TokenTypeLowerName[56:69]: LineHighlight,
_TokenTypeName[69:85]: LineNumbersTable,
_TokenTypeLowerName[69:85]: LineNumbersTable,
_TokenTypeName[85:96]: LineNumbers,
_TokenTypeLowerName[85:96]: LineNumbers,
_TokenTypeName[96:100]: Line,
_TokenTypeLowerName[96:100]: Line,
_TokenTypeName[100:110]: PreWrapper,
_TokenTypeLowerName[100:110]: PreWrapper,
_TokenTypeName[110:120]: Background,
_TokenTypeLowerName[110:120]: Background,
_TokenTypeName[120:127]: EOFType,
_TokenTypeLowerName[120:127]: EOFType,
_TokenTypeName[127:134]: Keyword,
_TokenTypeLowerName[127:134]: Keyword,
_TokenTypeName[134:149]: KeywordConstant,
_TokenTypeLowerName[134:149]: KeywordConstant,
_TokenTypeName[149:167]: KeywordDeclaration,
_TokenTypeLowerName[149:167]: KeywordDeclaration,
_TokenTypeName[167:183]: KeywordNamespace,
_TokenTypeLowerName[167:183]: KeywordNamespace,
_TokenTypeName[183:196]: KeywordPseudo,
_TokenTypeLowerName[183:196]: KeywordPseudo,
_TokenTypeName[196:211]: KeywordReserved,
_TokenTypeLowerName[196:211]: KeywordReserved,
_TokenTypeName[211:222]: KeywordType,
_TokenTypeLowerName[211:222]: KeywordType,
_TokenTypeName[222:226]: Name,
_TokenTypeLowerName[222:226]: Name,
_TokenTypeName[226:239]: NameAttribute,
_TokenTypeLowerName[226:239]: NameAttribute,
_TokenTypeName[239:250]: NameBuiltin,
_TokenTypeLowerName[239:250]: NameBuiltin,
_TokenTypeName[250:267]: NameBuiltinPseudo,
_TokenTypeLowerName[250:267]: NameBuiltinPseudo,
_TokenTypeName[267:276]: NameClass,
_TokenTypeLowerName[267:276]: NameClass,
_TokenTypeName[276:288]: NameConstant,
_TokenTypeLowerName[276:288]: NameConstant,
_TokenTypeName[288:301]: NameDecorator,
_TokenTypeLowerName[288:301]: NameDecorator,
_TokenTypeName[301:311]: NameEntity,
_TokenTypeLowerName[301:311]: NameEntity,
_TokenTypeName[311:324]: NameException,
_TokenTypeLowerName[311:324]: NameException,
_TokenTypeName[324:336]: NameFunction,
_TokenTypeLowerName[324:336]: NameFunction,
_TokenTypeName[336:353]: NameFunctionMagic,
_TokenTypeLowerName[336:353]: NameFunctionMagic,
_TokenTypeName[353:364]: NameKeyword,
_TokenTypeLowerName[353:364]: NameKeyword,
_TokenTypeName[364:373]: NameLabel,
_TokenTypeLowerName[364:373]: NameLabel,
_TokenTypeName[373:386]: NameNamespace,
_TokenTypeLowerName[373:386]: NameNamespace,
_TokenTypeName[386:398]: NameOperator,
_TokenTypeLowerName[386:398]: NameOperator,
_TokenTypeName[398:407]: NameOther,
_TokenTypeLowerName[398:407]: NameOther,
_TokenTypeName[407:417]: NamePseudo,
_TokenTypeLowerName[407:417]: NamePseudo,
_TokenTypeName[417:429]: NameProperty,
_TokenTypeLowerName[417:429]: NameProperty,
_TokenTypeName[429:436]: NameTag,
_TokenTypeLowerName[429:436]: NameTag,
_TokenTypeName[436:448]: NameVariable,
_TokenTypeLowerName[436:448]: NameVariable,
_TokenTypeName[448:469]: NameVariableAnonymous,
_TokenTypeLowerName[448:469]: NameVariableAnonymous,
_TokenTypeName[469:486]: NameVariableClass,
_TokenTypeLowerName[469:486]: NameVariableClass,
_TokenTypeName[486:504]: NameVariableGlobal,
_TokenTypeLowerName[486:504]: NameVariableGlobal,
_TokenTypeName[504:524]: NameVariableInstance,
_TokenTypeLowerName[504:524]: NameVariableInstance,
_TokenTypeName[524:541]: NameVariableMagic,
_TokenTypeLowerName[524:541]: NameVariableMagic,
_TokenTypeName[541:548]: Literal,
_TokenTypeLowerName[541:548]: Literal,
_TokenTypeName[548:559]: LiteralDate,
_TokenTypeLowerName[548:559]: LiteralDate,
_TokenTypeName[559:571]: LiteralOther,
_TokenTypeLowerName[559:571]: LiteralOther,
_TokenTypeName[571:584]: LiteralString,
_TokenTypeLowerName[571:584]: LiteralString,
_TokenTypeName[584:602]: LiteralStringAffix,
_TokenTypeLowerName[584:602]: LiteralStringAffix,
_TokenTypeName[602:619]: LiteralStringAtom,
_TokenTypeLowerName[602:619]: LiteralStringAtom,
_TokenTypeName[619:640]: LiteralStringBacktick,
_TokenTypeLowerName[619:640]: LiteralStringBacktick,
_TokenTypeName[640:660]: LiteralStringBoolean,
_TokenTypeLowerName[640:660]: LiteralStringBoolean,
_TokenTypeName[660:677]: LiteralStringChar,
_TokenTypeLowerName[660:677]: LiteralStringChar,
_TokenTypeName[677:699]: LiteralStringDelimiter,
_TokenTypeLowerName[677:699]: LiteralStringDelimiter,
_TokenTypeName[699:715]: LiteralStringDoc,
_TokenTypeLowerName[699:715]: LiteralStringDoc,
_TokenTypeName[715:734]: LiteralStringDouble,
_TokenTypeLowerName[715:734]: LiteralStringDouble,
_TokenTypeName[734:753]: LiteralStringEscape,
_TokenTypeLowerName[734:753]: LiteralStringEscape,
_TokenTypeName[753:773]: LiteralStringHeredoc,
_TokenTypeLowerName[753:773]: LiteralStringHeredoc,
_TokenTypeName[773:794]: LiteralStringInterpol,
_TokenTypeLowerName[773:794]: LiteralStringInterpol,
_TokenTypeName[794:811]: LiteralStringName,
_TokenTypeLowerName[794:811]: LiteralStringName,
_TokenTypeName[811:829]: LiteralStringOther,
_TokenTypeLowerName[811:829]: LiteralStringOther,
_TokenTypeName[829:847]: LiteralStringRegex,
_TokenTypeLowerName[829:847]: LiteralStringRegex,
_TokenTypeName[847:866]: LiteralStringSingle,
_TokenTypeLowerName[847:866]: LiteralStringSingle,
_TokenTypeName[866:885]: LiteralStringSymbol,
_TokenTypeLowerName[866:885]: LiteralStringSymbol,
_TokenTypeName[885:898]: LiteralNumber,
_TokenTypeLowerName[885:898]: LiteralNumber,
_TokenTypeName[898:914]: LiteralNumberBin,
_TokenTypeLowerName[898:914]: LiteralNumberBin,
_TokenTypeName[914:932]: LiteralNumberFloat,
_TokenTypeLowerName[914:932]: LiteralNumberFloat,
_TokenTypeName[932:948]: LiteralNumberHex,
_TokenTypeLowerName[932:948]: LiteralNumberHex,
_TokenTypeName[948:968]: LiteralNumberInteger,
_TokenTypeLowerName[948:968]: LiteralNumberInteger,
_TokenTypeName[968:992]: LiteralNumberIntegerLong,
_TokenTypeLowerName[968:992]: LiteralNumberIntegerLong,
_TokenTypeName[992:1008]: LiteralNumberOct,
_TokenTypeLowerName[992:1008]: LiteralNumberOct,
_TokenTypeName[1008:1025]: LiteralNumberByte,
_TokenTypeLowerName[1008:1025]: LiteralNumberByte,
_TokenTypeName[1025:1033]: Operator,
_TokenTypeLowerName[1025:1033]: Operator,
_TokenTypeName[1033:1045]: OperatorWord,
_TokenTypeLowerName[1033:1045]: OperatorWord,
_TokenTypeName[1045:1056]: Punctuation,
_TokenTypeLowerName[1045:1056]: Punctuation,
_TokenTypeName[1056:1063]: Comment,
_TokenTypeLowerName[1056:1063]: Comment,
_TokenTypeName[1063:1078]: CommentHashbang,
_TokenTypeLowerName[1063:1078]: CommentHashbang,
_TokenTypeName[1078:1094]: CommentMultiline,
_TokenTypeLowerName[1078:1094]: CommentMultiline,
_TokenTypeName[1094:1107]: CommentSingle,
_TokenTypeLowerName[1094:1107]: CommentSingle,
_TokenTypeName[1107:1121]: CommentSpecial,
_TokenTypeLowerName[1107:1121]: CommentSpecial,
_TokenTypeName[1121:1135]: CommentPreproc,
_TokenTypeLowerName[1121:1135]: CommentPreproc,
_TokenTypeName[1135:1153]: CommentPreprocFile,
_TokenTypeLowerName[1135:1153]: CommentPreprocFile,
_TokenTypeName[1153:1160]: Generic,
_TokenTypeLowerName[1153:1160]: Generic,
_TokenTypeName[1160:1174]: GenericDeleted,
_TokenTypeLowerName[1160:1174]: GenericDeleted,
_TokenTypeName[1174:1185]: GenericEmph,
_TokenTypeLowerName[1174:1185]: GenericEmph,
_TokenTypeName[1185:1197]: GenericError,
_TokenTypeLowerName[1185:1197]: GenericError,
_TokenTypeName[1197:1211]: GenericHeading,
_TokenTypeLowerName[1197:1211]: GenericHeading,
_TokenTypeName[1211:1226]: GenericInserted,
_TokenTypeLowerName[1211:1226]: GenericInserted,
_TokenTypeName[1226:1239]: GenericOutput,
_TokenTypeLowerName[1226:1239]: GenericOutput,
_TokenTypeName[1239:1252]: GenericPrompt,
_TokenTypeLowerName[1239:1252]: GenericPrompt,
_TokenTypeName[1252:1265]: GenericStrong,
_TokenTypeLowerName[1252:1265]: GenericStrong,
_TokenTypeName[1265:1282]: GenericSubheading,
_TokenTypeLowerName[1265:1282]: GenericSubheading,
_TokenTypeName[1282:1298]: GenericTraceback,
_TokenTypeLowerName[1282:1298]: GenericTraceback,
_TokenTypeName[1298:1314]: GenericUnderline,
_TokenTypeLowerName[1298:1314]: GenericUnderline,
_TokenTypeName[1314:1318]: Text,
_TokenTypeLowerName[1314:1318]: Text,
_TokenTypeName[1318:1332]: TextWhitespace,
_TokenTypeLowerName[1318:1332]: TextWhitespace,
_TokenTypeName[1332:1342]: TextSymbol,
_TokenTypeLowerName[1332:1342]: TextSymbol,
_TokenTypeName[1342:1357]: TextPunctuation,
_TokenTypeLowerName[1342:1357]: TextPunctuation,
}
var _TokenTypeNames = []string{
_TokenTypeName[0:4],
_TokenTypeName[4:9],
_TokenTypeName[9:14],
_TokenTypeName[14:22],
_TokenTypeName[22:30],
_TokenTypeName[30:41],
_TokenTypeName[41:50],
_TokenTypeName[50:63],
_TokenTypeName[63:79],
_TokenTypeName[79:90],
_TokenTypeName[90:94],
_TokenTypeName[94:104],
_TokenTypeName[104:114],
_TokenTypeName[114:121],
_TokenTypeName[121:128],
_TokenTypeName[128:143],
_TokenTypeName[143:161],
_TokenTypeName[161:177],
_TokenTypeName[177:190],
_TokenTypeName[190:205],
_TokenTypeName[205:216],
_TokenTypeName[216:220],
_TokenTypeName[220:233],
_TokenTypeName[233:244],
_TokenTypeName[244:261],
_TokenTypeName[261:270],
_TokenTypeName[270:282],
_TokenTypeName[282:295],
_TokenTypeName[295:305],
_TokenTypeName[305:318],
_TokenTypeName[318:330],
_TokenTypeName[330:347],
_TokenTypeName[347:358],
_TokenTypeName[358:367],
_TokenTypeName[367:380],
_TokenTypeName[380:392],
_TokenTypeName[392:401],
_TokenTypeName[401:411],
_TokenTypeName[411:423],
_TokenTypeName[423:430],
_TokenTypeName[430:442],
_TokenTypeName[442:463],
_TokenTypeName[463:480],
_TokenTypeName[480:498],
_TokenTypeName[498:518],
_TokenTypeName[518:535],
_TokenTypeName[535:542],
_TokenTypeName[542:553],
_TokenTypeName[553:565],
_TokenTypeName[565:578],
_TokenTypeName[578:596],
_TokenTypeName[596:613],
_TokenTypeName[613:634],
_TokenTypeName[634:654],
_TokenTypeName[654:671],
_TokenTypeName[671:693],
_TokenTypeName[693:709],
_TokenTypeName[709:728],
_TokenTypeName[728:747],
_TokenTypeName[747:767],
_TokenTypeName[767:788],
_TokenTypeName[788:805],
_TokenTypeName[805:823],
_TokenTypeName[823:841],
_TokenTypeName[841:860],
_TokenTypeName[860:879],
_TokenTypeName[879:892],
_TokenTypeName[892:908],
_TokenTypeName[908:926],
_TokenTypeName[926:942],
_TokenTypeName[942:962],
_TokenTypeName[962:986],
_TokenTypeName[986:1002],
_TokenTypeName[1002:1010],
_TokenTypeName[1010:1022],
_TokenTypeName[1022:1033],
_TokenTypeName[1033:1040],
_TokenTypeName[1040:1055],
_TokenTypeName[1055:1071],
_TokenTypeName[1071:1084],
_TokenTypeName[1084:1098],
_TokenTypeName[1098:1112],
_TokenTypeName[1112:1130],
_TokenTypeName[1130:1137],
_TokenTypeName[1137:1151],
_TokenTypeName[1151:1162],
_TokenTypeName[1162:1174],
_TokenTypeName[1174:1188],
_TokenTypeName[1188:1203],
_TokenTypeName[1203:1216],
_TokenTypeName[1216:1229],
_TokenTypeName[1229:1242],
_TokenTypeName[1242:1259],
_TokenTypeName[1259:1275],
_TokenTypeName[1275:1291],
_TokenTypeName[1291:1295],
_TokenTypeName[1295:1309],
_TokenTypeName[1309:1319],
_TokenTypeName[1319:1334],
_TokenTypeName[0:6],
_TokenTypeName[6:10],
_TokenTypeName[10:15],
_TokenTypeName[15:20],
_TokenTypeName[20:28],
_TokenTypeName[28:36],
_TokenTypeName[36:47],
_TokenTypeName[47:56],
_TokenTypeName[56:69],
_TokenTypeName[69:85],
_TokenTypeName[85:96],
_TokenTypeName[96:100],
_TokenTypeName[100:110],
_TokenTypeName[110:120],
_TokenTypeName[120:127],
_TokenTypeName[127:134],
_TokenTypeName[134:149],
_TokenTypeName[149:167],
_TokenTypeName[167:183],
_TokenTypeName[183:196],
_TokenTypeName[196:211],
_TokenTypeName[211:222],
_TokenTypeName[222:226],
_TokenTypeName[226:239],
_TokenTypeName[239:250],
_TokenTypeName[250:267],
_TokenTypeName[267:276],
_TokenTypeName[276:288],
_TokenTypeName[288:301],
_TokenTypeName[301:311],
_TokenTypeName[311:324],
_TokenTypeName[324:336],
_TokenTypeName[336:353],
_TokenTypeName[353:364],
_TokenTypeName[364:373],
_TokenTypeName[373:386],
_TokenTypeName[386:398],
_TokenTypeName[398:407],
_TokenTypeName[407:417],
_TokenTypeName[417:429],
_TokenTypeName[429:436],
_TokenTypeName[436:448],
_TokenTypeName[448:469],
_TokenTypeName[469:486],
_TokenTypeName[486:504],
_TokenTypeName[504:524],
_TokenTypeName[524:541],
_TokenTypeName[541:548],
_TokenTypeName[548:559],
_TokenTypeName[559:571],
_TokenTypeName[571:584],
_TokenTypeName[584:602],
_TokenTypeName[602:619],
_TokenTypeName[619:640],
_TokenTypeName[640:660],
_TokenTypeName[660:677],
_TokenTypeName[677:699],
_TokenTypeName[699:715],
_TokenTypeName[715:734],
_TokenTypeName[734:753],
_TokenTypeName[753:773],
_TokenTypeName[773:794],
_TokenTypeName[794:811],
_TokenTypeName[811:829],
_TokenTypeName[829:847],
_TokenTypeName[847:866],
_TokenTypeName[866:885],
_TokenTypeName[885:898],
_TokenTypeName[898:914],
_TokenTypeName[914:932],
_TokenTypeName[932:948],
_TokenTypeName[948:968],
_TokenTypeName[968:992],
_TokenTypeName[992:1008],
_TokenTypeName[1008:1025],
_TokenTypeName[1025:1033],
_TokenTypeName[1033:1045],
_TokenTypeName[1045:1056],
_TokenTypeName[1056:1063],
_TokenTypeName[1063:1078],
_TokenTypeName[1078:1094],
_TokenTypeName[1094:1107],
_TokenTypeName[1107:1121],
_TokenTypeName[1121:1135],
_TokenTypeName[1135:1153],
_TokenTypeName[1153:1160],
_TokenTypeName[1160:1174],
_TokenTypeName[1174:1185],
_TokenTypeName[1185:1197],
_TokenTypeName[1197:1211],
_TokenTypeName[1211:1226],
_TokenTypeName[1226:1239],
_TokenTypeName[1239:1252],
_TokenTypeName[1252:1265],
_TokenTypeName[1265:1282],
_TokenTypeName[1282:1298],
_TokenTypeName[1298:1314],
_TokenTypeName[1314:1318],
_TokenTypeName[1318:1332],
_TokenTypeName[1332:1342],
_TokenTypeName[1342:1357],
}
// TokenTypeString retrieves an enum value from the enum constants string name.
+3
View File
@@ -41,6 +41,8 @@ const (
Other
// No highlighting.
None
// Don't emit this token to the output.
Ignore
// Used as an EOF marker / nil token
EOFType TokenType = 0
)
@@ -122,6 +124,7 @@ const (
LiteralNumberInteger
LiteralNumberIntegerLong
LiteralNumberOct
LiteralNumberByte
)
// Operators.
+3
View File
@@ -0,0 +1,3 @@
/debug
/.vscode
/.idea
+33
View File
@@ -0,0 +1,33 @@
linters:
enable:
- revive
- govet
- unconvert
- megacheck
- gas
- gocyclo
- dupl
- misspell
- unparam
- unused
- typecheck
- ineffassign
- stylecheck
- gochecknoinits
- exportloopref
- gocritic
- nakedret
- gosimple
- prealloc
fast: false
disable-all: true
issues:
exclude-rules:
- path: _test\.go
linters:
- dupl
- text: "Errors unhandled"
linters:
- gosec
exclude-use-default: false
+21
View File
@@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2015 Didip Kerabat
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
+217
View File
@@ -0,0 +1,217 @@
[![GoDoc](https://godoc.org/github.com/didip/tollbooth?status.svg)](http://godoc.org/github.com/didip/tollbooth)
[![license](http://img.shields.io/badge/license-MIT-red.svg?style=flat)](https://raw.githubusercontent.com/didip/tollbooth/master/LICENSE)
## Tollbooth
This is a generic middleware to rate-limit HTTP requests.
**NOTE 1:** This library is considered finished.
**NOTE 2:** Major version changes are backward-incompatible. `v2.0.0` streamlines the ugliness of the old API.
## Versions
**v1.0.0:** This version maintains the old API but all the thirdparty modules are moved to their own repo.
**v2.x.x:** Brand-new API for the sake of code cleanup, thread safety, & auto-expiring data structures.
**v3.x.x:** Apparently we have been using golang.org/x/time/rate incorrectly. See issue #48. It always limits X number per 1 second. The time duration is not changeable, so it does not make sense to pass TTL to tollbooth.
**v4.x.x:** Float64 for max requests per second
**v5.x.x:** go.mod and go.sum
**v6.x.x:** Replaced `go-cache` with `github.com/go-pkgz/expirable-cache` because `go-cache` leaks goroutines.
**v7.x.x:** Replaced `time/rate` with `embedded time/rate` so that we can support more rate limit headers.
**v8.x.x:** Address `RemoteIP` vulnerability concern by replacing `SetIPLookups` with `SetIPLookup`, an explicit way to pick the IP address.
## Five Minute Tutorial
```go
package main
import (
"net/http"
"github.com/didip/tollbooth/v8"
"github.com/didip/tollbooth/v8/limiter"
)
func HelloHandler(w http.ResponseWriter, req *http.Request) {
w.Write([]byte("Hello, World!"))
}
func main() {
// Create a request limiter per handler.
lmt := tollbooth.NewLimiter(1, nil)
// New in version >= 8, you must explicitly define how to pick the IP address.
lmt.SetIPLookup(limiter.IPLookup{
Name: "X-Real-IP",
IndexFromRight: 0,
})
http.Handle("/", tollbooth.LimitFuncHandler(lmt, HelloHandler))
http.ListenAndServe(":12345", nil)
}
```
## Features
1. Rate-limit by request's remote IP, path, methods, custom headers, & basic auth usernames.
```go
import (
"time"
"github.com/didip/tollbooth/v8"
"github.com/didip/tollbooth/v8/limiter"
)
lmt := tollbooth.NewLimiter(1, nil)
// or create a limiter with expirable token buckets
// This setting means:
// create a 1 request/second limiter and
// every token bucket in it will expire 1 hour after it was initially set.
lmt = tollbooth.NewLimiter(1, &limiter.ExpirableOptions{DefaultExpirationTTL: time.Hour})
// New in version >= 8, you must explicitly define how to pick the IP address.
// If IP address cannot be found, rate limiter will not be activated.
lmt.SetIPLookup(limiter.IPLookup{
// The name of lookup method.
// Possible options are: RemoteAddr, X-Forwarded-For, X-Real-IP, CF-Connecting-IP
// All other headers are considered unknown and will be ignored.
Name: "X-Real-IP",
// The index position to pick the ip address from a comma separated list.
// The index goes from right to left.
//
// When there are multiple of the same headers,
// we will concat them together in the order of first to last seen.
// And then we pick the IP using this index position.
IndexFromRight: 0,
})
// In version >= 8, lmt.SetIPLookups and lmt.GetIPLookups are removed.
// Limit only GET and POST requests.
lmt.SetMethods([]string{"GET", "POST"})
// Limit based on basic auth usernames.
// You add them on-load, or later as you handle requests.
lmt.SetBasicAuthUsers([]string{"bob", "jane", "didip", "vip"})
// You can remove them later as well.
lmt.RemoveBasicAuthUsers([]string{"vip"})
// Limit request headers containing certain values.
// You add them on-load, or later as you handle requests.
lmt.SetHeader("X-Access-Token", []string{"abc123", "xyz098"})
// You can remove all entries at once.
lmt.RemoveHeader("X-Access-Token")
// Or remove specific ones.
lmt.RemoveHeaderEntries("X-Access-Token", []string{"limitless-token"})
// By the way, the setters are chainable. Example:
lmt.SetMethods([]string{"GET", "POST"}).
SetBasicAuthUsers([]string{"sansa"}).
SetBasicAuthUsers([]string{"tyrion"})
```
2. Compose your own middleware by using `LimitByKeys()`.
3. Header entries and basic auth users can expire over time (to conserve memory).
```go
import "time"
lmt := tollbooth.NewLimiter(1, nil)
// Set a custom expiration TTL for token bucket.
lmt.SetTokenBucketExpirationTTL(time.Hour)
// Set a custom expiration TTL for basic auth users.
lmt.SetBasicAuthExpirationTTL(time.Hour)
// Set a custom expiration TTL for header entries.
lmt.SetHeaderEntryExpirationTTL(time.Hour)
```
4. Upon rejection, the following HTTP response headers are available to users:
* `X-Rate-Limit-Limit` The maximum request limit.
* `X-Rate-Limit-Duration` The rate-limiter duration.
* `X-Rate-Limit-Request-Forwarded-For` The rejected request `X-Forwarded-For`.
* `X-Rate-Limit-Request-Remote-Addr` The rejected request `RemoteAddr`.
Upon both success and rejection [RateLimit](https://datatracker.ietf.org/doc/html/draft-ietf-httpapi-ratelimit-headers) headers are sent:
* `RateLimit-Limit` The maximum request limit within the time window (1s).
* `RateLimit-Reset` The rate-limiter time window duration in seconds (always 1s).
* `RateLimit-Remaining` The remaining tokens.
5. Customize your own message or function when limit is reached.
```go
lmt := tollbooth.NewLimiter(1, nil)
// New in version >= 8, you must explicitly define how to pick the IP address.
lmt.SetIPLookup(limiter.IPLookup{
Name: "X-Forwarded-For",
IndexFromRight: 0,
})
// Set a custom message.
lmt.SetMessage("You have reached maximum request limit.")
// Set a custom content-type.
lmt.SetMessageContentType("text/plain; charset=utf-8")
// Set a custom function for rejection.
lmt.SetOnLimitReached(func(w http.ResponseWriter, r *http.Request) { fmt.Println("A request was rejected") })
```
6. Tollbooth does not require external storage since it uses an algorithm called [Token Bucket](http://en.wikipedia.org/wiki/Token_bucket) [(Go library: golang.org/x/time/rate)](https://godoc.org/golang.org/x/time/rate).
## Other Web Frameworks
Sometimes, other frameworks require a little bit of shim to use Tollbooth. These shims below are contributed by the community, so I make no promises on how well they work. The one I am familiar with are: Chi, Gin, and Negroni.
* [Chi](https://github.com/didip/tollbooth_chi)
* [Echo](https://github.com/didip/tollbooth_echo)
* [FastHTTP](https://github.com/didip/tollbooth_fasthttp)
* [Gin](https://github.com/didip/tollbooth_gin)
* [GoRestful](https://github.com/didip/tollbooth_gorestful)
* [HTTPRouter](https://github.com/didip/tollbooth_httprouter)
* [Iris](https://github.com/didip/tollbooth_iris)
* [Negroni](https://github.com/didip/tollbooth_negroni)
## My other Go libraries
* [ErrStack](https://github.com/didip/errstack): A small library to combine errors and also display filename and line number.
* [Stopwatch](https://github.com/didip/stopwatch): A small library to measure latency of things. Useful if you want to report latency data to Graphite.
* [LaborUnion](https://github.com/didip/laborunion): A dynamic worker pool library.
* [Gomet](https://github.com/didip/gomet): Simple HTTP client & server long poll library for Go. Useful for receiving live updates without needing Websocket.
## Contributions
Before sending a PR with code changes, please make sure altered code is covered with tests which are passing, and that golangci-lint shows no errors.
To check the linter output, [install it](https://golangci-lint.run/usage/install/#local-installation) and then run `golangci-lint run` in the root directory of the repository.
+15
View File
@@ -0,0 +1,15 @@
// Package errors provide data structure for errors.
package errors
import "fmt"
// HTTPError is an error struct that returns both message and status code.
type HTTPError struct {
Message string
StatusCode int
}
// Error returns error message.
func (httperror *HTTPError) Error() string {
return fmt.Sprintf("%v: %v", httperror.StatusCode, httperror.Message)
}
+3
View File
@@ -0,0 +1,3 @@
# This source code refers to The Go Authors for copyright purposes.
# The master list of authors is in the main Go distribution,
# visible at http://tip.golang.org/AUTHORS.
@@ -0,0 +1,3 @@
# This source code was written by the Go contributors.
# The master list of contributors is in the main Go distribution,
# visible at http://tip.golang.org/CONTRIBUTORS.
+27
View File
@@ -0,0 +1,27 @@
Copyright (c) 2009 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+22
View File
@@ -0,0 +1,22 @@
Additional IP Rights Grant (Patents)
"This implementation" means the copyrightable works distributed by
Google as part of the Go project.
Google hereby grants to You a perpetual, worldwide, non-exclusive,
no-charge, royalty-free, irrevocable (except as stated in this section)
patent license to make, have made, use, offer to sell, sell, import,
transfer and otherwise run, modify and propagate the contents of this
implementation of Go, where such license applies only to those patent
claims, both currently owned or controlled by Google and acquired in
the future, licensable by Google that are necessarily infringed by this
implementation of Go. This grant does not include claims that would be
infringed only as a consequence of further modification of this
implementation. If you or your agent or exclusive licensee institute or
order or agree to the institution of patent litigation against any
entity (including a cross-claim or counterclaim in a lawsuit) alleging
that this implementation of Go or any code incorporated within this
implementation of Go constitutes direct or contributory patent
infringement, or inducement of patent infringement, then any patent
rights granted to you under this License for this implementation of Go
shall terminate as of the date such litigation is filed.
+396
View File
@@ -0,0 +1,396 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package rate provides a rate limiter.
package rate
import (
"context"
"fmt"
"math"
"sync"
"time"
)
// Limit defines the maximum frequency of some events.
// Limit is represented as number of events per second.
// A zero Limit allows no events.
type Limit float64
// Inf is the infinite rate limit; it allows all events (even if burst is zero).
const Inf = Limit(math.MaxFloat64)
// Every converts a minimum time interval between events to a Limit.
func Every(interval time.Duration) Limit {
if interval <= 0 {
return Inf
}
return 1 / Limit(interval.Seconds())
}
// A Limiter controls how frequently events are allowed to happen.
// It implements a "token bucket" of size b, initially full and refilled
// at rate r tokens per second.
// Informally, in any large enough time interval, the Limiter limits the
// rate to r tokens per second, with a maximum burst size of b events.
// As a special case, if r == Inf (the infinite rate), b is ignored.
// See https://en.wikipedia.org/wiki/Token_bucket for more about token buckets.
//
// The zero value is a valid Limiter, but it will reject all events.
// Use NewLimiter to create non-zero Limiters.
//
// Limiter has three main methods, Allow, Reserve, and Wait.
// Most callers should use Wait.
//
// Each of the three methods consumes a single token.
// They differ in their behavior when no token is available.
// If no token is available, Allow returns false.
// If no token is available, Reserve returns a reservation for a future token
// and the amount of time the caller must wait before using it.
// If no token is available, Wait blocks until one can be obtained
// or its associated context.Context is canceled.
//
// The methods AllowN, ReserveN, and WaitN consume n tokens.
type Limiter struct {
mu sync.Mutex
limit Limit
burst int
tokens float64
// last is the last time the limiter's tokens field was updated
last time.Time
// lastEvent is the latest time of a rate-limited event (past or future)
lastEvent time.Time
}
// Limit returns the maximum overall event rate.
func (lim *Limiter) Limit() Limit {
lim.mu.Lock()
defer lim.mu.Unlock()
return lim.limit
}
// Burst returns the maximum burst size. Burst is the maximum number of tokens
// that can be consumed in a single call to Allow, Reserve, or Wait, so higher
// Burst values allow more events to happen at once.
// A zero Burst allows no events, unless limit == Inf.
func (lim *Limiter) Burst() int {
lim.mu.Lock()
defer lim.mu.Unlock()
return lim.burst
}
// NewLimiter returns a new Limiter that allows events up to rate r and permits
// bursts of at most b tokens.
func NewLimiter(r Limit, b int) *Limiter {
return &Limiter{
limit: r,
burst: b,
}
}
// Allow is shorthand for AllowN(time.Now(), 1).
func (lim *Limiter) Allow() bool {
return lim.AllowN(time.Now(), 1)
}
// TokensAt returns the number of tokens available for the given time.
func (lim *Limiter) TokensAt(t time.Time) float64 {
lim.mu.Lock()
_, _, tokens := lim.advance(t) // does not mutate lim
lim.mu.Unlock()
return tokens
}
// AllowN reports whether n events may happen at time now.
// Use this method if you intend to drop / skip events that exceed the rate limit.
// Otherwise use Reserve or Wait.
func (lim *Limiter) AllowN(now time.Time, n int) bool {
return lim.reserveN(now, n, 0).ok
}
// A Reservation holds information about events that are permitted by a Limiter to happen after a delay.
// A Reservation may be canceled, which may enable the Limiter to permit additional events.
type Reservation struct {
ok bool
lim *Limiter
tokens int
timeToAct time.Time
// This is the Limit at reservation time, it can change later.
limit Limit
}
// OK returns whether the limiter can provide the requested number of tokens
// within the maximum wait time. If OK is false, Delay returns InfDuration, and
// Cancel does nothing.
func (r *Reservation) OK() bool {
return r.ok
}
// Delay is shorthand for DelayFrom(time.Now()).
func (r *Reservation) Delay() time.Duration {
return r.DelayFrom(time.Now())
}
// InfDuration is the duration returned by Delay when a Reservation is not OK.
const InfDuration = time.Duration(1<<63 - 1)
// DelayFrom returns the duration for which the reservation holder must wait
// before taking the reserved action. Zero duration means act immediately.
// InfDuration means the limiter cannot grant the tokens requested in this
// Reservation within the maximum wait time.
func (r *Reservation) DelayFrom(now time.Time) time.Duration {
if !r.ok {
return InfDuration
}
delay := r.timeToAct.Sub(now)
if delay < 0 {
return 0
}
return delay
}
// Cancel is shorthand for CancelAt(time.Now()).
func (r *Reservation) Cancel() {
r.CancelAt(time.Now())
}
// CancelAt indicates that the reservation holder will not perform the reserved action
// and reverses the effects of this Reservation on the rate limit as much as possible,
// considering that other reservations may have already been made.
func (r *Reservation) CancelAt(now time.Time) {
if !r.ok {
return
}
r.lim.mu.Lock()
defer r.lim.mu.Unlock()
if r.lim.limit == Inf || r.tokens == 0 || r.timeToAct.Before(now) {
return
}
// calculate tokens to restore
// The duration between lim.lastEvent and r.timeToAct tells us how many tokens were reserved
// after r was obtained. These tokens should not be restored.
restoreTokens := float64(r.tokens) - r.limit.tokensFromDuration(r.lim.lastEvent.Sub(r.timeToAct))
if restoreTokens <= 0 {
return
}
// advance time to now
now, _, tokens := r.lim.advance(now)
// calculate new number of tokens
tokens += restoreTokens
if burst := float64(r.lim.burst); tokens > burst {
tokens = burst
}
// update state
r.lim.last = now
r.lim.tokens = tokens
if r.timeToAct == r.lim.lastEvent {
prevEvent := r.timeToAct.Add(r.limit.durationFromTokens(float64(-r.tokens)))
if !prevEvent.Before(now) {
r.lim.lastEvent = prevEvent
}
}
}
// Reserve is shorthand for ReserveN(time.Now(), 1).
func (lim *Limiter) Reserve() *Reservation {
return lim.ReserveN(time.Now(), 1)
}
// ReserveN returns a Reservation that indicates how long the caller must wait before n events happen.
// The Limiter takes this Reservation into account when allowing future events.
// The returned Reservations OK() method returns false if n exceeds the Limiter's burst size.
// Usage example:
// r := lim.ReserveN(time.Now(), 1)
// if !r.OK() {
// // Not allowed to act! Did you remember to set lim.burst to be > 0 ?
// return
// }
// time.Sleep(r.Delay())
// Act()
// Use this method if you wish to wait and slow down in accordance with the rate limit without dropping events.
// If you need to respect a deadline or cancel the delay, use Wait instead.
// To drop or skip events exceeding rate limit, use Allow instead.
func (lim *Limiter) ReserveN(now time.Time, n int) *Reservation {
r := lim.reserveN(now, n, InfDuration)
return &r
}
// Wait is shorthand for WaitN(ctx, 1).
func (lim *Limiter) Wait(ctx context.Context) (err error) {
return lim.WaitN(ctx, 1)
}
// WaitN blocks until lim permits n events to happen.
// It returns an error if n exceeds the Limiter's burst size, the Context is
// canceled, or the expected wait time exceeds the Context's Deadline.
// The burst limit is ignored if the rate limit is Inf.
func (lim *Limiter) WaitN(ctx context.Context, n int) (err error) {
lim.mu.Lock()
burst := lim.burst
limit := lim.limit
lim.mu.Unlock()
if n > burst && limit != Inf {
return fmt.Errorf("rate: Wait(n=%d) exceeds limiter's burst %d", n, burst)
}
// Check if ctx is already cancelled
select {
case <-ctx.Done():
return ctx.Err()
default:
}
// Determine wait limit
now := time.Now()
waitLimit := InfDuration
if deadline, ok := ctx.Deadline(); ok {
waitLimit = deadline.Sub(now)
}
// Reserve
r := lim.reserveN(now, n, waitLimit)
if !r.ok {
return fmt.Errorf("rate: Wait(n=%d) would exceed context deadline", n)
}
// Wait if necessary
delay := r.DelayFrom(now)
if delay == 0 {
return nil
}
t := time.NewTimer(delay)
defer t.Stop()
select {
case <-t.C:
// We can proceed.
return nil
case <-ctx.Done():
// Context was canceled before we could proceed. Cancel the
// reservation, which may permit other events to proceed sooner.
r.Cancel()
return ctx.Err()
}
}
// SetLimit is shorthand for SetLimitAt(time.Now(), newLimit).
func (lim *Limiter) SetLimit(newLimit Limit) {
lim.SetLimitAt(time.Now(), newLimit)
}
// SetLimitAt sets a new Limit for the limiter. The new Limit, and Burst, may be violated
// or underutilized by those which reserved (using Reserve or Wait) but did not yet act
// before SetLimitAt was called.
func (lim *Limiter) SetLimitAt(now time.Time, newLimit Limit) {
lim.mu.Lock()
defer lim.mu.Unlock()
now, _, tokens := lim.advance(now)
lim.last = now
lim.tokens = tokens
lim.limit = newLimit
}
// SetBurst is shorthand for SetBurstAt(time.Now(), newBurst).
func (lim *Limiter) SetBurst(newBurst int) {
lim.SetBurstAt(time.Now(), newBurst)
}
// SetBurstAt sets a new burst size for the limiter.
func (lim *Limiter) SetBurstAt(now time.Time, newBurst int) {
lim.mu.Lock()
defer lim.mu.Unlock()
now, _, tokens := lim.advance(now)
lim.last = now
lim.tokens = tokens
lim.burst = newBurst
}
// reserveN is a helper method for AllowN, ReserveN, and WaitN.
// maxFutureReserve specifies the maximum reservation wait duration allowed.
// reserveN returns Reservation, not *Reservation, to avoid allocation in AllowN and WaitN.
func (lim *Limiter) reserveN(now time.Time, n int, maxFutureReserve time.Duration) Reservation {
lim.mu.Lock()
if lim.limit == Inf {
lim.mu.Unlock()
return Reservation{
ok: true,
lim: lim,
tokens: n,
timeToAct: now,
}
}
now, last, tokens := lim.advance(now)
// Calculate the remaining number of tokens resulting from the request.
tokens -= float64(n)
// Calculate the wait duration
var waitDuration time.Duration
if tokens < 0 {
waitDuration = lim.limit.durationFromTokens(-tokens)
}
// Decide result
ok := n <= lim.burst && waitDuration <= maxFutureReserve
// Prepare reservation
r := Reservation{
ok: ok,
lim: lim,
limit: lim.limit,
}
if ok {
r.tokens = n
r.timeToAct = now.Add(waitDuration)
}
// Update state
if ok {
lim.last = now
lim.tokens = tokens
lim.lastEvent = r.timeToAct
} else {
lim.last = last
}
lim.mu.Unlock()
return r
}
// advance calculates and returns an updated state for lim resulting from the passage of time.
// lim is not changed.
// advance requires that lim.mu is held.
func (lim *Limiter) advance(now time.Time) (newNow time.Time, newLast time.Time, newTokens float64) {
last := lim.last
if now.Before(last) {
last = now
}
// Calculate the new number of tokens, due to time that passed.
elapsed := now.Sub(last)
delta := lim.limit.tokensFromDuration(elapsed)
tokens := lim.tokens + delta
if burst := float64(lim.burst); tokens > burst {
tokens = burst
}
return now, last, tokens
}
// durationFromTokens is a unit conversion function from the number of tokens to the duration
// of time it takes to accumulate them at a rate of limit tokens per second.
func (limit Limit) durationFromTokens(tokens float64) time.Duration {
seconds := tokens / float64(limit)
return time.Duration(float64(time.Second) * seconds)
}
// tokensFromDuration is a unit conversion function from a time duration to the number of tokens
// which could be accumulated during that duration at a rate of limit tokens per second.
func (limit Limit) tokensFromDuration(d time.Duration) float64 {
return d.Seconds() * float64(limit)
}
+100
View File
@@ -0,0 +1,100 @@
// Package libstring provides various string related functions.
package libstring
import (
"net"
"net/http"
"strings"
"github.com/didip/tollbooth/v8/limiter"
)
// StringInSlice finds needle in a slice of strings.
func StringInSlice(sliceString []string, needle string) bool {
for _, b := range sliceString {
if b == needle {
return true
}
}
return false
}
// RemoteIPFromIPLookup picks an ip address explicitly from limiter.IPLookup criteria.
// This function is intended to replace RemoteIP function.
func RemoteIPFromIPLookup(ipLookup limiter.IPLookup, r *http.Request) string {
switch ipLookup.Name {
case "RemoteAddr":
// 1. Cover the basic use cases for both ipv4 and ipv6
ip, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
// 2. Upon error, just return the remote addr.
return r.RemoteAddr
}
return ip
case "X-Forwarded-For", "X-Real-IP", "CF-Connecting-IP":
ipAddrListCommaSeparated := r.Header.Values(ipLookup.Name)
ipAddrCommaSeparated := strings.Join(ipAddrListCommaSeparated, ",")
ips := strings.Split(ipAddrCommaSeparated, ",")
for i, p := range ips {
ips[i] = strings.TrimSpace(p)
}
ipIndex := len(ips) - 1 - ipLookup.IndexFromRight
if ipIndex < 0 {
ipIndex = 0
}
return ips[ipIndex]
}
return ""
}
// CanonicalizeIP returns a form of ip suitable for comparison to other IPs.
// For IPv4 addresses, this is simply the whole string.
// For IPv6 addresses, this is the /64 prefix.
func CanonicalizeIP(ip string) string {
isIPv6 := false
// This is how net.ParseIP decides if an address is IPv6
// https://cs.opensource.google/go/go/+/refs/tags/go1.17.7:src/net/ip.go;l=704
for i := 0; !isIPv6 && i < len(ip); i++ {
switch ip[i] {
case '.':
// IPv4
return ip
case ':':
// IPv6
isIPv6 = true
}
}
if !isIPv6 {
// Not an IP address at all
return ip
}
// By default, the string representation of a net.IPNet (masked IP address) is just
// "full_address/mask_bits". But using that will result in different addresses with
// the same /64 prefix comparing differently. So we need to zero out the last 64 bits
// so that all IPs in the same prefix will be the same.
//
// Note: When 1.18 is the minimum Go version, this can be written more cleanly like:
// netip.PrefixFrom(netip.MustParseAddr(ipv6), 64).Masked().Addr().String()
// (With appropriate error checking.)
ipv6 := net.ParseIP(ip)
if ipv6 == nil {
return ip
}
const bytesToZero = (128 - 64) / 8
for i := len(ipv6) - bytesToZero; i < len(ipv6); i++ {
ipv6[i] = 0
}
// Note that this doesn't have the "/64" suffix customary with a CIDR representation,
// but those three bytes add nothing for us.
return ipv6.String()
}
+624
View File
@@ -0,0 +1,624 @@
// Package limiter provides data structure to configure rate-limiter.
package limiter
import (
"net/http"
"sync"
"time"
cache "github.com/go-pkgz/expirable-cache/v3"
"github.com/didip/tollbooth/v8/internal/time/rate"
)
// New is a constructor for Limiter.
func New(generalExpirableOptions *ExpirableOptions) *Limiter {
lmt := &Limiter{}
lmt.SetMessageContentType("text/plain; charset=utf-8").
SetMessage("You have reached maximum request limit.").
SetStatusCode(429).
SetOnLimitReached(nil).
SetForwardedForIndexFromBehind(0).
SetHeaders(make(map[string][]string)).
SetContextValues(make(map[string][]string)).
SetIgnoreURL(false)
if generalExpirableOptions != nil {
lmt.generalExpirableOptions = generalExpirableOptions
} else {
lmt.generalExpirableOptions = &ExpirableOptions{}
}
// Default for DefaultExpirationTTL is 10 years.
if lmt.generalExpirableOptions.DefaultExpirationTTL <= 0 {
lmt.generalExpirableOptions.DefaultExpirationTTL = 87600 * time.Hour
}
lmt.tokenBuckets = cache.NewCache[string, *rate.Limiter]().WithTTL(lmt.generalExpirableOptions.DefaultExpirationTTL)
lmt.basicAuthUsers = cache.NewCache[string, bool]().WithTTL(lmt.generalExpirableOptions.DefaultExpirationTTL)
return lmt
}
// IPLookup is a config struct to define how users want to pick the remote IP address.
type IPLookup struct {
// The name of lookup method.
// Possible options are: RemoteAddr, X-Forwarded-For, X-Real-IP, CF-Connecting-IP
// All other headers are considered unknown and will be ignored.
Name string
// The index position to pick the ip address from a comma separated list.
// The index goes from right to left.
IndexFromRight int
}
// Limiter is a config struct to limit a particular request handler.
type Limiter struct {
// Maximum number of requests to limit per second.
max float64
// Limiter burst size
burst int
// HTTP message when limit is reached.
message string
// Content-Type for Message
messageContentType string
// HTTP status code when limit is reached.
statusCode int
// A function to call when a request is rejected.
onLimitReached func(w http.ResponseWriter, r *http.Request)
// An option to write back what you want upon reaching a limit.
overrideDefaultResponseWriter bool
// Explicitly define how to look up IP address.
// This is intended to replace ipLookups
explicitIPLookup IPLookup
forwardedForIndex int
// List of HTTP Methods to limit (GET, POST, PUT, etc.).
// Empty means limit all methods.
methods []string
// Able to configure token bucket expirations.
generalExpirableOptions *ExpirableOptions
// List of basic auth usernames to limit.
basicAuthUsers cache.Cache[string, bool]
// Map of HTTP headers to limit.
// Empty means skip headers checking.
headers map[string]cache.Cache[string, bool]
// Map of Context values to limit.
contextValues map[string]cache.Cache[string, bool]
// Map of limiters with TTL
tokenBuckets cache.Cache[string, *rate.Limiter]
// Ignore URL on the rate limiter keys
ignoreURL bool
tokenBucketExpirationTTL time.Duration
basicAuthExpirationTTL time.Duration
headerEntryExpirationTTL time.Duration
contextEntryExpirationTTL time.Duration
sync.RWMutex
}
// SetTokenBucketExpirationTTL is thread-safe way of setting custom token bucket expiration TTL.
func (l *Limiter) SetTokenBucketExpirationTTL(ttl time.Duration) *Limiter {
l.Lock()
l.tokenBucketExpirationTTL = ttl
l.Unlock()
return l
}
// GetTokenBucketExpirationTTL is thread-safe way of getting custom token bucket expiration TTL.
func (l *Limiter) GetTokenBucketExpirationTTL() time.Duration {
l.RLock()
defer l.RUnlock()
return l.tokenBucketExpirationTTL
}
// SetBasicAuthExpirationTTL is thread-safe way of setting custom basic auth expiration TTL.
func (l *Limiter) SetBasicAuthExpirationTTL(ttl time.Duration) *Limiter {
l.Lock()
l.basicAuthExpirationTTL = ttl
l.Unlock()
return l
}
// GetBasicAuthExpirationTTL is thread-safe way of getting custom basic auth expiration TTL.
func (l *Limiter) GetBasicAuthExpirationTTL() time.Duration {
l.RLock()
defer l.RUnlock()
return l.basicAuthExpirationTTL
}
// SetHeaderEntryExpirationTTL is thread-safe way of setting custom basic auth expiration TTL.
func (l *Limiter) SetHeaderEntryExpirationTTL(ttl time.Duration) *Limiter {
l.Lock()
l.headerEntryExpirationTTL = ttl
l.Unlock()
return l
}
// GetHeaderEntryExpirationTTL is thread-safe way of getting custom basic auth expiration TTL.
func (l *Limiter) GetHeaderEntryExpirationTTL() time.Duration {
l.RLock()
defer l.RUnlock()
return l.headerEntryExpirationTTL
}
// SetContextValueEntryExpirationTTL is thread-safe way of setting custom Context value expiration TTL.
func (l *Limiter) SetContextValueEntryExpirationTTL(ttl time.Duration) *Limiter {
l.Lock()
l.contextEntryExpirationTTL = ttl
l.Unlock()
return l
}
// GetContextValueEntryExpirationTTL is thread-safe way of getting custom Context value expiration TTL.
func (l *Limiter) GetContextValueEntryExpirationTTL() time.Duration {
l.RLock()
defer l.RUnlock()
return l.contextEntryExpirationTTL
}
// SetMax is thread-safe way of setting maximum number of requests to limit per second.
func (l *Limiter) SetMax(max float64) *Limiter {
l.Lock()
l.max = max
l.Unlock()
return l
}
// GetMax is thread-safe way of getting maximum number of requests to limit per second.
func (l *Limiter) GetMax() float64 {
l.RLock()
defer l.RUnlock()
return l.max
}
// SetBurst is thread-safe way of setting maximum burst size.
func (l *Limiter) SetBurst(burst int) *Limiter {
l.Lock()
l.burst = burst
l.Unlock()
return l
}
// GetBurst is thread-safe way of setting maximum burst size.
func (l *Limiter) GetBurst() int {
l.RLock()
defer l.RUnlock()
return l.burst
}
// SetMessage is thread-safe way of setting HTTP message when limit is reached.
func (l *Limiter) SetMessage(msg string) *Limiter {
l.Lock()
l.message = msg
l.Unlock()
return l
}
// GetMessage is thread-safe way of getting HTTP message when limit is reached.
func (l *Limiter) GetMessage() string {
l.RLock()
defer l.RUnlock()
return l.message
}
// SetMessageContentType is thread-safe way of setting HTTP message Content-Type when limit is reached.
func (l *Limiter) SetMessageContentType(contentType string) *Limiter {
l.Lock()
l.messageContentType = contentType
l.Unlock()
return l
}
// GetMessageContentType is thread-safe way of getting HTTP message Content-Type when limit is reached.
func (l *Limiter) GetMessageContentType() string {
l.RLock()
defer l.RUnlock()
return l.messageContentType
}
// SetStatusCode is thread-safe way of setting HTTP status code when limit is reached.
func (l *Limiter) SetStatusCode(statusCode int) *Limiter {
l.Lock()
l.statusCode = statusCode
l.Unlock()
return l
}
// GetStatusCode is thread-safe way of getting HTTP status code when limit is reached.
func (l *Limiter) GetStatusCode() int {
l.RLock()
defer l.RUnlock()
return l.statusCode
}
// SetOnLimitReached is thread-safe way of setting after-rejection function when limit is reached.
func (l *Limiter) SetOnLimitReached(fn func(w http.ResponseWriter, r *http.Request)) *Limiter {
l.Lock()
l.onLimitReached = fn
l.Unlock()
return l
}
// ExecOnLimitReached is thread-safe way of executing after-rejection function when limit is reached.
func (l *Limiter) ExecOnLimitReached(w http.ResponseWriter, r *http.Request) {
l.RLock()
fn := l.onLimitReached
l.RUnlock()
if fn != nil {
fn(w, r)
}
}
// SetOverrideDefaultResponseWriter is a thread-safe way of setting the response writer override variable.
func (l *Limiter) SetOverrideDefaultResponseWriter(override bool) *Limiter {
l.Lock()
l.overrideDefaultResponseWriter = override
l.Unlock()
return l
}
// GetOverrideDefaultResponseWriter is a thread-safe way of getting the response writer override variable.
func (l *Limiter) GetOverrideDefaultResponseWriter() bool {
l.RLock()
defer l.RUnlock()
return l.overrideDefaultResponseWriter
}
// SetIPLookup is thread-safe way of setting an explicit way to look up IP address.
// This method is intended to replace SetIPLookups (version 6 or older).
func (l *Limiter) SetIPLookup(lookup IPLookup) *Limiter {
l.Lock()
l.explicitIPLookup = lookup
l.Unlock()
return l
}
// GetIPLookup is thread-safe way of getting an explicit way to look up IP address.
// This method is intended to replace the old GetIPLookups (version 6 or older).
func (l *Limiter) GetIPLookup() IPLookup {
l.RLock()
defer l.RUnlock()
return l.explicitIPLookup
}
// SetIgnoreURL is thread-safe way of setting whenever ignore the URL on rate limit keys
func (l *Limiter) SetIgnoreURL(enabled bool) *Limiter {
l.Lock()
l.ignoreURL = enabled
l.Unlock()
return l
}
// GetIgnoreURL returns whether the URL is ignored in the rate limit key set
func (l *Limiter) GetIgnoreURL() bool {
l.RLock()
defer l.RUnlock()
return l.ignoreURL
}
// SetForwardedForIndexFromBehind is thread-safe way of setting which X-Forwarded-For index to choose.
func (l *Limiter) SetForwardedForIndexFromBehind(forwardedForIndex int) *Limiter {
l.Lock()
l.forwardedForIndex = forwardedForIndex
l.Unlock()
return l
}
// GetForwardedForIndexFromBehind is thread-safe way of getting which X-Forwarded-For index to choose.
func (l *Limiter) GetForwardedForIndexFromBehind() int {
l.RLock()
defer l.RUnlock()
return l.forwardedForIndex
}
// SetMethods is thread-safe way of setting list of HTTP Methods to limit (GET, POST, PUT, etc.).
func (l *Limiter) SetMethods(methods []string) *Limiter {
l.Lock()
l.methods = methods
l.Unlock()
return l
}
// GetMethods is thread-safe way of getting list of HTTP Methods to limit (GET, POST, PUT, etc.).
func (l *Limiter) GetMethods() []string {
l.RLock()
defer l.RUnlock()
return l.methods
}
// SetBasicAuthUsers is thread-safe way of setting list of basic auth usernames to limit.
func (l *Limiter) SetBasicAuthUsers(basicAuthUsers []string) *Limiter {
ttl := l.GetBasicAuthExpirationTTL()
if ttl <= 0 {
ttl = l.generalExpirableOptions.DefaultExpirationTTL
}
for _, basicAuthUser := range basicAuthUsers {
l.basicAuthUsers.Set(basicAuthUser, true, ttl)
}
return l
}
// GetBasicAuthUsers is thread-safe way of getting list of basic auth usernames to limit.
func (l *Limiter) GetBasicAuthUsers() []string {
return l.basicAuthUsers.Keys()
}
// RemoveBasicAuthUsers is thread-safe way of removing basic auth usernames from existing list.
func (l *Limiter) RemoveBasicAuthUsers(basicAuthUsers []string) *Limiter {
for _, toBeRemoved := range basicAuthUsers {
l.basicAuthUsers.Invalidate(toBeRemoved)
}
return l
}
// DeleteExpiredTokenBuckets is thread-safe way of deleting expired token buckets
func (l *Limiter) DeleteExpiredTokenBuckets() {
l.tokenBuckets.DeleteExpired()
}
// SetHeaders is thread-safe way of setting map of HTTP headers to limit.
func (l *Limiter) SetHeaders(headers map[string][]string) *Limiter {
if l.headers == nil {
l.headers = make(map[string]cache.Cache[string, bool])
}
for header, entries := range headers {
l.SetHeader(header, entries)
}
return l
}
// GetHeaders is thread-safe way of getting map of HTTP headers to limit.
func (l *Limiter) GetHeaders() map[string][]string {
results := make(map[string][]string)
l.RLock()
defer l.RUnlock()
for header, entriesAsGoCache := range l.headers {
results[header] = entriesAsGoCache.Keys()
}
return results
}
// SetHeader is thread-safe way of setting entries of 1 HTTP header.
func (l *Limiter) SetHeader(header string, entries []string) *Limiter {
l.RLock()
existing, found := l.headers[header]
l.RUnlock()
ttl := l.GetHeaderEntryExpirationTTL()
if ttl <= 0 {
ttl = l.generalExpirableOptions.DefaultExpirationTTL
}
if !found {
existing = cache.NewCache[string, bool]().WithTTL(ttl)
}
for _, entry := range entries {
existing.Set(entry, true, ttl)
}
l.Lock()
l.headers[header] = existing
l.Unlock()
return l
}
// GetHeader is thread-safe way of getting entries of 1 HTTP header.
func (l *Limiter) GetHeader(header string) []string {
l.RLock()
entriesAsGoCache := l.headers[header]
l.RUnlock()
return entriesAsGoCache.Keys()
}
// RemoveHeader is thread-safe way of removing entries of 1 HTTP header.
func (l *Limiter) RemoveHeader(header string) *Limiter {
ttl := l.GetHeaderEntryExpirationTTL()
if ttl <= 0 {
ttl = l.generalExpirableOptions.DefaultExpirationTTL
}
l.Lock()
l.headers[header] = cache.NewCache[string, bool]().WithTTL(ttl)
l.Unlock()
return l
}
// RemoveHeaderEntries is thread-safe way of removing new entries to 1 HTTP header rule.
func (l *Limiter) RemoveHeaderEntries(header string, entriesForRemoval []string) *Limiter {
l.RLock()
entries, found := l.headers[header]
l.RUnlock()
if !found {
return l
}
for _, toBeRemoved := range entriesForRemoval {
entries.Invalidate(toBeRemoved)
}
return l
}
// SetContextValues is thread-safe way of setting map of HTTP headers to limit.
func (l *Limiter) SetContextValues(contextValues map[string][]string) *Limiter {
if l.contextValues == nil {
l.contextValues = make(map[string]cache.Cache[string, bool])
}
for contextValue, entries := range contextValues {
l.SetContextValue(contextValue, entries)
}
return l
}
// GetContextValues is thread-safe way of getting a map of Context values to limit.
func (l *Limiter) GetContextValues() map[string][]string {
results := make(map[string][]string)
l.RLock()
defer l.RUnlock()
for contextValue, entriesAsGoCache := range l.contextValues {
results[contextValue] = entriesAsGoCache.Keys()
}
return results
}
// SetContextValue is thread-safe way of setting entries of 1 Context value.
func (l *Limiter) SetContextValue(contextValue string, entries []string) *Limiter {
l.RLock()
existing, found := l.contextValues[contextValue]
l.RUnlock()
ttl := l.GetContextValueEntryExpirationTTL()
if ttl <= 0 {
ttl = l.generalExpirableOptions.DefaultExpirationTTL
}
if !found {
existing = cache.NewCache[string, bool]().WithTTL(ttl)
}
for _, entry := range entries {
existing.Set(entry, true, ttl)
}
l.Lock()
l.contextValues[contextValue] = existing
l.Unlock()
return l
}
// GetContextValue is thread-safe way of getting 1 Context value entry.
func (l *Limiter) GetContextValue(contextValue string) []string {
l.RLock()
entriesAsGoCache := l.contextValues[contextValue]
l.RUnlock()
return entriesAsGoCache.Keys()
}
// RemoveContextValue is thread-safe way of removing entries of 1 Context value.
func (l *Limiter) RemoveContextValue(contextValue string) *Limiter {
ttl := l.GetContextValueEntryExpirationTTL()
if ttl <= 0 {
ttl = l.generalExpirableOptions.DefaultExpirationTTL
}
l.Lock()
l.contextValues[contextValue] = cache.NewCache[string, bool]().WithTTL(ttl)
l.Unlock()
return l
}
// RemoveContextValuesEntries is thread-safe way of removing entries to a ContextValue.
func (l *Limiter) RemoveContextValuesEntries(contextValue string, entriesForRemoval []string) *Limiter {
l.RLock()
entries, found := l.contextValues[contextValue]
l.RUnlock()
if !found {
return l
}
for _, toBeRemoved := range entriesForRemoval {
entries.Invalidate(toBeRemoved)
}
return l
}
func (l *Limiter) limitReachedWithTokenBucketTTL(key string, tokenBucketTTL time.Duration) bool {
lmtMax := l.GetMax()
lmtBurst := l.GetBurst()
l.Lock()
defer l.Unlock()
if _, found := l.tokenBuckets.Get(key); !found {
l.tokenBuckets.Set(
key,
rate.NewLimiter(rate.Limit(lmtMax), lmtBurst),
tokenBucketTTL,
)
}
expiringMap, found := l.tokenBuckets.Get(key)
if !found {
return false
}
return !expiringMap.Allow()
}
// LimitReached returns a bool indicating if the Bucket identified by key ran out of tokens.
func (l *Limiter) LimitReached(key string) bool {
ttl := l.GetTokenBucketExpirationTTL()
if ttl <= 0 {
ttl = l.generalExpirableOptions.DefaultExpirationTTL
}
return l.limitReachedWithTokenBucketTTL(key, ttl)
}
// Tokens returns current amount of tokens left in the Bucket identified by key.
func (l *Limiter) Tokens(key string) int {
expiringMap, found := l.tokenBuckets.Get(key)
if !found {
return 0
}
return int(expiringMap.TokensAt(time.Now()))
}
@@ -0,0 +1,14 @@
package limiter
import (
"time"
)
// ExpirableOptions are options used for new limiter creation
type ExpirableOptions struct {
DefaultExpirationTTL time.Duration
// How frequently expire job triggers
// Deprecated: not used anymore
ExpireJobInterval time.Duration
}
+375
View File
@@ -0,0 +1,375 @@
// Package tollbooth provides rate-limiting logic to HTTP request handler.
package tollbooth
import (
"fmt"
"math"
"net/http"
"strings"
"github.com/didip/tollbooth/v8/errors"
"github.com/didip/tollbooth/v8/libstring"
"github.com/didip/tollbooth/v8/limiter"
)
// setResponseHeaders configures X-Rate-Limit-Limit and X-Rate-Limit-Duration
func setResponseHeaders(lmt *limiter.Limiter, w http.ResponseWriter, r *http.Request) {
w.Header().Add("X-Rate-Limit-Limit", fmt.Sprintf("%.2f", lmt.GetMax()))
w.Header().Add("X-Rate-Limit-Duration", "1")
xForwardedFor := r.Header.Get("X-Forwarded-For")
if strings.TrimSpace(xForwardedFor) != "" {
w.Header().Add("X-Rate-Limit-Request-Forwarded-For", xForwardedFor)
}
w.Header().Add("X-Rate-Limit-Request-Remote-Addr", r.RemoteAddr)
}
// setRateLimitResponseHeaders configures RateLimit-Limit, RateLimit-Remaining and RateLimit-Reset
// as seen at https://datatracker.ietf.org/doc/html/draft-ietf-httpapi-ratelimit-headers
func setRateLimitResponseHeaders(lmt *limiter.Limiter, w http.ResponseWriter, tokensLeft int) {
w.Header().Add("RateLimit-Limit", fmt.Sprintf("%d", int(math.Round(lmt.GetMax()))))
w.Header().Add("RateLimit-Reset", "1")
w.Header().Add("RateLimit-Remaining", fmt.Sprintf("%d", tokensLeft))
}
// NewLimiter is a convenience function to limiter.New.
func NewLimiter(max float64, tbOptions *limiter.ExpirableOptions) *limiter.Limiter {
return limiter.New(tbOptions).
SetMax(max).
SetBurst(int(math.Max(1, max)))
}
// LimitByKeys keeps track number of request made by keys separated by pipe.
// It returns HTTPError when limit is exceeded.
func LimitByKeys(lmt *limiter.Limiter, keys []string) *errors.HTTPError {
err, _ := LimitByKeysAndReturn(lmt, keys)
return err
}
// LimitByKeysAndReturn keeps track number of request made by keys separated by pipe.
// It returns HTTPError when limit is exceeded, and also returns the current limit value.
func LimitByKeysAndReturn(lmt *limiter.Limiter, keys []string) (*errors.HTTPError, int) {
if lmt.LimitReached(strings.Join(keys, "|")) {
return &errors.HTTPError{Message: lmt.GetMessage(), StatusCode: lmt.GetStatusCode()}, 0
}
return nil, lmt.Tokens(strings.Join(keys, "|"))
}
// ShouldSkipLimiter is a series of filter that decides if request should be limited or not.
func ShouldSkipLimiter(lmt *limiter.Limiter, r *http.Request) bool {
// ---------------------------------
// Filter by remote ip
// If we are unable to find remoteIP, skip limiter
remoteIP := libstring.RemoteIPFromIPLookup(lmt.GetIPLookup(), r)
remoteIP = libstring.CanonicalizeIP(remoteIP)
if remoteIP == "" {
return true
}
// ---------------------------------
// Filter by request method
lmtMethods := lmt.GetMethods()
lmtMethodsIsSet := len(lmtMethods) > 0
if lmtMethodsIsSet {
// If request does not contain all of the methods in limiter,
// skip limiter
requestMethodDefinedInLimiter := libstring.StringInSlice(lmtMethods, r.Method)
if !requestMethodDefinedInLimiter {
return true
}
}
// ---------------------------------
// Filter by request headers
lmtHeaders := lmt.GetHeaders()
lmtHeadersIsSet := len(lmtHeaders) > 0
if lmtHeadersIsSet {
// If request does not contain all of the headers in limiter,
// skip limiter
requestHeadersDefinedInLimiter := false
for headerKey := range lmtHeaders {
reqHeaderValue := r.Header.Get(headerKey)
if reqHeaderValue != "" {
requestHeadersDefinedInLimiter = true
break
}
}
if !requestHeadersDefinedInLimiter {
return true
}
// ------------------------------
// If request contains the header key but not the values,
// skip limiter
requestHeadersDefinedInLimiter = false
for headerKey, headerValues := range lmtHeaders {
if len(headerValues) == 0 {
requestHeadersDefinedInLimiter = true
continue
}
for _, headerValue := range headerValues {
if r.Header.Get(headerKey) == headerValue {
requestHeadersDefinedInLimiter = true
break
}
}
}
if !requestHeadersDefinedInLimiter {
return true
}
}
// ---------------------------------
// Filter by context values
lmtContextValues := lmt.GetContextValues()
lmtContextValuesIsSet := len(lmtContextValues) > 0
if lmtContextValuesIsSet {
// If request does not contain all of the contexts in limiter,
// skip limiter
requestContextValuesDefinedInLimiter := false
for contextKey := range lmtContextValues {
reqContextValue := fmt.Sprintf("%v", r.Context().Value(contextKey))
if reqContextValue != "" {
requestContextValuesDefinedInLimiter = true
break
}
}
if !requestContextValuesDefinedInLimiter {
return true
}
// ------------------------------
// If request contains the context key but not the values,
// skip limiter
requestContextValuesDefinedInLimiter = false
for contextKey, contextValues := range lmtContextValues {
for _, contextValue := range contextValues {
if r.Header.Get(contextKey) == contextValue {
requestContextValuesDefinedInLimiter = true
break
}
}
}
if !requestContextValuesDefinedInLimiter {
return true
}
}
// ---------------------------------
// Filter by basic auth usernames
lmtBasicAuthUsers := lmt.GetBasicAuthUsers()
lmtBasicAuthUsersIsSet := len(lmtBasicAuthUsers) > 0
if lmtBasicAuthUsersIsSet {
// If request does not contain all of the basic auth users in limiter,
// skip limiter
requestAuthUsernameDefinedInLimiter := false
username, _, ok := r.BasicAuth()
if ok && libstring.StringInSlice(lmtBasicAuthUsers, username) {
requestAuthUsernameDefinedInLimiter = true
}
if !requestAuthUsernameDefinedInLimiter {
return true
}
}
return false
}
// BuildKeys generates a slice of keys to rate-limit by given limiter and request structs.
func BuildKeys(lmt *limiter.Limiter, r *http.Request) [][]string {
remoteIP := libstring.RemoteIPFromIPLookup(lmt.GetIPLookup(), r)
remoteIP = libstring.CanonicalizeIP(remoteIP)
path := r.URL.Path
sliceKeys := make([][]string, 0)
lmtMethods := lmt.GetMethods()
lmtHeaders := lmt.GetHeaders()
lmtContextValues := lmt.GetContextValues()
lmtBasicAuthUsers := lmt.GetBasicAuthUsers()
lmtIgnoreURL := lmt.GetIgnoreURL()
lmtHeadersIsSet := len(lmtHeaders) > 0
lmtContextValuesIsSet := len(lmtContextValues) > 0
lmtBasicAuthUsersIsSet := len(lmtBasicAuthUsers) > 0
usernameToLimit := ""
if lmtBasicAuthUsersIsSet {
username, _, ok := r.BasicAuth()
if ok && libstring.StringInSlice(lmtBasicAuthUsers, username) {
usernameToLimit = username
}
}
headerValuesToLimit := [][]string{}
if lmtHeadersIsSet {
for headerKey, headerValues := range lmtHeaders {
reqHeaderValue := r.Header.Get(headerKey)
if reqHeaderValue == "" {
continue
}
if len(headerValues) == 0 {
// If header values are empty, rate-limit all request containing headerKey.
headerValuesToLimit = append(headerValuesToLimit, []string{headerKey, reqHeaderValue})
} else {
// If header values are not empty, rate-limit all request with headerKey and headerValues.
for _, headerValue := range headerValues {
if r.Header.Get(headerKey) == headerValue {
headerValuesToLimit = append(headerValuesToLimit, []string{headerKey, headerValue})
break
}
}
}
}
}
contextValuesToLimit := [][]string{}
if lmtContextValuesIsSet {
for contextKey, contextValues := range lmtContextValues {
reqContextValue := fmt.Sprintf("%v", r.Context().Value(contextKey))
if reqContextValue == "" {
continue
}
if len(contextValues) == 0 {
// If context values are empty, rate-limit all request containing contextKey.
contextValuesToLimit = append(contextValuesToLimit, []string{contextKey, reqContextValue})
} else {
// If context values are not empty, rate-limit all request with contextKey and contextValues.
for _, contextValue := range contextValues {
if reqContextValue == contextValue {
contextValuesToLimit = append(contextValuesToLimit, []string{contextKey, contextValue})
break
}
}
}
}
}
sliceKey := []string{remoteIP}
if !lmtIgnoreURL {
sliceKey = append(sliceKey, path)
}
sliceKey = append(sliceKey, lmtMethods...)
for _, header := range headerValuesToLimit {
sliceKey = append(sliceKey, header[0], header[1])
}
for _, contextValue := range contextValuesToLimit {
sliceKey = append(sliceKey, contextValue[0], contextValue[1])
}
sliceKey = append(sliceKey, usernameToLimit)
sliceKeys = append(sliceKeys, sliceKey)
return sliceKeys
}
// LimitByRequest builds keys based on http.Request struct,
// loops through all the keys, and check if any one of them returns HTTPError.
func LimitByRequest(lmt *limiter.Limiter, w http.ResponseWriter, r *http.Request) *errors.HTTPError {
setResponseHeaders(lmt, w, r)
shouldSkip := ShouldSkipLimiter(lmt, r)
if shouldSkip {
return nil
}
sliceKeys := BuildKeys(lmt, r)
// Get the lowest value over all keys to return in headers.
// Start with high arbitrary number so that any limit returned would be lower and would
// overwrite the value we start with.
var tokensLeft = math.MaxInt32
// Loop sliceKeys and check if one of them has error.
for _, keys := range sliceKeys {
httpError, keysLimit := LimitByKeysAndReturn(lmt, keys)
if tokensLeft > keysLimit {
tokensLeft = keysLimit
}
if httpError != nil {
setRateLimitResponseHeaders(lmt, w, tokensLeft)
return httpError
}
}
setRateLimitResponseHeaders(lmt, w, tokensLeft)
return nil
}
// LimitHandler is a middleware that performs rate-limiting given http.Handler struct.
func LimitHandler(lmt *limiter.Limiter, next http.Handler) http.Handler {
middle := func(w http.ResponseWriter, r *http.Request) {
httpError := LimitByRequest(lmt, w, r)
if httpError != nil {
lmt.ExecOnLimitReached(w, r)
if lmt.GetOverrideDefaultResponseWriter() {
return
}
w.Header().Add("Content-Type", lmt.GetMessageContentType())
w.WriteHeader(httpError.StatusCode)
w.Write([]byte(httpError.Message))
return
}
// There's no rate-limit error, serve the next handler.
next.ServeHTTP(w, r)
}
return http.HandlerFunc(middle)
}
// LimitFuncHandler is a middleware that performs rate-limiting given request handler function.
func LimitFuncHandler(lmt *limiter.Limiter, nextFunc func(http.ResponseWriter, *http.Request)) http.Handler {
return LimitHandler(lmt, http.HandlerFunc(nextFunc))
}
// HTTPMiddleware wraps http.Handler with tollbooth limiter
func HTTPMiddleware(lmt *limiter.Limiter) func(http.Handler) http.Handler {
// // set IP lookup only if not set
if lmt.GetIPLookup().Name == "" {
lmt.SetIPLookup(limiter.IPLookup{Name: "RemoteAddr"})
}
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
select {
case <-r.Context().Done():
http.Error(w, "Context was canceled", http.StatusServiceUnavailable)
return
default:
if httpError := LimitByRequest(lmt, w, r); httpError != nil {
lmt.ExecOnLimitReached(w, r)
w.Header().Add("Content-Type", lmt.GetMessageContentType())
w.WriteHeader(httpError.StatusCode)
w.Write([]byte(httpError.Message)) //nolint:gosec // not much we can do here with failed write
return
}
next.ServeHTTP(w, r)
}
})
}
}
+55 -18
View File
@@ -1,8 +1,8 @@
linters-settings:
govet:
check-shadowing: true
shadow: true
golint:
min-confidence: 0
min-confidence: 0.6
gocyclo:
min-complexity: 15
maligned:
@@ -23,44 +23,81 @@ linters-settings:
- experimental
disabled-checks:
- wrapperFunc
- hugeParam
- rangeValCopy
linters:
disable-all: true
enable:
- megacheck
- revive
- govet
- unconvert
- megacheck
- structcheck
- gas
- gocyclo
- dupl
- misspell
- gosec
- unparam
- varcheck
- deadcode
- unused
- typecheck
- ineffassign
- varcheck
- stylecheck
- gochecknoinits
- exportloopref
- gocritic
- nakedret
- gosimple
- prealloc
fast: false
disable-all: true
run:
output:
format: tab
skip-dirs:
- vendor
concurrency: 4
issues:
exclude-dirs:
- vendor
exclude-rules:
- text: "should have a package comment, unless it's in another file for this package"
linters:
- golint
- text: "exitAfterDefer:"
linters:
- gocritic
- text: "whyNoLint: include an explanation for nolint directive"
linters:
- gocritic
- text: "go.mongodb.org/mongo-driver/bson/primitive.E"
linters:
- govet
- text: "weak cryptographic primitive"
linters:
- gosec
- text: "integer overflow conversion"
linters:
- gosec
- text: "should have a package comment"
linters:
- revive
- text: "at least one file in a package should have a package comment"
linters:
- stylecheck
- text: "commentedOutCode: may want to remove commented-out code"
linters:
- gocritic
- text: "unnamedResult: consider giving a name to these results"
linters:
- gocritic
- text: "var-naming: don't use an underscore in package name"
linters:
- revive
- text: "should not use underscores in package names"
linters:
- stylecheck
- text: "struct literal uses unkeyed fields"
linters:
- govet
- linters:
- unparam
- unused
- revive
path: _test\.go$
text: "unused-parameter"
exclude-use-default: false
+6 -4
View File
@@ -8,8 +8,8 @@ import (
"sync"
"time"
"github.com/didip/tollbooth/v7"
"github.com/didip/tollbooth_chi"
"github.com/didip/tollbooth/v8"
"github.com/didip/tollbooth/v8/limiter"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/go-chi/render"
@@ -116,7 +116,9 @@ func (s *Server) Run(port int) error {
router.Use(logInfoWithBody)
if s.limits.clientLimit > 0 {
router.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(s.limits.clientLimit, nil)))
lmt := tollbooth.NewLimiter(s.limits.clientLimit, nil)
lmt.SetIPLookup(limiter.IPLookup{Name: "X-Real-IP"})
router.Use(tollbooth.HTTPMiddleware(lmt))
}
router.Use(middleware.NoCache)
@@ -243,4 +245,4 @@ type LoggerFunc func(format string, args ...interface{})
func (f LoggerFunc) Logf(format string, args ...interface{}) { f(format, args...) }
// NoOpLogger logger does nothing
var NoOpLogger = LoggerFunc(func(format string, args ...interface{}) {})
var NoOpLogger = LoggerFunc(func(format string, args ...interface{}) {}) //nolint