tokenizer — independent software & tools

3 independent tokenizer listings the big search engines bury.

• chevrotain

  npm · Free · by GitHub Actions

  Chevrotain is a high performance fault tolerant javascript parsing DSL for building recursive decent parsers

• css-tree

  npm · Free · by lahmatiy

  A tool set for CSS: fast detailed parser (CSS → AST), walker (AST traversal), generator (AST → CSS) and lexer (validation and matching) based on specs and browser implementations

• tokenx

  npm · Free · by GitHub Actions

  Fast token estimation at 96% accuracy of a full tokenizer in a 2kB bundle