Sensitive-lexicon provides developers with flexibility in algorithm selection. Depending on performance requirements, one can choose either the basic regular expression matching or the efficient DFA/Trie tree implementation. Regular expressions are suitable for scenarios with small text volumes, while DFA and Trie trees are more suitable for high concurrency production environments.
The choice of algorithm directly affects the filtering efficiency: regular expressions are simple and easy to implement, but the performance is low; DFA algorithm matching time complexity is directly proportional to the length of the text to be matched, independent of the size of the thesaurus; Trie tree is also efficient and supports prefix matching. Project documents provide algorithm guidance, but does not limit the specific implementation, the developer can be freely optimized according to business scenarios.
This answer comes from the articleSensitive-lexicon: a continuously updated thesaurus of Chinese sensitive wordsThe
English 
简体中文 
日本語 
Deutsch 
Português do Brasil