Package perl-String-Tokenizer: Information
Source package: perl-String-Tokenizer
Version: 0.06-alt1
Build time: Nov 19, 2018, 10:00 AM
Category: Development/Perl
Report package bugHome page: http://www.cpan.org
License: Artistic
Summary: A simple string tokenizer
Description:
A simple string tokenizer which takes a string and splits it on whitespace. It also optionally takes a string of characters to use as delimiters, and returns them with the token set as well. This allows for splitting the string in many different ways. This is a very basic tokenizer, so more complex needs should be either addressed with a custom written tokenizer or post-processing of the output generated by this module. Basically, this will not fill everyones needs, but it spans a gap between simple "split / /, $string" and the other options that involve much larger and complex modules. Also note that this is not a lexical analyser. Many people confuse tokenization with lexical analysis. A tokenizer mearly splits its input into specific chunks, a lexical analyzer classifies those chunks. Sometimes these two steps are combined, but not here.
Maintainer: Vitaly Lipatov
Last changed
March 3, 2016 Igor Vlasenko 0.06-alt1
- automated CPAN update
Nov. 22, 2010 Igor Vlasenko 0.05-alt2.1
- repair after perl 5.12 upgrade using girar-nmu
Sept. 6, 2008 Vitaly Lipatov 0.05-alt2
- fix directory ownership violation