Wouter van Vliet - 2010-02-15 13:50:03
Hi!
This class looks interesting, but could you perhaps post some example code?
Also, to make clashes with other classes in a bigger system less likely - perhaps you can come up with some more unique classnames? Your package is called Tokenizer, so it would make sense to rename 'marker' to 'Tokenizer_Marker'.
Thanks!
Wouter