jfun.parsec
Interface Tokenizer

All Superinterfaces:
java.io.Serializable
All Known Implementing Classes:
ConstTokenizer, TokenString

public interface Tokenizer
extends java.io.Serializable

A tokenizer is used by a scanner object to interpret the output of a Scanner.

Author:
Ben Yu 2004-11-7

Method Summary
 java.lang.Object toToken(java.lang.CharSequence cs, int from, int len)
          interprets the matched input range to a token object.
 

Method Detail

toToken

java.lang.Object toToken(java.lang.CharSequence cs,
                         int from,
                         int len)
interprets the matched input range to a token object.

Parameters:
cs - - the input
from - - the starting point
len - - the length of the range
Returns:
- the token object.