Package scone.util.tokenstream

This package provides means for convenient tokenizing of documents.

See:
Description

Interface Summary

Token	Tokens are pieces of a document.
Tokenizer	A Tokenizer tokenizes (parses) a Reader.

Class Summary

DummyTokenOutputStream	a dummy TokenOutputStream.
HtmlCommentToken	an HTML comment.
HtmlScriptToken	an HTML script element.
HtmlStyleToken	an HTML style element.
HtmlTagToken	an HTML tag.
HtmlTextToken	HTML text.
HtmlTokenizer	parses an HTML document into Tokens
SconePipe	helper class for the use of TokenStreams.
TokenBuffer	A Buffer to store Tokens.
TokenInputStream	an InputStream that works on Tokens.
TokenInputStreamBufferImpl	reads Tokens from the TokenBuffer a TokenOutputStream writes to.
TokenInputStreamTokenizerImpl	converts a Reader into a TokenInputStream.
TokenOutputStream	writes Tokens into a TokenBuffer

Exception Summary

TokenBufferIndexOutOfBoundsException

indicates a TokenBufferIndexOutOfBoundsException

Package scone.util.tokenstream Description

This package provides means for convenient tokenizing of documents.

TokenStreams enable Scone applications to easily analyze and modify documents. A TokenInputStream utilizes a Tokenizer to parse a document into Token objects. This package also contains implementations of Tokenizer and Tokens for HTML.