dbrown
Class Tokenizer

java.lang.Object
  dbrown.Tokenizer

public class Tokenizer
extends java.lang.Object

Tokenizer takes an equation string as input and tokenizes it, breaking it up into integers (the operands) and single characters (the operators). Each call to nextOperand or nextOperator retrieves the next token as an integer or a string respectively and removes that token from the remaining string. When all tokens have been removed the string is empty. The initial String is assigned by passing it to the Tokenizer constructor.

This methods of this class follow the approach of similar methods in the Scanner class. Thus nextOperand throws the same exceptions as those thrown by Scanner.nextInt.

The job of determining whether or not a token is an operator, operand, or a variable is left up to the Operator, Operand, and Variable classes respectively. This allows the definition of an operator, operand, or a variable to be changed without the necessity to change the Tokenizer class.

Note that the tokenizer does not distinguish between a minus sign used as an operator and a minus sign at the beginning of an operand value. Thus the string

 45 - 67
 

could be interpreted either as the operand 45 followed by the operator - and the operand 67, or as the operand 45 followed by the operand -67. Similarly the string 32*-14 could be (mis)interpreted in more than one way. It is up to the programmer to keep track of whether an operator or an operand is next expected in the sequence of tokens.

Version:

2007-10-12 (For CP213 Fall 2007 - Assignment 1)

Author:

David Brown

Field Summary

private boolean	closed A flag for determining if the tokenizer is open or closed.
java.lang.String	string The string to tokenize.

Constructor Summary

Tokenizer(java.lang.String string)
Create a Tokenizer object.

Method Summary

void	close() Closes this tokenizer.
boolean	hasNext() Returns true if potentially there are tokens remaining in the tokenizer string.
boolean	hasNextOperand() Returns true if the next token in the tokenizer string can be interpreted as an operand.
boolean	hasNextOperator() Returns true if the next token in the tokenizer string can be interpreted as an operator.
boolean	hasNextVariable() Returns true if the next token in the tokenizer string can be interpreted as a variable.
Operand	nextOperand() Attempts to extract an operand from the beginning of the tokenizer string.
Operator	nextOperator() Attempts to extract an operator from the beginning of the tokenizer string.
Variable	nextVariable() Attempts to extract a variable from the beginning of the tokenizer string.
void	skip() Removes the first character from the tokenizer string only if the character is not part of an operand, operator, or variable.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

closed

private boolean closed

A flag for determining if the tokenizer is open or closed.

string

public java.lang.String string

The string to tokenize.

Constructor Detail

Tokenizer

public Tokenizer(java.lang.String string)

Create a Tokenizer object.

Parameters:

string - The string to tokenize.

Method Detail

close

public void close()

Closes this tokenizer. The tokenizer string is set to null. Tokens cannot be extracted from a closed tokenizer.

hasNext

public boolean hasNext()

Returns true if potentially there are tokens remaining in the tokenizer string. If the string has a length greater than zero there are potentially tokens available in the string.

Returns:

true if and only if potentially there are tokens remaining in this tokenizer

Throws:

java.lang.IllegalStateException - - if this tokenizer is closed

hasNextOperand

public boolean hasNextOperand()

Returns true if the next token in the tokenizer string can be interpreted as an operand.

Returns:

true if and only if the next token in the tokenizer is an operand

Throws:

java.lang.IllegalStateException - - if this tokenizer is closed

hasNextOperator

public boolean hasNextOperator()

Returns true if the next token in the tokenizer string can be interpreted as an operator.

Returns:

true if and only if the next token in the tokenizer is an operator

Throws:

java.lang.IllegalStateException - - if this tokenizer is closed

hasNextVariable

public boolean hasNextVariable()

Returns true if the next token in the tokenizer string can be interpreted as a variable.

Returns:

true if and only if the next token in the tokenizer is a variable

Throws:

java.lang.IllegalStateException - - if this tokenizer is closed

nextOperand

public Operand nextOperand()

Attempts to extract an operand from the beginning of the tokenizer string.

Returns:

an operand token from the tokenizer string

Throws:

java.lang.IllegalStateException - - if this tokenizer is closed

java.util.NoSuchElementException - - if the tokenizer string is exhausted

java.util.InputMismatchException - - if the next token is not an operand (thrown from 'matchResult')

nextOperator

public Operator nextOperator()

Attempts to extract an operator from the beginning of the tokenizer string.

Returns:

an operator token from the tokenizer string

Throws:

java.lang.IllegalStateException - - if this tokenizer is closed

java.util.NoSuchElementException - - if the tokenizer string is exhausted

java.util.InputMismatchException - - if the next token is not an operator (thrown from 'matchResult')

nextVariable

public Variable nextVariable()

Attempts to extract a variable from the beginning of the tokenizer string.

Returns:

a Variable

Throws:

java.lang.IllegalStateException - - if this tokenizer is closed

java.util.NoSuchElementException - - if the tokenizer string is exhausted

java.util.InputMismatchException - - if the next token is not a variable (thrown from 'matchResult')

skip

public void skip()

Removes the first character from the tokenizer string only if the character is not part of an operand, operator, or variable. It cannot be used to remove an arbitrary character from the beginning of the tokenizer string.

Throws:

java.lang.IllegalStateException - - if this tokenizer is closed