In file ../include/EST_Token.h:

## class EST_Token

> This class is similar to EST_String but also maintains the original punctuation and whitespace found around the token.

### Public Methods

  | EST_Token ()
  | EST_Token (const EST_String p)
  void | init ()
  | Basic access to fields
  void | set_token (const EST_String &p) set token from a string
  void | set_token (const char* p)
  void | set_whitespace (const EST_String &p) set whitespace of token
  void | set_whitespace (const char* p)
  void | set_punctuation (const EST_String &p) set (post) punctuation of token
  void | set_punctuation (const char* p)
  void | set_prepunctuation (const EST_String &p) set prepunction
  void | set_prepunctuation (const char* p)
  const EST_String& | whitespace ()
  const EST_String& | punctuation ()
  const EST_String& | prepunctuation ()
  | Access token as a string
  const EST_String& | S () const Access token as a string
  const EST_String& | String () const Access token as a string
  | operator EST_String () const For automatic coercion to EST_String
  | Access token as a int
  | Access token as a long
  | Access token as a float
  | Access token as a double
  void | set_row (int r)
  void | set_col (int c)
  void | set_filepos (int c) Set file position in original EST_TokenStream
  EST_String | lstring () Return lower case version of token name
  EST_String | ustring () Return upper case version of token name
  int | row (void) const Line number in original EST_TokenStream
  int | col (void) const Line position in original EST_TokenStream
  int | filepos (void) const file position in original EST_TokenStream
  const EST_String | pos_description () const A string describing current position, suitable for error messages
  EST_Token& | operator = (const EST_Token &a)
  EST_Token& | operator = (const EST_String &a)
  int | operator == (const EST_String &a)
  int | operator != (const EST_String &a)
  int | operator == (const char* a)
  int | operator != (const char* a)

### Public

## Documentation

> This class is similar to EST_String but also maintains the original punctuation and whitespace found around the token.

> EST_Token's primary use is with EST_TokenStream class which allows easy tokenizing of ascii files.

> A token consists of four parts, any of which may be empty: a name, the actual token, preceding whitespace, preceding punctuation, the name and succeeding punctuation.

`EST_Token()`

`EST_Token(const EST_String p)`

`void init()`

`Basic access to fields`

`void set_token(const EST_String &p)`
  set token from a string

`void set_token(const char* p)`

`void set_whitespace(const EST_String &p)`
  set whitespace of token

`void set_whitespace(const char* p)`

`void set_punctuation(const EST_String &p)`
  set (post) punctuation of token

`void set_punctuation(const char* p)`

`void set_prepunctuation(const EST_String &p)`
  set prepunction

`void set_prepunctuation(const char* p)`

`const EST_String& whitespace()`

`const EST_String& punctuation()`

`const EST_String& prepunctuation()`

`Access token as a string`

`const EST_String& S() const`
  Access token as a string

`const EST_String& String() const`
  Access token as a string

`operator EST_String() const`
  For automatic coercion to EST_String

`Access token as a int`

`Access token as a long`

`Access token as a float`

`Access token as a double`

``

`void set_quoted(int q)`
  Note that this token was quoted (or not)

`int quoted() const`
  TRUE is token was quoted

`void set_row(int r)`

`void set_col(int c)`

`void set_filepos(int c)`
  Set file position in original EST_TokenStream

`EST_String lstring()`
  Return lower case version of token name

`EST_String ustring()`
  Return upper case version of token name

`int row(void) const`
  Line number in original EST_TokenStream

`int col(void) const`
  Line position in original EST_TokenStream

`int filepos(void) const`
  file position in original EST_TokenStream

`const EST_String pos_description() const`
  A string describing current position, suitable for error messages

`EST_Token& operator = (const EST_Token &a)`

`EST_Token& operator = (const EST_String &a)`

`int operator == (const EST_String &a)`

`int operator != (const EST_String &a)`

`int operator == (const char* a)`

`int operator != (const char* a)`

This class has no child classes.

Friends:
  ostream& operator << (ostream& s, const EST_Token &p)
Author:
  Alan W Black (awb@cstr.ed.ac.uk): April 1996

Alphabetic index HTML hierarchy of classes or Java

This page is part of the Edinburgh Speech Tools Library documentation
Copyright University of Edinburgh 1997
Contact: speech_tools@cstr.ed.ac.uk