Class Token

Abstract class for different token implementations.

Inheritance

System.Object

Token

GeneralPunctuationToken

GenericPlaceableToken

Implements

ICloneable<Token>

Namespace: Sdl.Core.Bcm.BcmModel.Tokenization.Tokens

Assembly: Sdl.Core.Bcm.BcmModel.dll

Syntax

public abstract class Token : Object, ICloneable<Token>

Constructors

Token()

Initializes a new instance with default values.

Declaration

protected Token()

Token(String)

Initializes a new instance with the specified token text.

Declaration

protected Token(string text)

Parameters

Type	Name	Description
System.String	text	The token text

Properties

Span

Gets or sets the span of text in the Segment representing the Token.

Declaration

public SegmentRange Span { get; set; }

Property Value

Type	Description
SegmentRange	The text span.

Text

Gets or sets the token text.

Declaration

public string Text { get; set; }

Property Value

Type	Description
System.String	The token text.

Type

Gets the type of Token. Should be overriden with names that uniquely identify the implementations.

Declaration

protected abstract string Type { get; }

Property Value

Type	Description
System.String	The type.

Methods

Clone()

Clones this instance.

Declaration

public virtual Token Clone()

Returns

Type	Description
Token

ToString()

Converts to string.

Declaration

public override string ToString()

Returns

Type	Description
System.String	A System.String that represents this instance.

Implements

ICloneable<T>