Class Token

Abstract class for different token implementations.

Inheritance

object

Token

GeneralPunctuationToken

GenericPlaceableToken

Implements

ICloneable<Token>

Inherited Members

object.GetType()

object.MemberwiseClone()

object.Equals(object)

object.Equals(object, object)

object.ReferenceEquals(object, object)

object.GetHashCode()

Namespace: Sdl.Core.Bcm.BcmModel.Tokenization.Tokens

Assembly: Sdl.Core.Bcm.BcmModel.dll

Syntax

[DataContract]
[JsonConverter(typeof(TokenCreator))]
public abstract class Token : ICloneable<Token>

Constructors

Token()

Initializes a new instance with default values.

Declaration

protected Token()

Token(string)

Initializes a new instance with the specified token text.

Declaration

protected Token(string text)

Parameters

Type	Name	Description
string	text	The token text

Properties

Span

Gets or sets the span of text in the Segment representing the Token.

Declaration

[DataMember(Name = "span")]
public SegmentRange Span { get; set; }

Property Value

Type	Description
SegmentRange	The text span.

Text

Gets or sets the token text.

Declaration

[DataMember(Name = "text")]
public string Text { get; set; }

Property Value

Type	Description
string	The token text.

Type

Gets the type of Token. Should be overriden with names that uniquely identify the implementations.

Declaration

[DataMember(Name = "type")]
protected abstract string Type { get; }

Property Value

Type	Description
string	The type.

Methods

Clone()

Clones this instance.

Declaration

public virtual Token Clone()

Returns

Type	Description
Token

ToString()

Converts to string.

Declaration

public override string ToString()

Returns

Type	Description
string	A string that represents this instance.

Overrides

object.ToString()

Implements

ICloneable<T>