Toro

JsonConfig

JSON configuration helpers for Toro model-family packages.

FunctionDescription
boolValue label root nameRead a required Boolean property.
floatValue label root nameRead a required floating-point property.
int64Element label name valueRead an integer JSON element.
int64Value label root nameRead a required integer property.
property label root nameGet a required object property.
stringValue label root nameRead a required string property.
tryProperty root nameFind an optional object property.
validateObject label rootValidate that a configuration root is an object without duplicate keys.

boolValue

boolValue label root name

Read a required Boolean property.

Parameters

  • label : string
  • root : JsonElement
  • name : string

Returns bool

floatValue

floatValue label root name

Read a required floating-point property.

Parameters

  • label : string
  • root : JsonElement
  • name : string

Returns float

int64Element

int64Element label name value

Read an integer JSON element.

Parameters

  • label : string
  • name : string
  • value : JsonElement

Returns int64

int64Value

int64Value label root name

Read a required integer property.

Parameters

  • label : string
  • root : JsonElement
  • name : string

Returns int64

property

property label root name

Get a required object property.

Parameters

  • label : string
  • root : JsonElement
  • name : string

Returns JsonElement

stringValue

stringValue label root name

Read a required string property.

Parameters

  • label : string
  • root : JsonElement
  • name : string

Returns string

tryProperty

tryProperty root name

Find an optional object property.

Parameters

  • root : JsonElement
  • name : string

Returns JsonElement option

validateObject

validateObject label root

Validate that a configuration root is an object without duplicate keys.

Parameters

  • label : string
  • root : JsonElement

LocalModelAssets

Local Hugging Face-style asset helpers for Toro model-family packages.

FunctionDescription
dtype label tensorName readerRead the dtype of a required tensor from model metadata.
openReader label directoryOpen a validated single-file or sharded SafeTensors model directory.

dtype

dtype label tensorName reader

Read the dtype of a required tensor from model metadata.

Parameters

  • label : string
  • tensorName : string
  • reader : SafeTensorReader

Returns ScalarType

openReader

openReader label directory

Open a validated single-file or sharded SafeTensors model directory.

Parameters

  • label : string
  • directory : string

Returns string * SafeTensorReader

TensorOwner

Tensor ownership helpers for Toro model-family packages.

FunctionDescription
disposeDistinct namedTensors ownerDispose each distinct tensor returned by a named-tensor projection exactly once.

disposeDistinct

disposeDistinct namedTensors owner

Dispose each distinct tensor returned by a named-tensor projection exactly once.

Parameters

  • namedTensors : 'Owner -> (string * Tensor) seq
  • owner : 'Owner

FixedKvCache

Fixed-capacity key/value storage shared by Toro causal model-family packages.

Constructors

NameDescription
FixedKvCache(ownerName, layerCount, batchSize, headCount, capacity, headSize, dtype, device)

Instance members

NameDescription
this.AdvanceAdvance the logical cache length after every layer has appended.
this.AppendAppend one layer's key and value tensors and return the populated views.
this.BatchSizeNumber of batch items stored by the cache.
this.CapacityMaximum number of tokens stored by the cache.
this.LengthNumber of tokens currently stored by the cache.
this.ResetRemove all logical entries without reallocating storage.
this.ValidateValidate an input batch and sequence against the cache state.

PreparedCausalInput

Validated and normalized causal model input for model-family packages.

Record fields

NameDescription
AttentionMaskBroadcastable padding and causal mask when an explicit mask is required.
CacheStartNumber of cached tokens preceding this invocation.
IsCausalWhether the attention implementation should apply its built-in causal mask.
PositionIdsValidated or generated absolute token positions.
SequenceLengthNumber of input tokens processed in this invocation.

CausalInput

Input preparation shared by Toro causal model-family packages.

FunctionDescription
prepare modelName maxPositions cacheLength validateCache inputValidate and normalize a causal language-model input for one forward pass.

prepare

prepare modelName maxPositions cacheLength validateCache input

Validate and normalize a causal language-model input for one forward pass.

Parameters

  • modelName : string
  • maxPositions : int64
  • cacheLength : 'Cache -> int64
  • validateCache : 'Cache -> int64 -> int64 -> unit
  • input : CausalLmInput<'Cache>

Returns PreparedCausalInput