deepbox/nn
Attention & Transformer
Attention modules, positional encoding, and encoder or decoder transformer stacks.
Transformer
FullTransformer
Full Transformer model with encoder and decoder stacks.
MultiheadAttention
MultiheadAttention is part of the deepbox/nn public API.
PositionalEncoding
Sinusoidal Positional Encoding.
TransformerDecoder
Transformer Decoder — a stack of N decoder layers.
TransformerDecoderLayer
Transformer Decoder Layer.
TransformerEncoder
Transformer Encoder — a stack of N encoder layers.
TransformerEncoderLayer
Transformer Encoder Layer.
causalMask
export declare function causalMask(seqLen: number): Tensor;
Multi-Head Attention mechanism.
nn-attention.ts
import { FullTransformer, MultiheadAttention, PositionalEncoding, TransformerEncoderLayer,} from "deepbox/nn";console.log(new MultiheadAttention(64, 8));console.log(new TransformerEncoderLayer(64, 8));console.log(new PositionalEncoding(64));console.log(new FullTransformer(64, 8));