core.transformer¶

Encoder-only transformer model for neural user simulator.

Classes¶

`PositionalEncoding`	Initializes positional encoding layer.
`TransformerEncoderModel`	Initializes a encoder-only transformer model.

Module Contents¶

class core.transformer.PositionalEncoding(d_model: int, dropout: float = 0.1, max_len: int = 5000, **kwargs)¶

Bases: torch.nn.Module

Initializes positional encoding layer.

Parameters:

d_model – Dimension of the model.
dropout – Dropout rate. Defaults to 0.1.
max_len – Maximum length of the input sequence. Defaults to 5000.

forward(x: torch.Tensor) → torch.Tensor¶

Performs forward pass.

Parameters:: x – Input tensor.
Returns:: Positional encoded tensor.

class core.transformer.TransformerEncoderModel(input_dim: int, output_dim: int, nhead: int, hidden_dim: int, num_encoder_layers: int, num_token: int, dropout: float = 0.5)¶

Bases: torch.nn.Module

Initializes a encoder-only transformer model.

Parameters:

input_dim – Size of the input vector.
output_dim – Size of the output vector.
nhead – Number of heads.
hidden_dim – Hidden dimension.
num_encoder_layers – Number of encoder layers.
num_token – Number of tokens in the vocabulary.
dropout – Dropout rate. Defaults to 0.5.

init_weights() → None¶: Initializes weights of the network.

forward(src: torch.Tensor, src_mask: torch.Tensor = None) → torch.Tensor¶

Performs forward pass.

Parameters:

src – Source tensor.
src_mask – Mask tensor.

Returns:

Output tensor.