Class TextFileIdTableInitializer
Inherits From: TextFileInitializer
Defined in tensorflow/python/ops/lookup_ops.py
.
Table initializer for string to int64
IDs tables from a text file.
__init__
__init__(
filename,
key_column_index=TextFileIndex.WHOLE_LINE,
value_column_index=TextFileIndex.LINE_NUMBER,
vocab_size=None,
delimiter='\t',
name='text_file_id_table_init',
key_dtype=tf.dtypes.string
)
Constructs an initializer for an string-to-id table from a text file.
It populates a table that its key and value types are string and int64, respectively. It generates one key-value pair per line. The content of the key and value are specified by the key_index and value_index.
- TextFileIndex.LINE_NUMBER means use the line number starting from zero, expects data type int64.
- TextFileIndex.WHOLE_LINE means use the whole line content, expects data type string.
- A value >=0 means use the index (starting at zero) of the split line based
on
delimiter
.
Args:
filename
: The filename of the text file to be used for initialization. The path must be accessible from wherever the graph is initialized (eg. trainer or eval workers). The filename may be a scalarTensor
.key_column_index
: The column index from the text file to get thekey
values from. The default is to use the whole line content.value_column_index
: The column index from the text file to get thevalue
values from. The default is to use the line number, starting from zero.vocab_size
: The number of elements in the file, if known.delimiter
: The delimiter to separate fields in a line.name
: Optional name for the op.key_dtype
: Thekey
data type.
Raises:
TypeError
: when the filename is empty, or when the table key and value data types do not match the expected data types.
Properties
key_dtype
The expected table key dtype.
value_dtype
The expected table value dtype.
Methods
tf.contrib.lookup.TextFileIdTableInitializer.initialize
initialize(table)
Initializes the table from a text file.
Args:
table
: The table to be initialized.
Returns:
The operation that initializes the table.
Raises:
TypeError
: when the keys and values data types do not match the table key and value data types.