EduNLP.I2V¶

It just a api, so you shouldn’t use it directly. If you want to get vector from item, you can use other model like D2V and W2V.

param tokenizer: the tokenizer name
type tokenizer: str
param t2v: the name of token2vector model
type t2v: str
param args: the parameters passed to t2v
param tokenizer_kwargs: the parameters passed to tokenizer
type tokenizer_kwargs: dict
param pretrained_t2v
type pretrained_t2v: bool
param kwargs: the parameters passed to t2v

Examples

>>> item = {"如图来自古希腊数学家希波克拉底所研究的几何图形．此图由三个半圆构成，三个半圆的直径分别为直角三角形$ABC$的斜边$BC$,     ... 直角边$AB$, $AC$.$\bigtriangleup ABC$的三边所围成的区域记为$I$,黑色部分记为$II$, 其余部分记为$III$.在整个图形中随机取一点，    ... 此点取自$I,II,III$的概率分别记为$p_1,p_2,p_3$,则$\SIFChoice$$\FigureID{1}$"}
>>> model_path = "examples/test_model/test_gensim_luna_stem_tf_d2v_256.bin" 
>>> i2v = D2V("text","d2v",filepath=model_path, pretrained_t2v = False) 
>>> i2v(item) 
([array([ ...dtype=float32)], None)

returns: i2v model
rtype: I2V

EduNLP.I2V.i2v.I2V.tokenize(self, items, indexing=True, padding=False, key=<function I2V.<lambda>>, *args, **kwargs) → list¶: tokenize item

EduNLP.I2V.D2V¶

Examples

>>> item = {"如图来自古希腊数学家希波克拉底所研究的几何图形．此图由三个半圆构成，三个半圆的直径分别为直角三角形$ABC$的斜边$BC$,     ... 直角边$AB$, $AC$.$\bigtriangleup ABC$的三边所围成的区域记为$I$,黑色部分记为$II$, 其余部分记为$III$.在整个图形中随机取一点，    ... 此点取自$I,II,III$的概率分别记为$p_1,p_2,p_3$,则$\SIFChoice$$\FigureID{1}$"}
>>> model_path = "examples/test_model/test_gensim_luna_stem_tf_d2v_256.bin" 
>>> i2v = D2V("text","d2v",filepath=model_path, pretrained_t2v = False) 
>>> i2v(item) 
([array([ ...dtype=float32)], None)

returns: i2v model
rtype: I2V

EduNLP.I2V.i2v.D2V.infer_vector(self, items, tokenize=True, indexing=False, padding=False, key=<function D2V.<lambda>>, *args, **kwargs) → tuple¶

Parameters

items (str) –
tokenize –
indexing –
padding –
key –
args –
kwargs –

Returns

Return type

vector

EduNLP.I2V.i2v.D2V.tokenize(self, items, indexing=True, padding=False, key=<function I2V.<lambda>>, *args, **kwargs) → list¶: tokenize item

EduNLP.I2V.W2V¶

Examples

>>> i2v = get_pretrained_i2v("test_w2v", "examples/test_model/data/w2v") 
>>> item_vector, token_vector = i2v(["有学者认为：‘学习’，必须适应实际"])
>>> item_vector 
array([[...]], dtype=float32)

returns: i2v model
rtype: W2V

EduNLP.I2V.i2v.W2V.tokenize(self, items, indexing=True, padding=False, key=<function I2V.<lambda>>, *args, **kwargs) → list¶: tokenize item

EduNLP.I2V.get_pretrained_i2v¶

param name
param model_dir
returns: i2v model
rtype: I2V

Examples

>>> item = {"如图来自古希腊数学家希波克拉底所研究的几何图形．此图由三个半圆构成，三个半圆的直径分别为直角三角形$ABC$的斜边$BC$,     ... 直角边$AB$, $AC$.$\bigtriangleup ABC$的三边所围成的区域记为$I$,黑色部分记为$II$, 其余部分记为$III$.在整个图形中随机取一点，    ... 此点取自$I,II,III$的概率分别记为$p_1,p_2,p_3$,则$\SIFChoice$$\FigureID{1}$"}
>>> i2v = get_pretrained_i2v("test_d2v", "examples/test_model/data/d2v") 
>>> print(i2v(item)) 
([array([ ...dtype=float32)], None)