fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT > Class Template Reference

Class that applies language model on the fly using kenlm. More...

#include <fstutils.applylmonthefly.hpp>

Inheritance diagram for fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >:

Collaboration diagram for fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >:

Public Member Functions
void	setMakeWeight (const MakeWeightT &mw)
	Public methods. More...
	ApplyLanguageModelOnTheFly (KenLMModelT &model, unordered_set< Label > &epsilons, bool natlog, float lmscale, float lmwp, const IdBridgeT &idbridge, MakeWeightT &mw)
	ApplyLanguageModelOnTheFly (KenLMModelT &model, bool natlog, float lmscale, float lmwp, const IdBridgeT &idbridge, MakeWeightT &mw)
void	init ()
	~ApplyLanguageModelOnTheFly ()
	Destructor. More...
VectorFst< Arc > *	run (const VectorFst< Arc > &fst)
VectorFst< Arc > *	run (const VectorFst< Arc > &fst, unordered_set< Label > const &epsilons)
VectorFst< Arc > *	run (const VectorFst< Arc > &fst, unsigned size, std::vector< std::vector< unsigned > > &srcWindows)
VectorFst< Arc > *	operator() (const VectorFst< Arc > &fst)
	functor: Run composition itself. Use for target-only LMs. More...
VectorFst< Arc > *	operator() (const VectorFst< Arc > &fst, unsigned srcSize, std::vector< std::vector< unsigned > > &srcw)
Public Member Functions inherited from fst::ApplyLanguageModelOnTheFlyInterface< Arc >
virtual VectorFst< Arc > *	run (VectorFst< Arc > const &fst, unordered_set< typename Arc::Label > const &epsilons)=0
virtual	~ApplyLanguageModelOnTheFlyInterface ()

Detailed Description

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>
class fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >

Class that applies language model on the fly using kenlm.

Remarks

This implementation could be optimized a little further, i.e. all visited states must be tracked down so that non-topsorted or cyclic fsts work correctly. But we could keep track of these states in a memory efficient way (i.e. only highest state n for consecutive 0-to-n seen).

Definition at line 312 of file fstutils.applylmonthefly.hpp.

Constructor & Destructor Documentation

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>

fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >::ApplyLanguageModelOnTheFly ( KenLMModelT &  model, unordered_set< Label > &  epsilons, bool  natlog, float  lmscale, float  lmwp, const IdBridgeT &  idbridge, MakeWeightT &  mw  )

inline

Constructor. Initializes on-the-fly composition with a language model.

Parameters

fst	Machine you want to apply the language to. Pass a delayed machine if you can, as it will expand it in constructor.
model	A KenLM language model
epsilons	List of words to work as epsilons
natlog	Use or not natural logs
lmscale	Language model scale

Definition at line 379 of file fstutils.applylmonthefly.hpp.

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>

fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >::ApplyLanguageModelOnTheFly ( KenLMModelT &  model, bool  natlog, float  lmscale, float  lmwp, const IdBridgeT &  idbridge, MakeWeightT &  mw  )

inline

Definition at line 399 of file fstutils.applylmonthefly.hpp.

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>

fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >::~ApplyLanguageModelOnTheFly ( )

inline

Destructor.

Definition at line 425 of file fstutils.applylmonthefly.hpp.

Member Function Documentation

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>

void fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >::init ( )

inline

Definition at line 417 of file fstutils.applylmonthefly.hpp.

Here is the call graph for this function:

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>

VectorFst<Arc>* fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >::operator() ( const VectorFst< Arc > &  fst )

inline

functor: Run composition itself. Use for target-only LMs.

Definition at line 445 of file fstutils.applylmonthefly.hpp.

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>

VectorFst<Arc>* fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >::operator() ( const VectorFst< Arc > &  fst, unsigned  srcSize, std::vector< std::vector< unsigned > > &  srcw  )

inline

Definition at line 455 of file fstutils.applylmonthefly.hpp.

Here is the call graph for this function:

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>

VectorFst<Arc>* fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >::run ( const VectorFst< Arc > &  fst )

inlinevirtual

Implements fst::ApplyLanguageModelOnTheFlyInterface< Arc >.

Definition at line 427 of file fstutils.applylmonthefly.hpp.

Here is the caller graph for this function:

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>

VectorFst<Arc>* fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >::run ( const VectorFst< Arc > &  fst, unordered_set< Label > const &  epsilons  )

inline

Definition at line 430 of file fstutils.applylmonthefly.hpp.

Here is the call graph for this function:

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>

VectorFst<Arc>* fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >::run ( const VectorFst< Arc > &  fst, unsigned  size, std::vector< std::vector< unsigned > > &  srcWindows  )

inlinevirtual

Implements fst::ApplyLanguageModelOnTheFlyInterface< Arc >.

Definition at line 436 of file fstutils.applylmonthefly.hpp.

template<class Arc, class MakeWeightT = MakeWeight<Arc>, class KenLMModelT = lm::ngram::Model, class IdBridgeT = ucam::fsttools::IdBridge, template< class > class HackScoreT = HackScore>

void fst::ApplyLanguageModelOnTheFly< Arc, MakeWeightT, KenLMModelT, IdBridgeT, HackScoreT >::setMakeWeight ( const MakeWeightT &  mw )

inline

Public methods.

Set MakeWeight functor

Definition at line 365 of file fstutils.applylmonthefly.hpp.

---

The documentation for this class was generated from the following file:

• fstutils.applylmonthefly.hpp