File List

Here is a list of all files with brief descriptions:

address.gtest.cpp	Unit testing: integerpatternaddress
addresshandler.hpp	Handles simple wildcard expansion for strings
AlignmentCountMapWritable.java
alilats2splats.main.cpp	Alignment lattices to sparse vector weight lattices
applylm.main.cpp	Main file for applylm tool
backtrace.hpp	Adapted from http://charette.no-ip.com:81/programming/2010-01-25_Backtrace/ Simple example of backtrace(), backtrace_symbols(), and __cxa_demangle() Remember to pass in the -rdynamic flag to GCC:
bleu.hpp
BleuStats.cpp
BleuStats.h
CLI.java
common-helpers.hpp
CommonFlags.h
constants-fsttools.hpp
constants-hifst.hpp
constants-lmert.hpp
CopyRecordsForTesting.java
countstrings.main.cpp
createssgrammar.main.cpp	Hifst main entry file
custom_assert.hpp	Provides smarter assert methods
data-main.alilats2splats.hpp	Data object for alilats to sparse weight lats binary
data-main.applylm.hpp	Data object for applylm tool
data-main.createssgrammar.hpp	Data object for hifst or related tools
data-main.disambig.hpp	Data object for disambig tool
data-main.hifst-client.hpp	Data object for hifst-client
data-main.hifst.hpp	Data object for hifst or related tools
data-main.lmbr.hpp
data-main.rules2weights.hpp	Data object for alilats to sparse weight lats binary
data.cykparser.cykbackpointers.hpp	Contains functor that provides access to cyk backpointers
data.cykparser.cykgrid.hpp	Contains functor for the cyk grid
data.cykparser.hpp	Contains structures and classes for GrammarData
data.grammar.comparetool.hpp	Contains structures and classes for GrammarData
data.grammar.hpp	Contains structures and classes for GrammarData
data.grammar.utilities.hpp	Contains structures and classes for GrammarData
data.lm.hpp	Implementation of a language model data structure using kenlm
data.lmbr.hpp
data.ssgrammar.hpp	Contains sentence-specific grammar data
data.stats.hpp	Relative to Stats across the pipeline
DebugMert.cpp
DebugMert.h
defs.cykparser.hpp	Contains definitions for cykparser data and task
defs.grammar.hpp	Contains definitions for cykparser data and task
defs.ssgrammar.hpp	Contains definitions for sentence-specific grammar data and task
disambig.main.cpp	Main file for disambig tool
disambignffst.main.cpp
EnumRuleType.java
ErrorSurface.cpp
ErrorSurface.h
ExtractedData.java
ExtractorDataLoader.java
ExtractorJob.java
fast-shortest-distance.h
Feature.java
FeatureFunctionRegistry.java
FeatureFunctions.java
FeatureMap.java
FeatureRegistry.java
fstio.gtest.cpp	Unit testing: Input/Output fst operations with compression
fstio.hpp	Contains convenience functions to write and read fsts
fstutils.applylmonthefly.hpp	Contains implementation of ApplyLanguageModelOnTheFly
fstutils.extractngrams.hpp
fstutils.ftcompose.hpp	Implementation of different type of compositions (i.e. failure transitions)
fstutils.gtest.cpp	Unit testing: String printing from lattice, multiepsilon composition, composition with failure transitions, generic weight mappers, multiple union of fsts, etc
fstutils.hpp	Utilites to extract vocabulary, pseudo-determinize lattices and build substring transducers
fstutils.mapper.hpp	Generalized weight mapper functor
fstutils.multiepsiloncompose.hpp	Multiepsilon composition
fstutils.multiunion.hpp	Implementations of multiple fst unions
fstutils.topofeatures.hpp	Support for Topological Features. See Iglesias et al. 2015
function-weight.cpp
function-weight.h
global_decls.hpp	General typedefs, defines..
global_funcs.hpp	General functions
global_incls.hpp	All included standard headers, boost headers,..
globalfunctions.gtest.cpp	Unit testing: General functions
googletesting.h	Unit testing: google testing common header
HFilePrint.java
HFileRuleQuery.java
HFileRuleReader.java
hifst-client.main.cpp	Hifst client main entry file
hifst.main.cpp	Hifst main entry file
hifst.task.cykparser.gtest.cpp	Unit testing: cyk parser
hifst.task.grammar.gtest.cpp	Unit testing: grammar task testing
hifst.task.hifst.gtest.cpp	Unit testing: hifst lattice-building
hifst.task.patternstoinstances.gtest.cpp	Unit testing: converting grammar-specific patterns into instance patterns
hifst.task.postpro.gtest.cpp	Unit testing: Postprocessing 1-best translation
hifst.task.prepro.gtest.cpp	Unit testing: Preprocess source test
hifst.task.referencefilter.gtest.cpp	Unit testing: Reference filter task
hifst.task.ssgrammar.gtest.cpp	Unit testing: sentence-specific grammar task
hifst.task.stats.gtest.cpp	Unit testing: Stats task testing
hifst_enumerate_vocab.hpp	Extend EnumerateVocab to access kenlm ids
idbridge.hpp	Maps between grammar targets ids and lm ids
IntervalData.cpp
IntervalData.h
IntWritableCache.java
kenlmdetect.hpp
latmert.main.cpp
LatMertMain.h
LexicalProbability.java
lexicographic-tropical-tropical-decls.h	Lexicographic stdarc registering
lexicographic-tropical-tropical-funcs.h	Convenience functors/functions for lexicographic<tropical,tropical> semiring
lexicographic-tropical-tropical-incls.h	Headers for standalone shared library
lexmap.main.cpp	Main file for applylm tool
lineoptimize.hpp
lmbr.gtest.cpp	Unit testing: String printing from lattice, multiepsilon composition, composition with failure transitions, generic weight mappers, multiple union of fsts, etc
lmbr.main.cpp	Alignment lattices to sparse vector weight lattices
LMert.cpp
LMert.h
lmert.hpp
lmert.main.cpp
logger.boost_log.hpp	Logger implementation – init method and macros around actual Boost Logger
logger.hpp
logger.openfstglog.hpp	Logger implementation – init method and macros around actual OpenFST logger – requires including openfst libraries first
main-run.alilats2splats.hpp	Implements single-threaded version of alilats2splats tool
main-run.applylm.hpp	Core implementation of applylm binary. Kicks off either singlethreaded or multithreaded language model application
main-run.createssgrammar.hpp	Contains createssgrammar core implementation, single-threaded or multithreaded
main-run.hifst-client.hpp	Contains hifst client core implementation, sends request to hifst in server mode
main-run.hifst.hpp	Contains hifst core: implements single threaded, multithreaded or as server
main-run.lmbr.hpp	Implements single-threaded version of alilats2splats tool
main-run.rules2weights.hpp	Implements lats2splats tool
main.alilats2splats.hpp	Included headers for all the binary should be defined here. This file should be included only once
main.alilats2splats.init_param_options.hpp	To initialize boost parameter options
main.applylm.hpp	Included headers for all the binary should be defined here. This file should be included only once
main.applylm.init_param_options.hpp	To initialize boost parameter options
main.applylm.init_param_options_common.hpp	To initialize boost parameter options
main.countstrings.hpp	Included headers for all the binary should be defined here. This file should be included only once
main.countstrings.init_param_options.hpp	To initialize boost parameter options
main.createssgrammar.hpp	Included headers for all the binary (createssgrammar) should be defined here. This file should be included only once
main.createssgrammar.init_param_options.hpp	To initialize boost parameter options for createssgrammar tool
main.createssgrammar.init_param_options_common.hpp	To initialize boost parameter options for createssgrammar tool
main.custom_assert.hpp	Static variable for custom_assert. Include only once from main file
main.disambig.hpp	Included headers for the binary (disambig) should be defined here. This file should be included only once
main.disambig.init_param_options.hpp	To initialize boost parameter options
main.disambignffst.hpp	Included headers for all the binary should be defined here. This file should be included only once
main.disambignffst.init_param_options.hpp	To initialize boost parameter options
main.gtest.hifst.cpp	Unit testing: main file
main.hifst-client.hpp	Included headers for all the binary (hifst-client) should be defined here. This file should be included only once
main.hifst-client.init_param_options.hpp	To initialize boost parameter options for hifst-client
main.hifst.hpp	Included headers for all the binary should be defined here. This file should be included only once
main.hifst.init_param_options.hpp	To initialize boost parameter options for hifst tool
main.hpp
main.lexmap.hpp
main.lexmap.init_param_options.hpp	To initialize boost parameter options
main.lmbr.hpp	Included headers for all the binary should be defined here. This file should be included only once
main.lmbr.init_param_options.hpp	To initialize boost parameter options
main.lmert.hpp
main.lmert.init_param_options.hpp
main.logger.hpp	Static variables for logger. Include only once from main file
main.printstrings.hpp
main.printstrings.init_param_options.hpp	To initialize boost parameter options
main.rules2weights.hpp	Included headers for all the binary should be defined here. This file should be included only once
main.rules2weights.init_param_options.hpp	To initialize boost parameter options
main.rules2weights.init_param_options_common.hpp	To initialize boost parameter options
main.samplehyps.hpp
main.samplehyps.init_param_options.hpp
main.tunewp.hpp	Included headers for all the binary should be defined here. This file should be included only once
main.tunewp.init_param_options.hpp	To initialize boost parameter options
mapping-shortest-path.h
MarginalReducer.java
MergeComparator.java
MergeJob.java
MergePartitioner.java
MertCommon.cpp
MertCommon.h
MertHashVec.cpp
MertHashVec.h
MertLib.h
MertPrune.cpp
MertPrune.h
multithreading.gtest.cpp	Unit testing: multithreading with a threadpool
multithreading.helpers.hpp
multithreading.hpp	Implements trivial threadpool using boost::asio library
openfst.h	Unit testing: google testing common header
openfstversion.hpp
Optimize.cpp
Optimize.h
Pair.java
PairWritable.java
params.h
params.hpp	Convenience functions to parse parameters from a string
ParamsConfig.cpp
ParamsConfig.h
PatternInstanceCreator.java
PhraseJob.java
printstrings.main.cpp	Main file for printstrings tool
ProvenanceCountMap.java
ProvenanceProbMap.java
randomlinesearch.hpp
range.gtest.cpp	Unit testing: getRange, InfiniteRange,IntegerRange, OneRange objects
range.hpp	Handles different type of integer ranges
RefsData.cpp
RefsData.h
registrypo.gtest.cpp	Unit testing: registrypo class and related functions
registrypo.hpp	Contains wrapper class RegistryPO, which uses boost::program_options to parse parameters, and provides methods to access them
RuleData.java
RuleExtractorTest.java
RuleFilter.java
RulePattern.java
RuleRetriever.java
rules2weights.main.cpp	Alignment lattices to sparse vector weight lattices
samplehyps.main.cpp
Score.cpp
Score.h
score.main.cpp
SequenceFilePrint.java
SidePattern.java
SimpleHFileOutputFormat.java
Source2TargetJob.java
szfstream.gtest.cpp	Unit testing: szfstream class for [file] operations
szfstream.hpp	Stream wrapper for pipe/text/compressed files
Target2SourceJob.java
TargetFeatureList.java
task.applylm.hpp	Implementation of a language model application task
task.applylm.kenlmtype.hpp	Wrapper to ApplyLanguageModelOnTheFly to apply different kenlm models
task.cykparser.hpp	Contains cyk parser implementation
task.disambig.flowerfst.hpp	Utilities for DisambigTask and related tasks
task.disambig.hpp	Implemention of class DisambigTask
task.dumpnbestfeatures.hpp	Contains task that dumps nbest and feature file
task.grammar.hpp	Describes class GrammarTask
task.grammar.nonterminalhierarchy.hpp	This class decides automatically the hierarchy of non-terminals
task.hifst-stats.hpp	Task that dumps statistics related specifically to the tool hifst
task.hifst.expandednumstates.hpp	Contains utility class to predict number of states of an RTN after expanding to equivalent FSA
task.hifst.hpp	Contains structures and classes for hifst task (target lattice building)
task.hifst.localpruningconditions.hpp	Contains functor and struct to handle local pruning conditions
task.hifst.makeweights.hpp
task.hifst.optimize.hpp	Contains Function objects that optimize a machine
task.hifst.replacefstbyarc.hpp	Contains Function objects that determine whether an FST is replaceable or not by an arc pointer
task.hifst.rtn.hpp	Implements RTN class. Stores pointers to cell FSAs of the RTN using a hiero-index representing cell coordinates (cc,x,y)
task.lmbr.applyposteriors.hpp	Based on Graeme Blackwood's PhD work and original code – implementation of posterior application to a hypotheses space
task.lmbr.common.hpp	Common lmbr functions
task.lmbr.computeposteriors.hpp	Based on Graeme Blackwood's PhD work and original code – implementation of posterior computation from evidence space
task.lmbr.hpp	Lattice MBR task – integrates lattice mbr as a task that can be used standalone (implemented) or included e.g. as another task in hifst
task.loadlm.hpp	Implementation of a language model task
task.loadsparseweightflowerfst.hpp	Implements a class that loads the grammar sparseweight flower lattice and stores a pointer on the data object
task.loadunimap.hpp	Implementation of a unigram transduction model loader task
task.loadwordmap.hpp	Wrapper around WordMapper loader
task.optimizefst.hpp	Implementation of a Fst writer taking the fst from data object
task.patternstoinstances.hpp	Contains patterns to instance-patterns implementation
task.postpro.hpp	Task that writes translation to a text file. This translation might be wordmapped and tokenized
task.prepro.hpp	Describes class PreProTask, which preprocesses (tokenizes and maps to integers with WordMapper) source input
task.readfst.hpp	Implementation of a Fst reader to data structure
task.referencefilter.hpp	Describes class ReferenceFilterTask (builds unweighted substring fst for lattice alignment )
task.sparseweightvectorlattices.hpp	Implements the task of creating sparse vector weight lattices – contains feature weight contributions separately in each arc and we can use it to dump features, MERT training, etc
task.ssgrammar.hpp	Contains implementation for sentence-specific grammar task
task.stats.hpp	Task that dumps statistics stored by any previous task in the pipeline
task.tunewpwrite.hpp	Tune word penalty and write output
task.writefst.hpp	Implementation of a Fst writer taking the fst from data object
taskinterface.gtest.cpp	Unit testing: TaskInterface methods
taskinterface.hpp	Interfaces with basic methods for iteration
TestCommandLineInterface.java
TestFeatureRegistry.java
TextArrayWritable.java
TGMert.cpp
TGMert.h
tokenizer.osr.hpp	Lower casing/Tokenization/Detokenization not available for open source release
fsttools/include/tropical-sparse-tuple-weight-decls.h	Basic declarations used for tropical sparse vector weight semiring
latmert/include/tropical-sparse-tuple-weight-decls.h
tropical-sparse-tuple-weight-funcs.h	Convenience functions for tropical sparse vector weight
tropical-sparse-tuple-weight-incls.h	Files to include for the tropical sparse tuple semiring
fsttools/include/tropical-sparse-tuple-weight.h	Implementation of tropical sparse tuple weight semiring
latmert/include/tropical-sparse-tuple-weight.h
tropical-sparse-tuple-weight.makeweight.h	Convenience functors that allow transparent handling for weights within hifst
tropical_LT_tropical-arc.so.cpp	Main target file that generates shared library
tropicalsparsetuple-arc.so.cpp	Main target file for compilation into a shared library
TTableClient.java
TTableServer.java
TuneSet.cpp
TuneSet.h
tuneset.hpp
tunewp.main.cpp
Util.java
vecmap.main.cpp
weights.gtest.cpp	Unit testing: Various weight makers on different semirings
wordmapper.gtest.cpp	Unit testing: WordMapper, class that maps words to integers and vice versa
wordmapper.hpp	Class WordMapper