Evaluate Agent

We have customized a set of test data and created an evaluator to test any agent.

If you want to test our existing agent or you’ve developed a new one, just evaluate it on our algorithm to compare it with others.

File: evaluator.py

Evaluation is based on two factors:

1. Time
2. Accuracy

---

To test

cd into atarashi/atarashi/evaluator/ directory

• DLD agent

  python evaluator.py -a DLD

• wordFrequencySimilarity agent

  python evaluator.py -a wordFrequencySimilarity

• tfidf agent

  python evaluator.py -a tfidf

• tfidf with Cosine similarity

  python evaluator.py -a tfidf -s CosineSim

• tfidf with Score similarity

  python evaluator.py -a tfidf -s ScoreSim

• Ngram agent

  python evaluator.py -a Ngram

• Ngram With Cosine similarity

  python evaluator.py -a Ngram -s CosineSim

• Ngram With Dice similarity

  python evaluator.py -a Ngram -s DiceSim

• Ngram With Bigram Cosine similarity

  python evaluator.py -a Ngram -s BigramCosineSim

To check out the latest evaluation report please refer to Atarashi Evaluation Report

---