Tools

  • RbF Curriculum Learner. This tool demonstrates spaced repetition for training any neural network. RbF is inspired by the broad evidence in psychology that shows human ability to retain information improves with repeated exposure and exponentially decays with delay since last exposure. RBF works based on spaced repetition in which training instances are repeatedly presented to the network on a schedule determined by a spaced repetition algorithm. RbF shortens or lengthens review intervals for training instances with respect to a few indicators.

  • Leitner system. Implementation of Leitner system for simultaneously training a given neural network and identifying spurious instances (those with wrong labels) in its input dataset.

  • Twitter Crawler. A crawler for searching on Twitter and obtaining tweets and user networks.

Datasets

  • Churn Dataset: The dataset contains labeled tweets about three telco brands: Verizon, AT&T, and T-Mobile. Tweet are labeled as churny or not-churny, where churny tweets indicate a high risk of canceling the brand's service. Labels are obtained through crowdsourcing and each tweet is labeled by at least three annotators. Fleiss’ kappa is 0.62, which indicates substantial agreement among annotators. Cohen's kappa computed over 1073 instances related to T-Mobile, that were independently labeled by one of our team members and compared against the aggregation of the three annotators judgments over these instances, is 0.93, which indicates substantial annotation agreement as well.