Polarity In The Classroom: An Application Leveraging Peer Sentiment Towards Scalable Assessment

Accurately grading open-ended assignments in large or massive open online courses (MOOCs) is non-trivial. Peer review is a promising solution but can be unreliable due to few reviewers and an unevaluated review form. To date, no work has 1) leveraged sentiment analysis in the peer-review process to inform or validate grades or 2) utilized aspect extraction to craft a review form from what students actually communicated. Our work utilizes, rather than discards, student data from review form comments to deliver better information to the instructor. In this work, we detail the process by which we create our domain-dependent lexicon and aspect-informed review form as well as our entire sentiment analysis algorithm which provides a fine-grained sentiment score from text alone. We end by analyzing validity and discussing conclusions from our corpus of over 6800 peer reviews from nine courses to understand the viability of sentiment in the classroom for increasing the information from and reliability of grading open-ended assignments in large courses.

Zachariah Beasley, Les Piegl, and Paul Rosen. Polarity In The Classroom: An Application Leveraging Peer Sentiment Towards Scalable Assessment. IEEE Transactions on Learning Technologies, 2021.

@article{beasley2021polarity,
  title = {Polarity in the Classroom: An Application Leveraging Peer Sentiment towards
    Scalable Assessment},
  author = {Beasley, Zachariah and Piegl, Les and Rosen, Paul},
  journal = {IEEE Transactions on Learning Technologies},
  volume = {14},
  pages = {515--525},
  year = {2021},
  abstract = {Accurately grading open-ended assignments in large or massive open online
    courses (MOOCs) is non-trivial. Peer review is a promising solution but can be
    unreliable due to few reviewers and an unevaluated review form. To date, no work has 1)
    leveraged sentiment analysis in the peer-review process to inform or validate grades or
    2) utilized aspect extraction to craft a review form from what students actually
    communicated. Our work utilizes, rather than discards, student data from review form
    comments to deliver better information to the instructor. In this work, we detail the
    process by which we create our domain-dependent lexicon and aspect-informed review form
    as well as our entire sentiment analysis algorithm which provides a fine-grained
    sentiment score from text alone. We end by analyzing validity and discussing conclusions
    from our corpus of over 6800 peer reviews from nine courses to understand the viability
    of sentiment in the classroom for increasing the information from and reliability of
    grading open-ended assignments in large courses.}
}