Improved multithreaded unit testing

By Vilas Jagannath, Milos Gligoric, Dongyun Jin, Qingzhou Luo, Grigore Rosu, and Darko Marinov. In SIGSOFT FSE, 223–233. ACM, 2011.

Publisher Link:

http://doi.acm.org/10.1145/2025113.2025145

Abstract

Multithreaded code is notoriously hard to develop and test. A multithreaded test exercises the code under test with two or more threads. Each test execution follows some schedule/interleaving of the multiple threads, and different schedules can give different results. Developers often want to enforce a particular schedule for test execution, and to do so, they use time delays (Thread.sleep in Java). Unfortunately, this approach can produce false positives or negatives, and can result in unnecessarily long testing time.

This paper presents IMUnit, a novel approach to specifying and executing schedules for multithreaded tests. We introduce a new language that allows explicit specification of schedules as orderings on events encountered during test execution. We present a tool that automatically instruments the code to control test execution to follow the specified schedule, and a tool that helps developers migrate their legacy, sleep-based tests into event-based tests in IMUnit. The migration tool uses novel techniques for inferring events and schedules from the executions of sleep-based tests. We describe our experience in migrating over 200 tests. The inference techniques have high precision and recall of over 75%, and IMUnit reduces testing time compared to sleep-based tests on average 3.39x.

BibTeX

@inproceedings{conf/sigsoft/JagannathGJLRM11,
    author = "Jagannath, Vilas and Gligoric, Milos and Jin, Dongyun
              and Luo, Qingzhou and Rosu, Grigore and Marinov, Darko",
    editor = "Gyimóthy, Tibor and Zeller, Andreas",
    title = "Improved multithreaded unit testing",
    booktitle = "SIGSOFT FSE",
    crossref = "conf/sigsoft/2011",
    ee = "http://doi.acm.org/10.1145/2025113.2025145",
    keywords = "software engineering",
    pages = "223-233",
    year = "2011",
}

@proceedings{conf/sigsoft/2011,
    editor = "Gyimóthy, Tibor and Zeller, Andreas",
    title = "SIGSOFT/FSE'11 19th ACM SIGSOFT Symposium on the
             Foundations of Software Engineering (FSE-19) and ESEC'11: 13rd
             European Software Engineering Conference (ESEC-13), Szeged,
             Hungary, September 5-9, 2011",
    isbn = "978-1-4503-0443-6",
    publisher = "ACM",
    year = "2011",
}