Evaluating Information Visualization in Large Companies
Researchers
Summary
This project is a result of the PhD work of Michael Sedlmair. It summarizes a number of challenges and recommendations arising in the context of information visualization evaluation in large companies with several thousand employees. Our papers describe a collection of evaluation challenges, discuss our own experiences conducting information visualization evaluation within the context of a large automotive company, and present a set of recommendations derived from our experiences. The set of challenges and recommendations can aid researchers and practitioners in preparing and conducting evaluations of their products
within a large company setting.
Challenges and Recommendations
In the paper we list and explain the following challenges and recommendations for evaluating information visualization or visual analytics tool in a large company context:
Challenges:
- Integrating Tools in Daily Work Processes
- Getting the Data
- Choosing an Evaluation Context
- Finding Domain Expert Participants
- Attachment to Conventional Techniques
- Confidentiality of Information
- Complex Work Processes
- Convincing the Stakeholders
- Publishing
Recommendations
- Familiarize yourself with currently used software environments
- Overcome technical obstacles of data integration
- Choose your study environment with care
- The magic one hour limit
- Convince your target audience
- Delight with usability and aesthetics
- Learn from the experts
- Try to get a license, do studies in any case
- Be in constant, close cooperation
- The magic metric: Money
- Factor in high skill with current techniques
- Clarify publishing conditions upfront
References
Michael Sedlmair, Petra Isenberg, Dominikus Baur, and Andreas Butz (2011) Information Visualization Evaluation in Large Companies: Challenges, Experiences and Recommendations. Information Visualization Journal, 10(3):248–266, July 2011.
|
|
×
BibTeX entry:
@ARTICLE{Sedlmair:2011:EIV,
author = {Michael Sedlmair and Petra Isenberg and Dominikus Baur and Andreas Butz},
title = {I}nformation {V}isualization {E}valuation in {L}arge {C}ompanies:
{C}hallenges, {E}xperiences and {R}ecommendations,
journal = {Information Visualization Journal},
year = {2011},
volume = {10},
number = {3},
month = jul,
pages = {248--266},
doi = {10.1177/1473871611413099},
doi_url = {https://doi.org/10.1177/1473871611413099},
oa_hal_url = {https://hal.inria.fr/inria-00638541},
pdf = {../publications/papers/Sedlmair_2011_IVE.pdf},
}
|
Michael Sedlmair, Petra Isenberg, Dominikus Baur, and Andreas Butz (2010) Evaluating Information Visualization in Large Companies: Challenges, Experiences and Recommendations. In Proceedings of the CHI Workshop Beyond Time and Errors: Novel Evaluation Methods for Information Visualization (BELIV). ACM Press, New York, pages 79–86, 2010. Received the best paper award at BELIV 2010.
|
|
×
BibTeX entry:
@INPROCEEDINGS{Sedlmair:2010:EIV,
author = {Michael Sedlmair and Petra Isenberg and Dominikus Baur and Andreas Butz},
title = {E}valuating {I}nformation {V}isualization in {L}arge {C}ompanies: {C}hallenges, {E}xperiences and {R}ecommendations,
booktitle = {Proceedings of the CHI Workshop Beyond Time and Errors: Novel Evaluation Methods for Information Visualization (BELIV)},
year = {2010},
pages = {79--86},
publisher = {ACM Press},
address = {New York},
doi = {10.1145/2110192.2110204},
doi_url = {https://doi.org/10.1145/2110192.2110204},
oa_hal_url = {https://hal.inria.fr/inria-00587680},
pdf = {../publications/papers/Sedlmair_2009_EIV.pdf},
}
|
This is a related paper which talks about the evaluation process in context of a specific project:
Michael Sedlmair, Petra Isenberg, Dominikus Baur, Michael Mauerer, Christian Pigorsch, and Andreas Butz (2011) Cardiogram: Visual Analytics for Automotive Engineers. In Proceedings of the Conference on Human Factors in Computing Systems (CHI). ACM, New York, pages 1727–1736, 2011.
|
|
×
BibTeX entry:
@INPROCEEDINGS{Sedlmair:2011:CVA,
author = {Michael Sedlmair and Petra Isenberg and Dominikus Baur and Michael Mauerer and Christian Pigorsch and Andreas Butz},
title = {Cardiogram: Visual Analytics for Automotive Engineers},
booktitle = {Proceedings of the Conference on Human Factors in Computing Systems (CHI)},
year = {2011},
pages = {1727--1736},
publisher = {ACM},
address = {New York},
doi = {10.1145/1978942.1979194},
doi_url = {https://doi.org/10.1145/1978942.1979194},
oa_hal_url = {https://hal.inria.fr/inria-00638544},
pdf = {../publications/papers/Sedlmair_2011_CAV.pdf},
}
|
back to the projects page