ROTHMAN Steven B.    Department   Ritsumeikan Asia Pacific University  College of Asia Pacific Studies    Position   Associate Professor
Language	English
Publication Date	2007/11
Type	Research paper (Academic/Professional Journal)
Peer Review	Peer reviewed
Title	Understanding Data Quality Through Reliability: A Comparison of Data Reliability Assessment from Three Treaty Datasets
Contribution Type	Single Work
Journal	International Studies Review
Volume, Issue, Page	9(Fall),pp.437-456
Details	Although recent data creation efforts in international relations have begun to focus on issues of reliability and validity more explicitly than previously, current efforts still contain significant problems. This essay focuses on three recent data generation projects that study international relations (the ICOW, ATOP, and River Treaty datasets) and shows the successes and failures of each in assessing reliability when generating data from qualitative evidence. All three datasets attempt to generate reliable data, document the procedures used, and present indications of data reliability. However, their efforts face problems when assessing the reliability of their case selection variables, in the development of reliability indicators, and in the presentation of reliability statistics. In addition to evaluating these recent efforts to generate large-N databases, this essay clarifies the difference between generating data from qualitative and quantitative evidence, explains the importance of reliability when coding qualitative evidence, and provides ways to improve the assessment of the quality of one’s data.