The Language Product Evaluation Tool

The Association for Information Management: Translating and the Computer Conference – 17 & 18 November 2011

ASLIB

The Language Product Evaluation Tool

Allison Blodgett and Erica Michael, University of Maryland Center for Advanced Study of Language (CASL)

Language Product Evaluation Tool (LPET)

LPET was developed to establish standards for creating language products in line with Intelligence Community analytic standards; to enable quality reviewers to more easily provide consistent, systematic feedback; and to provide aggregated data to describe organizational capability and performance.

Reasons:

  • No widely accepted standards for what constitutes a good translation in US government (USG) context
  • No standard method for evaluation of USG language products

Goals:

  • Establish translation standards
  • Facilitate the quality review process
  • Provide information for professional development
  • Provide aggregate data

Guiding Principles:

  • Flexibility of use
  • The quality of the language product

Three main components of product quality (LPET Schema):

  • Language performance
  • Analysis
  • Presentation

LPET Training:

  • 8-hour workshop
  • Exercises in evaluating sample language products
  • Focus on indentifying appropriate checkbox for each problem and rating

Findings:

 

Language analysts and quality reviewers report that LPET is easy to use and will be useful for evaluating translations in an operational context. Quality reviewers piloting the LPET found the following:

  • Individual components of the LPET are intuitive;
  • Documenting feedback with the LPET does not add much time to the quality review process;
  • The LPET helps increase awareness of the relationship between source characteristics and product quality; and
  • The LPET is likely to make feedback more accurate, objective, detailed, systematic, and consistent.

Ongoing research is examining the reliability, validity, and usability of the LPET.