It doesn’t matter if we’re talking about a loaf of bread, a new car or anything else: quality is something we all want. It speaks to excellence, elicits pride and doesn’t unravel on you. But how can quality be guaranteed? How can you put your finger on it?
When it comes to translating assessments, the greatest proof of quality is that the translation is so good it will never get you into trouble. Or put another way, when stakeholders and test takers alike find no fault.
To achieve that level of quality, we developed a best practice course of action with seven QA steps for translating assessments. It includes finding and engaging the processes, people, resources and company culture that produce trouble-free translation in our own work, and when we audit and correct the work of other translators (our specialty).
This is a simple version of how we do it, and how we think you should be doing it too.
Process serves as our roadmap: after all, flawed processes create flawed products. Certifying process is the best way to keep performance on the straight and narrow. While a number of certification standards exist for translation, with more on the way, our pick is a rigorous implementation of ISO 9001 QA standards for continuous quality improvement.
We live and breathe ISO 9001 certification to promote a culture where everyone is accountable. After all, everybody has to be on their game for continuous quality improvement to really work. ISO 9001 certification requires regular audits and ongoing review from outside parties as well as an internal duty to solve problems, create and constantly improve systems, and keep adequate records. The performance scores we solicit from customers are the key performance indicators we use when evaluating and compensating teams.
Successes, failures and effectiveness are regularly monitored for continuous improvement. Anything less than great work, as defined by customers, employees and/or providers, automatically triggers an investigation that examines the root cause and prompts corrective actions to be taken. That’s how we keep getting better.
A successful project begins with a team of the right people moving together in the same direction.
The project manager elicits and confirms all of the project’s requirements and specifications with the client. Once agreed, the project manager starts to select the subject matter experts, translators, translation editors and psychometrically trained linguists that will be needed. If a faster turnaround time than normal is required, the project manager coordinates the larger teams needed to meet tight deadlines.
The linguists who work on any assessment translation project are chosen for their expertise and specialization in testing and/or in the exam’s subject as well as for their qualifications and experience in the particular role they will play in the project. Translators work into their native language and are members of the American Translators Association or equivalent. Psychologists and trained assessment specialists work in their native language to ensure cognitive and psychometric integrity.
Every project depends on having the right tools. When it comes to translating assessments, that means starting with a glossary. The glossary ensures that all terms are correctly and consistently rendered during the translation phase.
First, the subject matter experts examine past translations and any other available reference materials provided by the client as well as the test or assessment to be translated. Next, they identify and review the most important and most used terms, and the concepts behind each test item. Then the subject matter experts define the terms to create a draft glossary.
Once the client has approved the glossary, the files are aligned and the final terminology base to be used during translation is created. The glossary helps produce a more accurate, more consistent translation and can be used in your future translation projects.
The core work of a quality translation is the three-step TEP process: translation, editing and proofreading. Each step is a central component in ensuring that the translation meets the highest standard.
The main translator begins by reviewing all available reference material and then translates with the aid of translation memory software and a solid terminology base. While it is important to stand behind the translator’s work, the power of multiple eyes to check, refine and improve is critical.
Next, the editor reviews the initial translation against the source documents, glossaries and style guides. She checks for grammar, word choice and clarity as well as accuracy, consistency and balance to improve the translation.
Finally, the proofreader reviews and refines the translation. She resolves errors and inconsistencies, paying special attention to cultural appropriateness and ease of understanding. At least two rounds of proofreading are done so that the translation is at its best before being submitted for review.
It’s important for test takers in any of the languages offered to enjoy a level playing field. Will the translation perform like it’s supposed to?
Validation is the best way to assess the validity, reliability and equivalence of the translation. Different validation methods are available, but one solid process includes cognitive debriefing and subject matter expert (SME) review.
Cognitive debriefing shines light on how balanced the exam is and identifies areas for improvement. The translation is tested in the field on a sample group of the target population and evaluated using interviews, focus groups and/or pilot testing data. The translation is analyzed for cultural relevance and appropriateness as well as clarity and ease of understanding. This includes the instructions, test items and response format.
Subject matter expert review in this workflow helps further identify where and how the translation should be refined. Based on the pilot testing data and her own subject matter expertise, the SME reviews the translation and ensures the assessment is appropriate, easy to understand and uses correct terminology. She makes suggestions for additional improvements.
After validation, harmonization is often the next step. Like validation, harmonization is tailored to each project’s requirements and results in a more accurate exam.
For single-language projects, this process may start with the client, who reviews the validation results and indicates to the harmonization editor whether the recommendations contained there should stand. The editor incorporates the agreed-upon recommendations into the next draft. She also works with the client to find the best resolution in the event there are any conflicting edits.
For multiple-language projects, an international harmonization process is frequently the best approach–putting a common standard in place. It guarantees greater consistency, effectiveness and comparability across all language versions of a test or assessment. The selected harmonization team compares each item in every language version with items from the original test. The goal is to make sure that each assessment is understood in the same way by test takers of all the different language versions. The international harmonization recommendations are then subject to harmonization review by each individual language team. If it is found necessary, another version of the translation is created with their recommendations in mind.
The final quality assurance check is where the project reaches its full potential and the translation is confirmed to be ready for the world. During this step, we begin with multiple automated quality assurance checks before a third editor’s eagle eyes put on the finishing touches and verify that everything is right.
The third editor compares all the original source files with the translation. Her task includes ensuring that every document is consistently formatted, laid out and complete. Then the project manager submits the final translation to the client.
As you can see, quality is not created by accident. The quality of a translated assessment is the result of a meticulous and thorough approach to ensuring flawless, trouble-free assessments. We hope you have enjoyed this report and can put this best practice course of action with seven QA steps to work for you.
Responsive Translation specializes in the translation, adaptation, validation and review of high-stakes assessments. To discuss your organization’s requirements, please contact Ken Clark at [email protected] or 212-818-1102.