Data Quality in Terminology Work using Lange Language Models
Data quality is a crucial element in the organization and creation of terminological data in research and business. At the same time, tools such as agentic AI and Large Language Models become increasingly important in this context for optimizing processes and supporting humans in their work. However, to achieve a high degree of data quality of terminological data, these tools need a high-quality data foundation. After all, this data is used to facilitate the translation process and support knowledge management.
There are already plenty of articles stating that the quality of AI results literally depends on the quality of the underlying training data — there’s no question about that. But how can this quality be achieved? The following section presents methodologies, challenges, and best practices for consistent and precise terminology to enable successful knowledge transfer within organizations and effective terminology management.
Terminology Work Between Standards, Data Quality, and Large Language Models
What is data quality?
In the field of data management, data quality ensures that data is accurate, complete, consistent, traceable, and suitable for its intended use. When this is ensured, data can be exchanged between different systems without any data loss.
And what do data quality and terminology have in common?
Terminology work and data quality share similar goals: Both aim to ensure consistency, unambiguity, understandability, and traceability. While terminology work ensures that concepts, designations, and definitions are clearly structured and standardized, data quality addresses the question of whether this information is stored and managed in a complete, accurate, and reliable manner.
For this purpose, ISO 8000 for data management and ISO 704 for terminology work can be combined to form a new framework for the use of Large Language Models. ISO 704 defines the content requirements for terminology, such as the creation of concepts, concept relationships, designations, and definitions. ISO 8000, on the other hand, specifies quality requirements for data, such as completeness, consistency, traceability, and validity.
This results in the following quality categories for a new terminological framework that can be implemented into Large Language Models for generating responses:
- Accuracy and Correctness
- Coherence and Integrity
- Completeness
- Consistency
- Non-redundancy
- Reliability and Traceability
- Unambiguity
- Understandability and Relevance
- Verifiability and Validity
What role do Large Language Models play in terminology work?
A new approach to effectively integrating Large Language Models into terminology work is the Terminology-Augmented Generation (TAG) model, which is applicable to both monolingual and multilingual terminology work. TAG is based on the previously established Retrieval-Augmented Generation (RAG) model. Unlike RAG, which works with vector-based text segments, TAG draws on structured and table-based databases, ontologies, or glossaries. These resources can contain definitions as well as other standard-based knowledge and data categories, enabling LLMs to access domain-specific knowledge in a more targeted manner. This allows for the generation of terminological content that is more precise, consistent, and context-oriented.
Prompt Engineering in Terminology Work
While TAG and the targeted integration of terminological data can create a defined framework for the Large Language Model, a precise methodology is required to ensure reproducible and standardized results. For this reason, prompt engineering is becoming an important skill for translators and terminologists. For this purpose, specific prompts should be developed for terminological queries.
Example:
Zero-Shot Prompt
| You are a terminologist. Create a definition for the concept ‘employee’. |
| Few-Shot Prompt | Role: You are a terminologist. Action: Create a definition for the concept [example]. Characteristics: Based on the underlying business concept; not more than 25 words; includes the superordinate concept; must begin with a verb; must be intensional. Example: [based on the company standards] |
From the perspective of terminology studies and the requirements for terminological definitions, however, zero-shot prompts are not very effective. For example, they lack information on the structure of the definition, the technical context, a distinguishing characteristic, and the type of definition (intensional or extensional).
Such adjustments enable LLMs to better access a company’s terminological data and use it as a basis for generating context-appropriate and technically precise responses. Only by providing relevant terminological information does the model gain the necessary technical context to achieve the desired level of accuracy. In addition, it is recommended to define a style guide that contains clear guidelines for writing style, terminology, and quality requirements. This facilitates the standardization of prompts and forms an important foundation for the development of automated workflows and repetitive tasks in terminology management. It should be noted here that these findings were evaluated manually and without system integration.
Use Cases for Addressing Current Challenges in Databases
In addition to creating definitions, LLMs are also well-suited for quality assurance and the refinement of terminology databases. For example, an iterative approach can be applied: First, a project is set up or synchronized in an LLM environment that includes a detailed style guide, a standards-based framework with defined quality categories, and domain-specific guidelines for the respective subject area. The relevant terminological data is then provided as a knowledge base.
Based on that, the LLM can independently analyze the given definitions and compare them with the defined quality and terminology rules. In a related study, only one definition was entered; the model then identified deviations from the established standards and suggested appropriate corrections. This example illustrates that LLMs can not only assist in creating new content but also serve as a quality assurance tool by systematically reviewing existing terminology data and identifying opportunities for improvement — the foundation for high-quality data in unstructured databases.
Limitations and Challenges
The study shows that Large Language Models have great potential for the creation, analysis, and quality assurance of terminological data. At the same time, it becomes clear that their successful use depends on several prerequisites and framework conditions.
A key insight is that complex problems can rarely be optimally solved with a single query. Significantly better results are achieved through a step-by-step, interactive approach and training, in which requirements are clarified, interim results are reviewed, and additional context is provided. This not only improves the quality of the results but also enhances their understandability and technical validity.
One of the biggest challenges remains the risk of hallucinations. Large Language Models generally generate a response even when they lack important information or the underlying data is incomplete. Instead of transparently communicating uncertainties, they can generate content that is plausible but factually incorrect. This problem occurs particularly when terminology databases, knowledge graphs, or ontologies have gaps or when relevant information is unavailable.
Furthermore, the results presented here are based on an independent study in which the terminology data was manually entered into the LLM. In practice, therefore, there is the additional challenge of integrating such models into existing terminology and content management systems via a programming interface. Only through a reliable technical connection can terminology data be used automatically, and discrepancies and data silos be addressed and integrated into existing workflows.
In the future, therefore, it will not be the amount of available data alone that matters, but above all its relevance and the quality of its processing. Successful LLM applications require precise search queries, clearly defined prompts, and a sufficiently documented subject-matter context. The focus should be less on collecting as much data as possible and more on the targeted retrieval of information that is actually relevant to the task. Only under these conditions can LLMs reliably realize their potential as a supportive tool for terminology work.
Would you like to learn more about this topic? Contact Lisa Oest on LinkedIn [Link to: Lisa Oest | LinkedIn] to learn more about the methodology and findings from her master's thesis.
tl;dr
Successful LLM applications require precise search queries, clearly defined prompts, and a sufficiently documented subject-matter context.
An Article by
