ISCSR Research Publishing
Journal of Algorithmic Discovery and Applied AI

Self-Assessment Without Self-Deception in Machine Translation

Read & download PDF
Abstract

How can intrinsic scores be checked when generator and evaluator share representations? This review answers by treating confidence auditing as a property of a sociotechnical workflow rather than a feature that can be read from average accuracy. The focal setting is conditional translation revision, where speed and fluency can conceal semantic loss, correlated self-evaluation errors, or domain-specific failure. Evidence from the assigned publications is synthesized with foundational studies of calibration, distribution shift, causal structure, and responsible deployment. Four requirements follow: preserve the lineage of source texts, alternative drafts, reward signals, confidence estimates, domain corpora, and human judgments; measure stability across relevant perturbations; connect confidence to a specific action; and maintain a route for human challenge and correction. The framework distinguishes descriptive performance from decision utility and separates uncertainty about the world from uncertainty created by the model and its evaluator. It also shows why faster inference or richer reasoning is valuable only when it improves a defined decision under a transparent resource budget. The article is a literature review and research agenda, not a report of a newly completed trial.

Keywords
machine translationconfidencetranslationevaluatorhumandecisionuncertainty
References
  1. Zhang, Yin, et al. "SAINF: Intrinsic Self-Correction for Robust Machine Translation with Large Language Models." *Frontiers of Computer Science* (2026).
  2. Li, Yuanhao, et al. "DRAFT-RL: Multi-Agent Chain-of-Draft Reasoning for Reinforcement Learning-Enhanced LLMs." *Proceedings of the AAAI Conference on Artificial Intelligence* 40.35 (2026): 29530-29537.
  3. Sang, Yinghao. "Adaptive Quantization Strategies for Robust ML Inference Under Distribution Shift." *Proceedings of the 2026 5th International Conference on Cyber Security, Artificial Intelligence and Digital Economy* (2026): 362-368.
  4. Deng, Huilin, et al. "IIB-LPO: Latent Policy Optimization via Iterative Information Bottleneck." *arXiv preprint arXiv:2601.05870* (2026).
  5. Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the Knowledge in a Neural Network." *NIPS Deep Learning and Representation Learning Workshop*, 2015.
  6. Guo, Chuan, et al. "On Calibration of Modern Neural Networks." *Proceedings of the 34th International Conference on Machine Learning*, 2017, pp. 1321-1330.
  7. Geifman, Yonatan, and Ran El-Yaniv. "Selective Classification for Deep Neural Networks." *Advances in Neural Information Processing Systems*, vol. 30, 2017.
  8. Araci, Dogu. "FinBERT: Financial Sentiment Analysis with Pre-Trained Language Models." *arXiv preprint arXiv:1908.10063*, 2019.
  9. Loughran, Tim, and Bill McDonald. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks." *Journal of Finance*, vol. 66, no. 1, 2011, pp. 35-65.
  10. Gururangan, Suchin, et al. "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks." *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, 2020, pp. 8342-8360.
  11. Vaswani, Ashish, et al. "Attention Is All You Need." *Advances in Neural Information Processing Systems*, vol. 30, 2017.
  12. Sennrich, Rico, Barry Haddow, and Alexandra Birch. "Neural Machine Translation of Rare Words with Subword Units." *Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics*, 2016, pp. 1715-1725.
  13. Papineni, Kishore, et al. "BLEU: A Method for Automatic Evaluation of Machine Translation." *Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics*, 2002, pp. 311-318.
  14. Rei, Ricardo, et al. "COMET: A Neural Framework for MT Evaluation." *Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing*, 2020, pp. 2685-2702.
Publication details
Journal
Journal of Algorithmic Discovery and Applied AI
Volume
1 (2026)
Issue
1 ยท Forthcoming issue
Article number
jadai20260004
License
CC BY 4.0