When AI Can’t Repeat Itself: Building Governance for Non-Deterministic AI Systems
Abstract
Artificial intelligence is rapidly becoming operational infrastructure within enterprise software. AI agents now power chatbots, voice assistants, and automated telephony systems that interpret intent, generate responses, and execute workflows across customer service platforms. These systems operate probabilistically: identical prompts can produce different yet contextually valid outputs, challenging testing built on deterministic input-output validation.
This presentation examines how test architecture can evolve into a governance layer for non-deterministic AI systems. Conventional regression testing validates a single expected response, whereas AI requires distribution-based behavioral evaluation. I introduce the Scenario-Driven Behavioral Evaluation Framework (SBEF), an architectural testing approach executing hundreds of interaction scenarios across chat, voice, and telephony channels, including ambiguous prompts, adversarial inputs, and multi-turn conversations that expose hallucination, contextual breakdown, bias signals, and policy violations.
SBEF measures AI behavior using metrics including task-completion rate, unsupported-claim frequency, policy-violation rate, contextual retention, and interaction efficiency. Repeated execution across scenario sets produces behavioral distributions that let engineers define reliability and safety thresholds. The talk also covers observability pipelines capturing traces and evaluation scores to detect drift and coherence degradation, helping organizations move from deterministic validation toward continuous governance for AI systems in open-ended environments.
- Published in
- Proceedings of the Raptors Conference, Vol. 1 (2026)
- Conference
- Raptors Conference 2026, 21 October 2026, online
- Accepted
- 2026-09-10
- DOI
- https://doi.org/10.68495/rc.2026.C041
- Presentation
- Slides (PDF)
- Licence
- © 2026Mani Deep Reddy Singireddy. Published under Creative Commons Attribution 4.0 International (CC BY 4.0).
- Publisher
- Hackathon Raptors, Community Interest Company 15557917, London, United Kingdom
- Permanent address
- https://proceedings.raptors.dev/singireddy-governance-non-deterministic-ai
Open access. Every entry in the Proceedings is free to read, with no charge to authors and no charge to readers. Slides are served from this site, so they stay available at the paper's permanent address.
Cite this paper
Mani Deep Reddy Singireddy (2026). When AI Can’t Repeat Itself: Building Governance for Non-Deterministic AI Systems. Proceedings of the Raptors Conference, Vol. 1 (2026). Hackathon Raptors, London. https://doi.org/10.68495/rc.2026.C041
@inproceedings{raptors2026-c041,
title = {{When AI Can’t Repeat Itself: Building Governance for Non-Deterministic AI Systems}},
author = {Singireddy, Mani Deep Reddy},
booktitle = {Proceedings of the Raptors Conference},
volume = {1},
year = {2026},
publisher = {Hackathon Raptors},
address = {London, United Kingdom},
doi = {10.68495/rc.2026.C041},
url = {https://doi.org/10.68495/rc.2026.C041},
abstract = {Artificial intelligence is rapidly becoming operational infrastructure within enterprise software. AI agents now power chatbots, voice assistants, and automated telephony systems that interpret intent, generate responses, and execute workflows across customer service platforms. These systems operate probabilistically: identical prompts can produce different yet contextually valid outputs, challenging testing built on deterministic input-output validation. This presentation examines how test architecture can evolve into a governance layer for non-deterministic AI systems. Conventional regression testing validates a single expected response, whereas AI requires distribution-based behavioral evaluation. I introduce the Scenario-Driven Behavioral Evaluation Framework (SBEF), an architectural testing approach executing hundreds of interaction scenarios across chat, voice, and telephony channels, including ambiguous prompts, adversarial inputs, and multi-turn conversations that expose hallucination, contextual breakdown, bias signals, and policy violations. SBEF measures AI behavior using metrics including task-completion rate, unsupported-claim frequency, policy-violation rate, contextual retention, and interaction efficiency. Repeated execution across scenario sets produces behavioral distributions that let engineers define reliability and safety thresholds. The talk also covers observability pipelines capturing traces and evaluation scores to detect drift and coherence degradation, helping organizations move from deterministic validation toward continuous governance for AI systems in open-ended environments.}
}
TY - CONF TI - When AI Can’t Repeat Itself: Building Governance for Non-Deterministic AI Systems AU - Singireddy, Mani Deep Reddy T2 - Proceedings of the Raptors Conference VL - 1 PY - 2026 DA - 2026/09/10 PB - Hackathon Raptors CY - London, United Kingdom DO - 10.68495/rc.2026.C041 UR - https://doi.org/10.68495/rc.2026.C041 AB - Artificial intelligence is rapidly becoming operational infrastructure within enterprise software. AI agents now power chatbots, voice assistants, and automated telephony systems that interpret intent, generate responses, and execute workflows across customer service platforms. These systems operate probabilistically: identical prompts can produce different yet contextually valid outputs, challenging testing built on deterministic input-output validation. This presentation examines how test architecture can evolve into a governance layer for non-deterministic AI systems. Conventional regression testing validates a single expected response, whereas AI requires distribution-based behavioral evaluation. I introduce the Scenario-Driven Behavioral Evaluation Framework (SBEF), an architectural testing approach executing hundreds of interaction scenarios across chat, voice, and telephony channels, including ambiguous prompts, adversarial inputs, and multi-turn conversations that expose hallucination, contextual breakdown, bias signals, and policy violations. SBEF measures AI behavior using metrics including task-completion rate, unsupported-claim frequency, policy-violation rate, contextual retention, and interaction efficiency. Repeated execution across scenario sets produces behavioral distributions that let engineers define reliability and safety thresholds. The talk also covers observability pipelines capturing traces and evaluation scores to detect drift and coherence degradation, helping organizations move from deterministic validation toward continuous governance for AI systems in open-ended environments. LA - en ER -