As AI agents gain autonomy, the need for explainability and transparency has never been more urgent. Four industry experts recently came together to explore the critical challenges of building trust in agentic systems, sharing insights from their work at Prudential Financial, LinkedIn, Parallaxis, and Agilent Technologies.
The conversation revealed a startling reality: while AI agents promise to automate complex workflows, their increasing autonomy creates new risks that traditional guardrails cannot address. From data leakage vulnerabilities to emergent behaviors in multi-agent systems, the panelists outlined both the promise and peril of autonomous AI.
Saradha Nagarajan, Senior Data Engineer at Agilent Technologies, emphasized that explainability begins with trust. “The trust and adaptability you have in the data or predictions from an agentic AI model is much better when you understand what’s happening behind the scenes,” she explained.
For agentic systems to succeed in enterprise environments, they need more than just technical capabilities. They require clearly defined ethical guidelines, robust observability layers, and comprehensive auditing mechanisms that function both before and after deployment.
This foundation becomes even more critical when considering regulatory requirements. Pankaj Agrawal, Staff Software Engineer at LinkedIn, noted that in regulated industries, transparency isn’t optional - it’s mission-critical. “Even with agentic AI, you need to ensure the agent has taken the steps it was supposed to,” he said. “It shouldn’t deviate from the graph it’s meant to follow.”
While ethical considerations dominate many AI discussions, Dan Chernoff, Data Scientist at Parallaxis, offered a more pragmatic perspective. “I don’t think it’s necessarily about ethics,” he argued. “It’s about governance and how your systems align with the rules that apply in your environment.”
This governance-first approach focuses on concrete compliance requirements around personal data, sensitive information, and auditing standards. When AI systems fail or behave unexpectedly, organizations must be able to:
Keshavan Seshadri, Senior Machine Learning Engineer at Prudential Financial, reinforced this point by highlighting how the EU AI Act provides practical frameworks for risk assessment. “Europe has always been the front-runner on regulation,” he noted. “The EU AI Act tells us what counts as acceptable risk, low risk, high risk, and what’s completely unacceptable.”
Building transparent agentic systems requires more than technical expertise. The panelists emphasized that successful AI deployment demands collaboration across business functions, with clear buy-in from leadership and active participation from non-technical stakeholders.
LinkedIn’s approach exemplifies this collaborative model. Agrawal described how his team created a playground environment where business users can experiment with prompts and observe model outputs firsthand. “That way, they can see what the model produces and what its limitations are,” he explained.
This hands-on approach serves multiple purposes: it grounds stakeholder expectations in reality, builds shared understanding of AI capabilities and limitations, and ensures that governance decisions align with actual business needs rather than theoretical concerns.
As AI systems become more autonomous, they introduce challenges that traditional machine learning governance cannot adequately address. Multi-agent systems, in particular, create new categories of risk through emergent behaviors and complex interaction patterns.
Nagarajan warned about the unpredictable nature of agent-to-agent interactions: “When agents are interacting with each other, you can get outputs that were never anticipated. That’s the danger of emergent behavior.”
These emergent behaviors can manifest in several ways:
The panel offered several concrete strategies for building more transparent agentic systems:
Rather than retrofitting transparency features, successful organizations build observability into their system architecture. This includes comprehensive logging, tool-level traceability, and clear audit trails that capture both successful operations and failures.
Effective governance combines automated safeguards with human oversight. This might include AI supervisors that audit agent decisions, evaluation frameworks that test system performance, and rollback mechanisms for when agents exceed their intended parameters.
While complex models offer sophisticated reasoning capabilities, simpler rule-based systems often provide better explainability. The key is finding the right balance for each use case, potentially using hybrid approaches that combine both methodologies.
Generic AI solutions rarely meet the transparency requirements of specific industries or use cases. Organizations should invest in domain-specific agents that can be more easily explained and audited within their particular context.
Looking ahead, the panelists identified several emerging trends that will shape the development of transparent agentic systems:
Evaluation-Driven Development: Rather than optimizing for perfect prompts, organizations are shifting focus to robust evaluation frameworks that ensure consistent performance even as underlying models evolve.
Human-AI Collaboration: The future lies not in fully autonomous systems but in collaborative arrangements where AI handles routine tasks while humans maintain oversight of critical decisions.
Industry-Specific Frameworks: As agentic AI matures, we can expect more industry-specific transparency standards that address the unique requirements of healthcare, finance, manufacturing, and other regulated sectors.
The expert panel revealed several crucial insights for organizations implementing agentic AI:
As AI agents become more capable and autonomous, the challenge of maintaining transparency and trust becomes increasingly complex. However, the strategies outlined by these industry experts provide a roadmap for organizations seeking to deploy agentic systems responsibly.
The key lies in recognizing that transparency isn’t just a technical problem - it’s an organizational challenge that requires collaboration across disciplines, clear governance frameworks, and a commitment to building trust through demonstrable reliability rather than theoretical assurances.
Organizations that invest in these foundations now will be better positioned to leverage the full potential of agentic AI while maintaining the trust of users, regulators, and other stakeholders. In an age where AI systems increasingly operate with human-level autonomy, this trust may be the most valuable asset of all.
25 September 2025
