Observational AI
What it is, where it sits next to your platforms and your governance, and the four things it hands an enterprise willing to look closely at what its own systems are actually doing.
Start reading ↓Five blind spots in most AI programs
Before anyone talks about a platform, a policy, or a pilot, it's worth naming what's actually happening on the ground inside most enterprises right now.
Missing AI strategy
Pilots multiply faster than the strategy meant to guide them. Enterprises end up with a dozen use cases and no shared answer for which ones matter, why, or what "done" looks like.
Agentic sprawl
Agents are being stood up team by team, each with its own tools, memory, and permissions. Nobody owns the full map of what's now acting on the business's behalf.
Bubbling cost
Token and compute spend is climbing faster than anyone budgeted for, and quietly — scattered across teams, tools, and invoices no one is looking at side by side.
Shift in regulations
The EU AI Act, state-level rules, and shifting federal guidance are moving faster than most governance programs can track, let alone prove compliance with.
Shadow AI
Employees are already using AI tools IT never approved, on data no one's reviewed. The unofficial AI footprint is usually bigger than the official one.
What is Observational AI?
Most AI programs get evaluated the way a brochure gets evaluated: by what it claims. A vendor demo. A benchmark score. A roadmap slide. All of it describes what a system is supposed to do.
Observational AI sets the claims aside. It studies a system the way a field scientist studies a live population — through direct, continuous observation of what it's actually doing once it's running in your environment, on your data, against your real traffic.
That distinction matters more than it sounds. A model that scores well on a public benchmark can still be slow in your architecture, expensive at your volume, exposed at your integration points, or simply disliked by the people using it. None of that shows up in a demo. All of it shows up in observation.
Where it fits
Observational AI isn't a replacement for platform strategy, Responsible AI, or governance. It's the connective layer between them — the discipline that tells you whether the platform, the principles, and the policy are actually holding up once real usage begins.
The tools, models, and infrastructure an enterprise chooses to run on.
The principles — fairness, transparency, safety — an organization commits to on paper.
The policies, controls, and accountability structures meant to enforce those commitments.
The ongoing, evidence-based check that platforms, principles, and policy are actually functioning together in practice — not just in the deck.
Four pillars of what observation gives you
Once you're actually watching a system in production, four things become measurable that were previously guesswork.
Performance
Latency, throughput, drift, and quality across the full inference pipeline. A model that looks great in a sandbox can degrade the moment it meets real concurrency, real context length, and real production traffic — observation is how you catch that before your users do.
ROI
Cost attribution and unit economics, tied to a business-value framework finance and engineering can both stand behind. Observation turns AI spend from a line item nobody can explain into a number connected directly to the workflows and outcomes producing it.
Risk
Security exposure, regulatory compliance, and the audit trail to prove both. Observation maps a system's live behavior against frameworks like the OWASP LLM Top 10, the EU AI Act, and NIST's AI RMF — producing the evidence trail auditors and regulators actually ask for.
Responsibility
Fairness, bias, and hallucination monitoring, backed by human-in-the-loop oversight where it matters most. Observation is how an organization shows — not just claims — that a system is behaving the way it's supposed to for the people it affects.
Thirty years of watching systems work
This part, I'll tell in my own words.
I've spent thirty years as a deep technology practitioner, long before "AI" was the word on every slide. I've built systems, integrated them, and watched them succeed and fail in front of real users under real load — across engagements at Atos, Deloitte, and Maven Wave, and observational research practice grounded at Caltech CTME.
What thirty years teaches you, if you're paying attention, is that technology rarely fails the way the pitch deck warned you it would. It fails in the gap between what a system was designed to do and what it's actually doing once real people, real data, and real money are running through it. That gap is where I've spent my career — and it's the reason Brian Ray LLC exists: to close it before it becomes expensive, embarrassing, or both.
Today, I lead Observational AI and complex data and AI initiatives, helping enterprises shape strategy. When an engagement requires broader delivery capacity, I bring in an extended network of trusted specialists and delivery partners across data, AI, and cloud. That gives clients a single senior point of accountability, while letting the delivery model scale to match what the engagement actually needs.
Video not loading? Watch it on Vimeo.
Look before you leap
Before you sign anything, a few questions are worth answering honestly:
- Where will AI actually create measurable value?
- How do we turn fragmented data into an AI-ready foundation?
- Build, buy, partner, or platform?
- Do we have the right talent and operating model to execute?
No slide deck. No sales team. Just a clear look at what's really there.
I advise enterprise leaders navigating complex data and AI challenges -- where the opportunity is significant, but the path from strategy to measurable business outcomes isn't always clear. This often means helping clients identify the right AI opportunities, make sense of fragmented data and technology choices, move initiatives beyond pilots, and address execution gaps.