Infrastructure for Emergence

How to build AI systems that get smarter as reality changes—without collapsing under their own history

Photo taken by the author - Almont, CO where the trees have eyes

I have spent a long time in rooms where people speak about systems as if they can be finished.

The model was trained.
The pipeline was built.
The platform shipped.

Each declaration carried the same quiet claim: the hard part was behind us.

It never was.

When the inputs and outputs of a system are alive—changing, drifting, refuting their own past—”done” is a category error. And yet the technology industry keeps trying to treat data products like appliances instead of memory systems. That conflation isn’t theoretical. It warps how companies are structured, how people are valued, how risk is calculated, and how failure is explained away.

I carried this dissonance for years before I stopped pretending it was subtle.

---

The question I always asked in M&A—and what’s changed since

In mergers and acquisitions, when an AI company pitched its “secret sauce,” I had one follow-up:

How do you engineer your data and knowledge pipelines?

Not the model. Not the UI. The substrate.

This was before LLMs became ubiquitous. Back then, the answers were depressingly consistent:

“Oh, that’s done in India.”
“We offshore that part to Costa Rica.”
“A services team handles that.”

Translation: knowledge and data engineering were treated as operational labor, not intellectual architecture. A cost center. A detail. Something assumed to be finite and outsourceable.

Now? The pattern has metastasized in a different direction.

Now people outsource their data engineering to LLMs themselves—letting GPT-4 or Claude generate schemas, write transformation logic, create taxonomies—without recognizing that these models are fundamentally interpolative machines trained on frequency distributions. They give you the most probable answer, not the most precise one. They smooth over the conceptual asperities that matter most.

And the annotation work? That still gets shipped to Kenyan workers paid by the task, clicking through endless classification interfaces with minimal context about why any of it matters. The human judgment that should be shaping the knowledge architecture has been reduced to piecework—stripped of the surrounding understanding that makes judgment meaningful.

So we’ve gone from outsourcing to humans we undervalue, to outsourcing to machines that can’t actually understand what they’re doing, back to humans we undervalue even more—now mediated through platforms that extract the reasoning from the labor.

In both cases, then and now, I usually knew the truth: they did not understand what they were actually building. They understood how to raise money for it.

And here’s what still rankles: they were outsourcing the only part that actually mattered. The part where meaning gets made. Where context gets preserved. Where the ontology either holds or collapses under interrogation.

You can’t outsource epistemology—not to underpaid contractors, not to stochastic parrots—and expect the system to reason.

Photo taken by the author - Felton, CA a crownshy redwood

---

Information architecture is not a phase

There is a reason that in large enterprises you’ll find roughly one information architect for every thousand application architects. That ratio signals difficulty, not importance.

Application architecture governs execution.
Information architecture governs meaning.

Meaning is harder. Meaning is recalcitrant.

If you want to structure data with integrity, you have to make your implicit choices explicit: scope, temporality, provenance, authority, interpretation. You must decide not only what something is, but when it was true, for whom it was true, and under what assumptions it can be reused.

This is why serious data warehousing in the Bill Inmon lineage is not a weekend course. It is a discipline that matures over decades, accumulating both technique and humility. In these systems, time is not just an attribute; it is the foundational dimension—the armature around which all other meaning is constructed. Miss that, and everything downstream becomes brittle theater.

You can always tell when temporality was bolted on as an afterthought. The system looks powerful right up until the first contradiction surfaces. Then the whole edifice starts to prevaricate with a straight face.

I’ve watched this play out in healthcare over and over. A protocol from 2019 gets surfaced as if it’s current guidance. A contraindication that expired gets treated as eternal verity. The AI doesn’t know it’s dissembling because nobody told it how to think about time.

---

The rarest experts are hiding in plain sight

Data architects. Knowledge managers. Ontologists. Taxonomists.

These roles do not scale like headcount in sales. They do not show up neatly in quarterly metrics. They rarely appear on pitch decks.

They are also vanishingly rare.

The people who actually know how to organize knowledge over time, across contexts, and for multiple audiences are often found in a building most tech executives haven’t visited since university: the library.

Librarians are trained to preserve meaning under change. They think in lineage, in versions, in authority files, in circulation and access. They know classification is never neutral and that every retrieval interface encodes a theory of use. They practice a form of custodianship that is simultaneously archival and anticipatory—they preserve what was while preparing for what might be needed.

This is intellectual infrastructure maintenance. It’s curatorial labor. It’s the work of keeping memory navigable as it grows.

The industry is trying to automate what it never bothered to deeply understand.

And look—I get the impulse. Really, I do. The fantasy is seductive: build the platform once, train the model, let it run. Hire a data science team, point them at the problem, ship the product. Scale through automation. Or better yet, ask ChatGPT to write your schema and call it “AI-native development.”

But here’s what that fantasy misses: knowledge work doesn’t scale through abstraction away from context. It scales through infrastructure that brings the people who hold the context closer to the tools that shape the output. It scales through recognizing that the rarest, most valuable work is the interpretive labor that happens at the boundaries—between clinical language and lived experience, between expert knowledge and novice comprehension, between what the documentation says and what people actually need to understand.

---

A small distinction that changes everything

Here is a line I use often because it lands:

What’s the difference between a knowledge graph and a social network?
In a knowledge graph, the relationships actually mean something.

A social network records proximity. A knowledge graph asserts semantics. It makes claims about how concepts relate, under which conditions, and under what constraints.

This is why ontologies matter—and why they are so frequently miscast as incantatory magic.

Ontologies do not solve your problems.
They make your problems finally visible.

They surface ambiguity. They expose inconsistency. They force decisions that teams would rather defer indefinitely. They puncture the fantasy that you can design a data system once and then simply “operate” it in perpetuity.

An ontology is not a panacea; it is a mirror with version control and an unforgiving memory.

Photo Credit to my friend Phil Komarny - Colorful Colorado

---

Why we built Bast the way we did

This is where Bast comes in, and why we’re not building yet another AI platform that promises to “solve” knowledge management through bigger models or better prompts.

We’re building infrastructure that puts domain experts back in control of their own knowledge systems.

Not data scientists. Not consultants. Not LLMs generating plausible-sounding taxonomies. The people who actually know what the knowledge means.

Because here’s the thing about complexity: you can’t manage it from outside the system. You can’t abstract your way to emergence. A good complexity theorist knows you work with the specific context, not against it. You create conditions where the people embedded in that context can cultivate the patterns that matter—and modify them as reality shifts.

That’s what our ontology-grounded infrastructure does. It gives subject matter experts—clinicians, researchers, librarians, anyone who holds institutional knowledge—the tools to build their own explainable AI systems. To encode their own semantics. To define what relationships mean in their world, under their conditions, with their constraints visible and traceable.

We’re not trying to replace their judgment. We’re trying to scale it without eviscerating the lineage.

In our pilot at Craig Hospital, we’re learning something that feels genuinely revelatory: the domain experts need to define the target audience and curate the reference materials, yes—but then the interaction data reveals insights we never anticipated. Which content actually resonates. What vocabulary chasms exist between clinical language and lived experience. How comprehension pathways bifurcate across recovery phases. The friction between documentation and understanding is where new knowledge germinates.

And the metadata we can create when we have grounded context and trusted AI systems? It’s gorgeous. Actually gorgeous in a way that makes me slightly breathless.

Think about it: when a caregiver asks “my husband is acting weird after his accident—is this normal?” and the system bridges from that vernacular desperation to clinical concepts like behavioral dysregulation, we’re not just answering a question. We’re capturing how humans actually try to understand medical information in extremis. That’s vocabulary mapping that doesn’t exist anywhere on the internet. That’s comprehension event data showing the temporal arc of recovery—what people ask in week three versus month six. That’s role-based framing revealing how patients, caregivers, and clinicians approach the same clinical reality from radically incommensurate perspectives.

This isn’t static content. This is the living record of meaning-making in motion. This is what happens when knowledge meets confusion and something new crystallizes in the encounter.

The system doesn’t hallucinate because it can only traverse paths that exist in the structure domain experts built. The reasoning is explainable because it’s grounded in relationships they defined. And as people interact with it, we learn what works—not from our assumptions, but from the patterns that emerge when real humans encounter real knowledge at moments of genuine need.

This is what it means to work with complexity instead of trying to automate it away. You create the conditions. You give the people closest to the knowledge the power to shape how it moves. You build infrastructure that preserves context, lineage, and meaning as things inevitably metamorphose. And then you pay attention—with genuine curiosity and methodological rigor—to what emerges.

Because they will change. That’s not a bug. That’s memory being alive.

---

The cost of pretending data is finished

When organizations treat data work as a project that can be “completed” and handed off, they don’t just incur technical debt. They incur epistemic debt.

Assumptions fossilize.
Context evaporates.
Models drift away from reality while dashboards continue to glow a soothing green.

The result is a new form of fragility: everything appears stable, right up until a critical decision depends on it. Then the failure is instantaneous and catastrophic.

This is why so many AI systems break precisely when they are most needed. They were optimized for output, not understanding. They were engineered for performance, not for the capacity to revise their priors in the face of contradictory evidence.

The systems that survive aren’t the ones with the most impressive demos. They’re the ones where someone can actually answer the question: “Why did it say that? And is that still true?”

Photo credit to my friend Phil Komarny - Crested Butte, CO

---

What becomes possible when we get this right

Here’s what I’m genuinely excited about, though: we’re not just building better guardrails against failure. We’re opening up entirely new possibilities for what healthcare knowledge can become.

When you have provenance and lineage built into the substrate, institutions can finally participate in the AI economy without relinquishing control of their intellectual property or exposing themselves to liability. They can transform content libraries from cost centers into revenue streams—not by selling access to static documents, but by licensing the interaction data that shows how humans actually comprehend complex information.

When you have ontology-grounded reasoning, you can surface insights that were always there but invisible: which patient populations struggle with which concepts, what metaphors land, how cultural context shapes information needs, where documentation lacunae create real-world confusion.

When you have domain experts controlling the knowledge architecture, the system can evolve with the evidence base instead of calcifying around the assumptions of whatever model was trained last year.

This is infrastructure for emergence. For letting new patterns surface. For honoring what we know while remaining permeable to what we’re still learning.

And here’s what really gets me: we’re doing this in a way that works against the homogenizing tendencies of large language models. LLMs are trained on frequency—they learn what’s common, what’s prevalent, what gets repeated ad nauseam across the internet. But the rarest vocabulary, the most precise terminology, the language that carries nuance and distinction? That gets washed out in the averaging. The technical term that appears ten times loses to the meme or trope that appears ten thousand times.

We’re building systems where domain experts can preserve the uncommon words. The technical terms that matter precisely because they’re not in common parlance. The distinctions that get lost when you optimize for the most probable next token. The specialized vocabulary that took decades to develop and carries conceptual freight no approximate synonym can match.

This is how you keep language alive. This is how you prevent the flattening of meaning into the most frequent approximation. This is how you resist the inexorable pull toward mediocrity that haunts every system optimized for the middle of the distribution.

When you let LLMs do your knowledge engineering, you get knowledge that sounds plausible but lacks the precision edges. When you let underpaid annotators click through taxonomies they don’t understand, you get classifications that follow the pattern without grasping the principle.

But when you give domain experts the infrastructure to encode what they actually know—with all its qualifications, exceptions, and contextual dependencies intact—you get knowledge systems that can actually be trusted. Not because they’re perfect, but because their limitations are visible and their reasoning is traceable.

---

A different standard of seriousness

If data is the product, governance is not overhead. Governance is the work.

Real governance means knowing where information comes from, how it changes, who it speaks for, and how its meaning shifts as humans and machines interact with it. It means designing systems that can be interrogated, revised, and corrected—without collapsing under the weight of their own history.

This kind of engineering runs slower than hype cycles and deeper than most incentive structures will tolerate. It is also the only kind of work that survives contact with reality.

Nothing is ever “done” when data is involved. That is not a failure of execution. That is the nature of building on living memory.

The sooner we stop pretending otherwise, the sooner we can build systems worthy of the trust we keep asking humans to place in them.

And maybe—just maybe—we can stop outsourcing the part that actually matters to whoever will do it cheapest or to whichever model gives the most plausible-sounding response, and start building infrastructure that respects the people who hold the knowledge we’re trying to scale.

That’s not idealism. That’s just engineering that understands its own substrate with appropriate humility and rigor.

And when you get the substrate right? When you build on living memory with proper lineage and provenance? The possibilities compound in ways we’re only beginning to apprehend.

I’m here for that future. It’s worth building with care, deliberation, and deep respect for the rarity of genuine expertise.