Your enterprise files are full of business events. You just need to see them

What if your enterprise files are telling you far more than you think?

Enterprise files are so familiar that we rarely stop to question what they actually represent. CSV, XML, EDI and countless other formats have become part of the everyday machinery of enterprise integration, moving information reliably between applications, organisations and business processes. We know how to generate them, transfer them, validate them, transform them and monitor their arrival, and after decades of doing so, it is natural that architecture has come to see the file itself as the unit of work.

But that familiar view can hide something far more interesting.

Every file exists because something happened somewhere in the business. Information changed, a process moved forward or a new state was reached and, although those changes may have occurred independently and at completely different moments, by the time they reach another system the architecture may see only a single file waiting to be processed.

What appears to be one technical artefact can therefore represent hundreds or thousands of individual moments in the life of a business, each carrying meaning that extends beyond the file in which it happens to travel.

That raises a different question. Instead of looking at a file and asking only Where does it need to go?, How should we process it? or When should the next batch run?, what happens if we ask:

That question opens up a very different way of looking at enterprise files. Rather than seeing them simply as containers of data, we can begin to recognise the business events already travelling inside them and consider what becomes possible when those events are understood independently from the way the information is packaged and transported.

From enterprise files to business events

To understand what changes when we look beyond the file, we first need to separate two things that enterprise integration has traditionally treated together: the way information is transported and the business activity that information represents.

Consider a customer export containing one thousand records. From an integration perspective, it is a single technical artefact with a defined structure, source and destination. It may be generated according to a schedule, transferred through an agreed mechanism and processed by a receiving application, with monitoring focused on whether it arrived, whether validation succeeded and whether every record was processed correctly.

Those are necessary engineering concerns, but they tell us very little about why those one thousand records exist.

Look at the same export from the perspective of the business and the file begins to fragment into individual changes. Some customers may have been created, others may have updated their contact details, several may have changed address and others may have become inactive. The architecture processes one file, while the business has experienced a series of independent changes.

This is where the concept of a business event becomes useful.

What is a business event?

A business event describes a meaningful change that has already occurred within an organisation. A customer account being created, a payment being settled, a shipment being dispatched or a laboratory result being authorised are all examples because each describes a change in operational reality that could matter to another person, process or system.

Compare, for example, payments_240824.xml with PaymentSettled. The first tells us something about how information has been packaged, whereas the second tells us what happened.

This distinction becomes increasingly valuable as information moves beyond the application that originally produced it. A payment being settled remains the same business occurrence whether another system learns about it through an API, a message, an event stream or a row inside a CSV file.

In other words, business meaning survives changes in transport.

Enterprise architecture has historically invested considerable effort in deciding how information should travel. Should systems communicate through FTP or SFTP? Should an interface use REST or SOAP? Should information be represented as CSV, XML, JSON or Avro? These are important engineering decisions because they determine how reliably, securely and efficiently information moves between systems, but they do not determine the business meaning of the information being moved.

An invoice remains an invoice whether it arrives as XML, JSON or a fixed-width file, just as a pathology result remains a pathology result whether it is transmitted through HL7, FHIR, PDF or CSV.

This leads to an important distinction:

When architecture focuses primarily on transport, integration is naturally concerned with moving information successfully between systems. Once we also consider the business events represented by that information, however, a broader question becomes possible: what meaningful change occurred within the organisation and who could benefit from knowing about it?

For organisations reviewing how information currently moves across applications and platforms, this also changes how we think about system integration. Connectivity remains essential, but the value of integration extends beyond moving information when we can also understand the business activity that information represents.

A file is a compressed stream of business events

The distinction becomes even clearer when we introduce time into the picture.

Imagine an order processing platform operating throughout a normal business day. At 10:17, a customer places an order. A few minutes later, another customer amends theirs. At 11:42, a payment is authorised; shortly afterwards, inventory changes and an item is dispatched. Throughout the rest of the day, thousands of similar activities take place independently across the organisation.

Now imagine that the platform exports those changes at midnight.

From the perspective of the receiving architecture, the significant moment may appear to be midnight because that is when the file arrives and processing begins. From the perspective of the business, however, the meaningful moments occurred throughout the preceding day.

The business did not wait until midnight for those things to happen. The architecture waited until midnight to package information about them together.

This is what we mean by a compressed event stream. The file brings together business activity that occurred at different moments and presents it to the receiving architecture as a single unit of work. What has been preserved is the information, but the timing and sequence of the underlying activity are no longer immediately visible.

Business activity happens continuously, even when information about that activity is accumulated and transported as a batch

The important distinction is that the batch is not necessarily the natural shape of the business activity, it is often simply the shape chosen by the architecture to transport information about that activity.

That does not make the file wrong, nor does it mean that every file-based process needs to disappear. It simply means that the file and the business events represented within it are two different things and separating those concepts allows us to think differently about where and when that information could create value.

The business event exists before the file arrives

This has an important implication for enterprise modernisation. In many enterprises, the underlying business activity already exists and is already being captured by operational systems.

A customer has changed their details before the customer export is generated, a payment has settled before it appears in a reconciliation file and a shipment has left the warehouse before the logistics extract reaches another platform. The same is true of a laboratory result: its authorisation is a meaningful business occurrence regardless of when information about it is subsequently packaged and transferred.

The event already exists. What may not exist is an architectural mechanism for making knowledge of that event independently available.

This changes the starting question from How do we replace this file? to something more useful:

The difference is significant because modernisation no longer has to begin with replacing the source system or eliminating the file. It can begin by understanding what existing systems are already telling us.

Introducing Business Event Extraction

Recognising that enterprise files contain business events raises an important practical question: how do we make those events visible without first replacing the systems and file-based processes that already exist?

The answer requires more than simply parsing a file or converting one format into another. A traditional integration process may understand that a particular field contains a date, status or identifier, but extracting a business event requires understanding what the information means in the context of the business and what change has actually occurred.

We call this Business Event Extraction.

Business Event Extraction is the architectural process of discovering and interpreting meaningful business events from existing enterprise artefacts, allowing the business activity represented within them to become independently available beyond the file or process in which it originally travelled.

Consider a customer export containing a change from CUST_STATUS = 1 to CUST_STATUS = 4. Technically, an integration platform can identify that a field has changed. Business Event Extraction goes further by interpreting what that change represents: for example, that a CustomerAccountSuspended event has occurred.

That distinction matters because extracting data and understanding business activity are not the same thing. One gives another system access to information; the other creates a reusable representation of something meaningful that has happened within the organisation.

Once identified, those business events can potentially be validated, enriched and governed before being made available to other authorised consumers. Crucially, this does not require the original file-based integration to disappear. The existing operational process can continue doing exactly what it was designed to do, while the business activity represented within it becomes available for new use cases.

This is what makes Business Event Extraction particularly relevant to enterprise modernisation. Rather than beginning with the assumption that legacy files and batch processes must be replaced, organisations can begin by asking what valuable business activity is already captured inside them and where making that activity independently visible could create value.

The Business Event Extraction lifecycle: transforming existing enterprise artefacts into governed, reusable business events while preserving context, lineage and business meaning.

The lifecycle illustrates how information can progress from the detection and interpretation of existing operational artefacts through validation, enrichment, modelling and governance before meaningful business events are made available for wider consumption. The important point is not that every organisation must implement an identical seven-stage process, but that moving from files to reusable business events requires both technical processing and an understanding of business meaning.

How business events create value beyond the original integration

Traditional file integrations are usually designed around a known relationship between a producer and a consumer. A finance application exports information because another finance system requires it, a warehouse platform creates an inventory extract for a reporting application or a clinical system generates a report because an archive needs to ingest it. The integration is consequently optimised around that particular journey.

The business activity represented inside the information, however, may have relevance far beyond its original destination.

Consider a payment settlement. The information may originate within a finance process, but knowing that the payment has settled could also matter to customer service, order fulfilment, fraud monitoring, analytics, notifications or automation. Likewise, an inventory change originating within warehouse operations may influence e-commerce availability, procurement planning, logistics and customer communications.

The information may have been packaged for one consumer, but its business significance is not necessarily limited to that consumer. The architecture simply happened to be designed around one consumer first.

One file, many potential consumers

A pathology report may historically be treated as a single document that needs to be archived or indexed. Viewed through the business events it represents, however, that same report can reflect multiple meaningful changes, from a specimen being received and analysis beginning through to a result being calculated and authorised, a critical observation being recorded or a clinician notification becoming necessary.

A single operational artefact can represent multiple business events, each potentially valuable to different parts of the organisation.

The file is therefore not just a document to be moved, it can contain business events with relevance beyond its original destination.

A laboratory may need the operational status, a clinician may need the result and another system may need to act on a critical finding. The original integration, however, may have been designed simply to move the report from one system to another.

Does every file need to become real-time?

If enterprise files contain business events, an obvious question follows: should every record inside every CSV, XML document or EDI exchange therefore become a real-time event?

No, and that distinction is fundamental to the Beyond Batch approach.

Beyond Batch is not an argument for real-time everything. Real-time does not necessarily mean instantaneous. It means making information available within the timeframe in which it can still influence a business decision or action. Turning every enterprise process into event streaming would simply replace one architectural assumption with another, regardless of whether doing so creates meaningful business value.

Some information genuinely belongs in batch processes. A nightly regulatory submission may remain entirely appropriate as a file because nobody gains meaningful value from receiving each record individually. A monthly financial extract may have no reason to become real-time, while a trading partner may continue exchanging EDI documents because the existing process is reliable, understood and aligned with the business requirement.

The more useful question is therefore not whether every record can become an event, but whether the business change represented by that record has value outside the batch in which it currently travels.

If knowing about a payment failure immediately could prevent a customer issue, timing may matter. If an inventory movement could prevent another system accepting an order it cannot fulfil, making that information available sooner may create tangible value. If nothing meaningful changes by waiting until tomorrow, however, the existing batch may remain exactly where it belongs.

The objective is not real-time for the sake of real-time. It is making business information available at the point where its timing creates value.

Where timing does create that value, real-time data streaming provides a different model in which information can move continuously rather than waiting for scheduled processing.

The decision is therefore not simply technical. It depends on whether knowing sooner changes what the business can do.

From file-based integration to business event discovery

Once enterprise files are viewed as potential sources of business events, the purpose of integration begins to broaden.

Historically, integration platforms have focused primarily on connecting systems. Files are moved, messages are translated and interfaces are maintained. Those capabilities remain essential, but we identify a larger opportunity: discovering meaningful business activity that was previously visible only as part of an operational artefact.

The architectural question therefore begins to shift from:

towards:

This transition matters because it moves enterprise integration beyond connectivity alone and towards understanding the operational knowledge already flowing through the organisation.

Business events need business meaning

Recognising that a record represents an event is only the beginning.

Enterprise systems naturally describe information according to their own internal models. A source application might represent a customer status as CUST_STATUS = 4, which may be perfectly meaningful within that application but communicates very little to another consumer without additional context. CustomerAccountSuspended, by contrast, describes a business occurrence that can be understood beyond the system that originally produced it.

As information becomes reusable across more applications, analytics, automation and AI, that shared understanding becomes increasingly important. Consumers should not need detailed knowledge of every source application's internal implementation simply to understand what happened.

This eventually introduces questions around canonical modelling, governance and semantic consistency, which become important later in the architecture. For now, the principle is simpler: extracting information is not enough; organisations also need to understand the business meaning it represents.

Modernisation does not have to start with replacement

This perspective also changes how we look at legacy integration estates.

Hundreds of scheduled CSV exports, XML documents, EDI exchanges, spreadsheets and nightly processing jobs can easily appear to represent a modernisation backlog: a collection of technologies that must eventually disappear before the organisation can become more responsive.

But the same landscape can be viewed differently. It is also a map of operational activity already moving through the enterprise.

The modernisation opportunity therefore does not necessarily begin by replacing those processes. It can begin by identifying which meaningful business changes already exist within them, where those changes have value beyond their original destination and whether making that knowledge available differently could create a measurable business outcome.

This creates a much more pragmatic route towards modernisation because organisations can build upon operational systems that already work rather than assuming value can only emerge once those systems have been replaced.

The modernisation opportunity may already be moving through your integration estate every day.

What business events are hidden inside your enterprise files?

For decades, enterprise integration has quite reasonably focused on questions about format, transport, schedules and destinations. What format is the file? Where should it be delivered? When should it be processed? Which application needs to consume it?

Those questions remain essential, but they describe the mechanics of integration rather than the meaning of the information being integrated.

The more important question is what becomes possible once that business activity is visible.

The file itself may remain, the systems producing it may remain and the original integration may continue operating exactly as it does today. What changes is our understanding of the information inside it.

The next challenge is making that business activity available to the systems and people that can act on it.

Could your enterprise data be working harder?

Your organisation already generates enormous amounts of valuable business information. The opportunity is understanding where that information could become more timely, connected and actionable across your enterprise.

At Claria, we help organisations rethink how data moves and creates value across their technology landscape, from integration and real-time data to automation and AI.

Ready to explore what your existing technology and data could do next? Get in touch now.

Jamie Carter

Share