Publications

Event Driven Architecture (EDA)

Fig. 1: EDA visibly in action. Dashboard of smart energy meters.

The latest paradigm in ICT

Let's go back to how it all began, the silos or stovepipes. Because physical filing systems had their limitations, as soon as the available development tools permitted is, the first applications were created. In fact, they were nothing more than an electronic version of the existing filing systems: straightforward management of data elements that were then stored in a DBMS, which later evolved in an RDBMS with accompanying query language. When the back end was sorted, we started focusing on the front end; we separated the presentation layer and the business logic and began distributing them across our hardware as smartly as we could: they were tailormade solutions designed to support a specific business process. The data element 'Employee' could be stored and processed redundantly in the HRM application, the Facility Management application (the access pass), etc. Synchronising these applications posed a major challenge that cried out for integration.

After the stovepipes came the Service Oriented Architecture (SOA), which assumed a 'new center of the universe', namely the integration itself rather than the individual applications, which meant that the data element 'Employee' could be accessed by all the various applications. The term application itself is a flexible one in this regard, because what is an application if it completely consists of (or uses) the same services that are being used by other applications? Naturally, with this new paradigm came new challenges, like managing an exploding number of components, authorisation fpr all those components, encryption of relevant communication with outside elements like the Internet, etc. Oracle called the products that were needed to meet these challenges needed Fusion Middleware, and they were provided in the SOA Suite. The latest developments with regard to SOA were a sum of the standardization of the accessing of services (SOAP), the so-called WebServices, and the standardization of a process specification language in which these services could be accessed, the Business Process Execution Language (BPEL). From that moment on modelling tools were able, for instance, to provide graphic models of business process that could then be linked to the services from the application landscape.

After the professionalization of ICT within companies via the SOA paradigm, Event Driven Architecture (EDA) made its appearance. As is so often the vase, analists like Gartner paved the way. Around 2000 (you remember the Internet hype) Gartner introduced the term ‘Zero Latency Enterprise’, which he replaced a few years later with the modified concept of ‘Real Time Enterprise’¹. As the name suggests, this refers to the speed with which organizations are able to respond to stimuli and changing circumstances that are relevant to those companies. Many ICT professionals are familiar with that concept in an ICT evironment. You could draw a state diagram of a company and whatever can cause a status transition is an event. Thus, companies become more Event Driven. This means that EDA is not so much a replacement of SOA, but an extension. SOA says something about the inside of an application, namely its layers and the granularity of the services. The extent to which ppplications have been built in accordance with the principles of SOA are more suitable for implementing EDA concepts. Because nowadays, more and more context-related information is measured realtime and made digitally available, is is hardly surprising to know that there is a growing interest in EDA.

---

EDA

From a business point of view, EDA is perfect in situations in which a company is able to benefit commercially from responding smartly to changes in its environment and thus create a competitive edge from acting more quickly on the additional information. On the other hand, a company can limit disadvantages by acting quickly. These kinds of situations are above to be found in the area of risk management. For instance, it is very important for a financial institution to have the latest information regarding interest rates. In many cases, interest rates will affect many of its systems, not only the actual interest rates, but also their development patterns within a give time window. Sometimes, a certain interest rate in itself does not have to be cause for alarm, but it can be cause for concern if it is part of a given pattern. In short, EDA sooms in on the events and analyses them further.

---

Business Events

Generally speaking, there are two kinds of events that are relevent within EDA: the (simple) Business Event and the Complex Event. Business Events were known even before the arrival of EDA. The database trigger as a functionality was added to the database to support this concept. In a time when applications carried out straightforward maintenance on data elements, a database trigger was very useful to implement a business event. Orders were placed by telephone or order form (the traditional channels) and in both cases were typed in by an employee of the company receiving the order. Storing the data element ‘Order’ and its ‘Order rules’ was an accurate reflection of the business event ‘Receiving Order’, allowing the system to take the appropriate follow-up action. However, these traditional channels are now used less and less. Vanuit a Business to Business (B2B) point of view, the process of chain integration is well under way and even consumers (B2C) increasingly by products and services online. That is why the (simple) business event ca be found in the traditional paradigm (stovepipes), in SOA and als in EDA.

---

Complex Events

In EDA, the two kinds of events (business and complex) can be most easily related to the integration pattern that applies. Most relevant are the so-called Request-and-Reply and the Sense-and-Respond patterns. In the case of Request-and-Reply, there is a predefined question and answer, for example Order and Order confirmation. In the B2B chain integration, this is reflected in standards like EDI and (with the emergence of XML) in standards like ebXML. With the existing middleware, the electronic reception of such a message, for example ‘Order’, can be seen as the business event ‘Order ontvangen’. We no longer need to wait until the event is stored in the database before we can determine electronically that the event has occurrrd. Of course, receiving messages is not the only business event. After all, in the broadest possible sense, a business event is an occurring event that is important to the business.

The Sense-and-Respond pattern is of an entirely different order and in the past it only occurred in very specific applications. In the case of Sense-and-Respond, something is measured and in response associated action is taken. In a majority of the Sense-and-Respond patterns, however, there is complete chaos of occurring events. That is to say: often, there are many, sometimes even very many events (up to 1 million per second) that are entirely unpredictable in terms of when they occur and how the values that are measured will develop. The challenge is the filter out the essence from these raw events and to translate it into events that are relevant to the business. The raw events are called Complex Events, while the translated, relevant events, as we know by now, are the Business Events. To translate the raw events, a Complex Event Processor is used, as we shall see later. Of course, there are also simpler occurrences of the Sense-and-Respond pattern that involve events that can serve immediately as straightforward business events.

---

Examples of EDA applications

Most of the current EDA applications are pure examples of EDA. The reason for this is obvious: applying the EDA concept was the only way to solve the issues in question. However, this is in line with regular applications. As far as business analysts are concerned, the challenge is to make sure that they are able to think far enough outside of the box to see the possibilities of EDA as an extension to existing solutions. To get an idea with regard to the scope of EDA, we discuss a number of examples that have already been translated into an EDA application, or where such a translation would be possible.

---

Counting events: Events

The simplest application is that whereby the event does not actually provide any specific information, apart from the fact that it occurred. A good example is the application that involves counting something that occurs, for example the number of people who have entered a concert hall via the entrance gates. Even though the concert itself may be free, which means that there is no need to scan people as they enter, it may be necessary to count them anyway for safety purposes. The event is the fact that someone has entered, and whether that person is a man or woman, boy or girl, remains unknown. The entrance gates could be closed when a certain predefined number of people has entered the concert hall.

---

Counting events: Conversion factor Retail

In retail, and in particular department stores, the same thing takes place. The number of visitors inside the store is counted – the dry click you could hear in the past as you entered the store – to compare that number with the number of customers who purchase something (= number of receipts). This is called the conversion factor and it indicates to what extent visitors are persuaded to buy anything. The number of visitors is influenced by seasonal factors and marketing. Whether or not they buy anything is determined by the experience in the store (product range, ambiance, design, route) and whether or not other kinds of purchase-enhancing attributes are used, such as fragrances. To determine the effect of changes in these variables, the conversion factor and with it (this simple form of) EDA are indispensable.

---

Border control

A somewhat more complicated example that is a little closer to home is the thermostat. The traditional thermostat is made up of mechanical parts and shuts down at a predefined temperature. A continuous flow of events (the current temperature) is measured and when the predefined limit is reached, action is taken. This means that, in this case, it is not only the fact that an event occurred that is important, but also its actual value. In other words: the complex events ‘the temperature being measured at a given moment x’ is translated by the thermostat, based on a built-in algorithm, into the business event ‘temperature y has been reached’.

---

Tracking and tracing: RFID

In logistics, EDA concepts have been used for dozens of years. In the 1970s, a company called NEDAP provided black boxes that could be placed underneath a truck and that would identify itself in response to a given radio signal with a unique identification code. This meant that the reverse parking of a truck at the loading dock of a distribution center was enough to automatically get the right cargo from the storage facility. As such, this technology was the predecessor of RFID.



Fig. 2: A modern RFID tag in the form of adhesive foil.

In recent years, Radio Frequency IDentification (RFID) is applied more and more. The RFID tag is actually a miniature version of the black box provided by NEDAP. The tag has the format of a postage stamp (even as far as its thickness is concerned!). Figure 2 contains an image of such a tag. Most of the tag is made up of spools that together form the antenna, to which (on the left of the image) a chip is attached that receives the incoming signal, actually managing to obtain the energy it needs to function from the signal, and that responds by sending a unique identification code. In combination with the appropriate infrastructure of tag readers and event processing software like the Complex Event Processor in the Oracle EDA Suite, a tagged object can be followed throughout its entire mechanical or biological life. On the Internet, there is an informative PowerPoint presentation with the necessary explanation and background of RFID². Thanks to the miniaturization of the tags in combination with mass production, tags nowadays cost less than 10 eurocents. For a number of chains, for instance the American company Wall-Mart, this has been reason enough to provide all the products in the store with a unique tag, allowing it to track each product throughout the distribution chain, from manufacturer to retail store. In addition, with regard to perishable products, environmental conditions can be stored onto the chip. Also, the tag is an invaluable took when it comes to theft prevention.

---

Tracking and tracing: Barcode

Another example, one which affects those of us who enjoy international travel, is the fully automated handling of luggage at airports. The Dutch company Vanderlande in Veghel is one of the main turnkey provides of these kinds of systems. Once you have seen their logo, you see it all around the world. During the luggage handling process, the barcode is scanned at various locations in the chain. The complex events in this case are the points at which unique codes are being scanned by certain scanners. By applying certain algorithms, patterns can be recognized and translated into business events. In the case of luggage, for example, once a piece of luggage has been scanned at one location, it has to be at the next location within 1 minute, or else it has been misplaced and an employee needs to be informed that he or she needs to go out and investigate.

Fig. 3: Inspired by current events. (Source: Nu.nl).

---

Tracking and tracing: Maria van de Hoeven's smart meters

When writing this article – April 7, 2009 – I read a press release (see Figure 5) announcing that Maria van der Hoeven’s (the Dutch Transport Minister) smart meters had failed to make it. Inspired by current events, I was unable to resist the temptation to see if I could translate this application of EDA into a working demo and assess all the aspects of the EDA Suite for myself. If it were up to Maria van der Hoeven, all Dutch households would have to install so-called smart meters to measure the amount of electricity they used. Although this may be a commendable initiative from an environmental point of view, there are definite privacy concerns as well. The idea was that the information on the meters could be read out at a distance and that, based on the information, individual households would receive information on how to save electricity. In addition, the information could be related to measurements at the major electricity distribution nodes or even the power plants themselves. A lot of money is made in the energy market by predicting energy demand in order to buy the right amount of energy. Energy trading companies have to pay massive fines when they use up considerable more energy than they have purchased from the energy producers. Needless to say, they are very interested in anything that helps them predict energy usage more accurately. Figure 4 contains the image of a BAM dashboard visualizing the usage that has been measured at smart meters at substations and households. Behind the scenes, a wealth of Oracle software is applied. The main components are the Oracle Complex Event Processor in combination with Oracle BPM and, of course, Oracle BAM. Due to a lack of smart meters, we have had to settle for a simulator, a Java program reading a data file and sending the information to the Event Processor at the preset speed. In addition, Figure 4 contains the information that is measure on the analog clocks in greater detail, making it possible to browse the values.



Fig. 4: Detail in BAM Dashboard of registered energy values.

---

…and much more

The examples discussed above are presented in ascending order of complexity. These days, the EDA has reached such a level of maturity that more and more industrial sectors use the most complex applications of EDA even in regular business processes. That is to say: the version where we try to correlate the occurring events and translate them into ‘information’ on the basis of recognizable patterns. Examples of areas where this is done are risk management, fraud detection, intrusion detection, compliance, etc.

---

The EDA Suite

After this list of examples of EDA concepts, it is time to look at the software that is needed to create these applications. To this end, Oracle has built the EDA Suite, which contains four products, three of which we already know from other applications:

· Business Activity Monitoring (BAM),
· Oracle Service Bus (OSB)
· Business Rules (BR)
· Complex Event Processor (CEP)

Our three old friends, BAM, OSB and BR have already been discussed extensively in Optimize year 9, number 4 (September 2006), when I evaluated the SOA Suite. Of course, since that time, there have been modifications to these products. BAM has been given a much prettier lay-out, which is essential for a product that depends on its ‘looks’. Conceptually speaking, OSB does the same things that the ESB used to do, but everybody knows that Oracle no longer sees its ESB in strategic terms (although it continues to provide full support). The bus that has been strategically positioned since the takeover of BEA is the BEA AquaLogic Service bus, which has been renamed Oracle Service Bus. In the last three years, BR has not changed in essence, although I expect that, with the takeover of Haley³ at the end of October 2008, this will happen in the near future. All you need to do is Google ‘business rules haley’ to see how enthusiastic the major players and analysts are about Haley’s Business Rules Platform. Haley Business Rules Engine is based on an improved version of the so-called ‘Rete Algorithm’, with which the most advanced business logic can be recorded and executed (I will come back to this in future issues of Optimize). However, the core product when it comes to EDA is the Complex Event Processor, about which there is so much to tell that it warrants a separate article, which I will write for the next issue of Optimize. As a sneak preview, Figures 5 and 6 contains screen shots of the development tool and management monitor of the Complex Event Processor.



Fig. 5: Development for Complex Event Processing in Eclipse.



Fig. 6: Management for Complex Event Processing in the Visualizer.

---

Future

Using the examples discussed above, I have tried to provide a sense of the range of possible applications of EDA. However, they are all examples of applications that we already see everywhere around us. As far as this is concerned, we can expect a major shift in the near future. The terms that is often used by futurists in this context is Ubiquitous Computing, or omnipresent computer power connected to everything and embedded in everything. On YouTube, there is a small clip (4 min. 17 sec.)⁴ of a great documentary that was made a few years ago in which this phenomenon is discussed. When we consider the speed with which technology is progressing in our daily lives, I have no trouble believing that the predictions made in the documentary will come true. And even if things develop at a slower pace, the fact remains that more and more things are being measured and made available digitally. Even today, athletes using an invention by a Dutch company called ‘The Champion Chip’ can let the folks back home know via the Internet where they are at any given moment⁵. Each contestant wears a sports-GPS and pushes events about their current position.

Or, something that is relevant to every Dutchman: more or less the entire Dutch highway network is under camera supervision, which means that, based on license plate recognition, it is possible to determine where each on of us is driving at any given moment. And if it is not on the basis of this technology, the fact that we are connected to a mobile phone network will do the job instead.

I am not entirely happy with some of these developments, but fortunately there is legislation in place to protect our privacy. My point may be clear, though. EDA is not a niche anomaly somewhere in the murky regions of ICT. It is the architecture that provides the basis for a new form of automation. And the way we will first notice it is when more and more companies find possibilities to apply EDA in their business processes, which means that Gartner was right when he predicted the Real Time Experience back in 2002.

---

References

1) Real Time Enterprise, Gartners introduction of this topic mid-2002:
http://www.gartner.com/pages/story.php.id.2632.s.8.jsp

2) Informative Powerpoint presentation containing an explanation and background of RFID:
http://www.cs.unc.edu/~pozefsky/COMP380_S08/RFID_Presentation%5B1%5D.ppt

3) Takeover by Oracle of Haley, end of October, 2008:
http://www.oracle.com/haley/index.html

4) Ubiquitous Computing, snippet on YouTube from the BBC documentary ‘New Visions of the Future’ with Dr. Michio Kaku:
http://www.youtube.com/watch?v=3TY4z7JylZA

5) Real Time Tracking of athletes with demo at Tour de Texel:
http://www.championchip.nl/ccnederland/
http://www.sport-track.com/
http://spectator.sport-track.com/7961

---

For all publications, check http://www.anewlink.nl/ict/en/publications/

---

(c) June 2009,  A New Link bv,  www.anewlink.nl.