Terry Ragon, Chief Executive Officer
InterSystems encourages the adoption of a flexible, practical approach to application development, rather than strict adherence to one of the prevalent development theories. This paper offers advice based upon our experience. However needs, attitudes, and styles vary; we recommend that each programmer choose the development approach that works best for them. Caché supports a wide range of development methodologies, not just those recommended here.
Use Objects for New Applications
We strongly recommend the adoption of objects for new database applications for the following reasons:
- Objects support a richer data structure that more naturally describes real world data.
- Programming is simpler; it is easier for developers to keep track of what they are doing and what they are manipulating.
- Customized versions of classes can easily replace standard ones, making it easier to customize an application and support it through new releases. This feature is particularly important for VARs who must support a variety of customized versions.
- The black box approach of encapsulation means programmers can improve the internal workings of objects without affecting the rest of the application.
- Objects provide a simple way to connect different technologies and different applications.
- Object technology is a natural match with Java, allowing easier web development, and with GUI based user interfaces.
- Many new tools assume object technology.
Objects provide a good insulation between the user interface and the rest of the application. Thus when it becomes necessary to adopt a new user interface technology (perhaps Web or some currently unforeseen future technology) developers can reuse most of their code.
Objects are most useful for database objects and significant temporary (“transient”) objects. Often for small temporary storage it is easier to simply use one or a few variables or arrays; in practice use what seems most practical.
Although most data storage will usually be through objects, there are times when it makes more sense to directly use “global references” to set and retrieve data from the Caché multidimensional data server. This would be true where there are unusual or very specialized structure requirements and no need to provide object or SQL access to them, or where the highest possible performance is required.
Overall Development Process – An Iterative Approach
Development usually occurs as an iterative process involving:
- Design of database classes, including the specification of properties and methods
- Writing code
- Creating the user interface
Although in theory it would be nice to completely design all aspects of an application before any coding occurs, in practice that can result in design paralysis, in an application not meeting its users’ true needs, and in an application never being finished. InterSystems recommends a more iterative process.
While we are strong believers in the importance of a thorough design, it is our belief that for complex applications the actual design becomes much clearer as the application evolves, and that a useful application requires changes based on feedback from users. One of the strengths of Caché is that programmers can develop code rapidly and easily modify it, making it feasible to implement design changes and rewrite portions of code as experience is gained. Development proceeds faster with fewer programmers (due to the productivity of Caché development) and results in an application that is more responsive to users needs.
In fact, the ability to rapidly develop with Caché means not only can projects be completed quicker with fewer resources, it can also facilitate developing applications that might not otherwise be completed. It is well known that adding additional programmers to a software project dramatically increases complexity – more people need to communicate and agree upon more issues and each understands less of the whole. Thus despite the utilization of more resources, the result is often a longer time to market and less reliability than if fewer people could have developed the software. Sometimes, the result is that the project can’t even be completed.
InterSystems recommends starting with the basic database design. Once that is completed, the business and database logic can usually be written at the same time as the user interface code. In some cases it will be useful to write key business logic code before the related user interface code to help determine what information needs to be solicited. Some people feel more comfortable starting with user interface code to help determine the nature and look and feel of the application. That alternate approach works particularly well when there is a relatively small number of database properties and a simple application, but for more complex applications we recommend starting with the database definition. In reality, it should all be an iterative process in which classes and code are continually refined. The important thing is to get started.
Informal diagrams outlining the logic flow of algorithms and other complex areas of code are highly useful. They can strengthen the reliability and predictability of the application, simplify coding, and facilitate the design specification of private properties and methods signatures. It is often useful to perform the coding of these algorithms immediately after they are diagramed, while they are still understood by the developer. This is another reason for our recommending an iterative approach to development.
Should You Use a Modeling Tool?
Caché supports various modeling tools, such as Rational Software’s Rose, which are favored by many organizations that prefer an extensive design phase. These tools are good at focusing attention on the static design of structures, such as database properties, and are often fun to use.
However, modeling tools are more useful when using a relational database due to the complexities of decomposing the natural rich structure of real-world data into flat relational tables with the resulting plethora of relationships linking them. This decomposition is not needed in Caché.
InterSystems does not usually recommend the use of modeling tools with Caché. Instead we recommend:
- Work directly with the Caché Object Architect to first define the data structures (and perhaps the signatures of some of the most important methods) of some of the main classes, particularly the main database classes.
- Then start writing code.
- Perform additional design as part of an iterative process.
The reasons for this approach are:
- It avoids extensive theoretical debates about the nature of objects and how they should be structured. Some projects fail because developers spend too much time in abstract modeling.
- Usually the nature and functionality of method code, as well as what methods and private properties are needed, becomes much clearer as code is being written. The programmer has something to demonstrate much earlier in the development cycle.
- With this approach, a single tool is used for both design and writing code. Developers are relieved of the overhead of constantly coordinating development work with a separate design tool as the design inevitably evolves.
For people new to Caché, we especially recommend gaining some experience with the product by writing some of the code (methods) as soon as possible.
While this is not normal advice, we believe it results in developing a superior application quicker. This iterative process allows one to constantly refine the design rather than being locked into a predetermined structure that becomes outdated and a liability as development proceeds. It also avoids the paralysis and over-analysis that can occur in the absence of pragmatic experience.
Modeling and Data Design in Caché
When performing object design, it is easy to become embroiled in abstract discussions of the nature of an object, and hence of what methods it should have. In addition to being time consuming, the resulting set of methods may often have little relevance to the practical needs of the application. InterSystems believes the need for and nature of most methods becomes much clearer during the coding process.
Clearly, however, some basic design should occur before any coding occurs. We recommend using the Caché Object Architect to specify:
- What are the main database classes? (e.g., Invoices, Suppliers, Patients)
- What are their public properties? (i.e., properties that can be seen by other objects, as opposed to private properties that are hidden from other objects and can only be accessed by methods within the same class)
- What references and relationships exist between them? (e.g., a Patient may have a PrimaryPhysician property which is a reference to a Doctor, and an Invoice has a one-to-many relationship with a set of LineItem objects)
At the same time that a property is being specified, it is usually useful to specify the characteristics of that property (e.g., Is a value for that property required? Must it be a unique value? Should it be indexed?). This is also a good time to write code for calculated properties (e.g., the Age property’s value is calculated based on the DateOfBirth property).
What about specifying the names and signatures of methods during the initial design? (The signature includes the type of the return value, if any, and the number and type of input parameters.) Developers may wish to define the signatures of obvious and important methods during the initial design, but most methods should probably be defined as their need is determined during coding. Also, it is common for the signature to change as the method’s code is being written. Often it is easiest to code a method at the same time as its signature is being defined.
Three Types of 3-Tier
When people talk about 3-tier architecture, there’s often confusion because they usually mean one of three very different things:
- 3-Tier Hardware Deployment – separate computers for database server, application server(s), and user interface computers.
- 3-Tier System Software Architecture – database server, application server, and user interface software layers, but they may not be on separate machines
- 3-Tier Application Software Development – database logic, business logic, and user interface logic, with strict separation of code into these three layers. Again, they may or may not be on the same computer.
Caché has a 3-tier system software architecture, and supports a wide range of hardware deployment options.
There is no need for the Caché developer to be cognizant of the deployment architecture; to move from everything being on a single computer to a two- or three-tier or peer-to-peer hardware architecture does not even involve recompiling or re-linking software modules.
3-Tier Application Software Development
Conceptually, most database applications include code for three different types of functions:
- user interface (UI) logic – This is code for input/output operations, such as client/server windows code, browser code, etc. This code specifies how information is solicited and displayed, what business logic to call in response to various I/O events, and how to respond visually.
- business logic – The business logic is usually the majority of the application code. It includes algorithmic code that implements “business rules and processes”, which is essentially what the application actually does. (An example of a business rule is: All invoices over $100,000 require the approval of two managers. An example of a business process is: To pay an invoice match the invoice with an authorized purchase order, verify that the goods and the necessary management approvals have been received, and schedule the printing of a check.)
- database logic – storing and retrieving data from disk, performing queries of the database, maintaining referential database integrity, etc. For example, filing an invoice in a relational database may involve updating numerous records (it is much simpler with object access) and various statistics.
In recent years, there has been a highly promoted philosophy of developing database applications with strict adherence to these three tiers. In that model, there are strict rules regarding which of the three layers can call code in which other layers, and it is common to wind up with a set of classes in one layer whose names, but not whose functionality, mirrors those in another layer. For example, there may be a business logic class Invoice_bl with a method called Save that simply calls a method called Save in a database logic class Invoice_db to do the actual filing.
The essence of the arguments in favor of 3-tier development is that the division between tiers makes it possible to have larger teams of programmers and to achieve database independence more easily.
Is 3-Tier Application Software Development Right for You?
Caché can support 3-Tier Application Development. However, we do not recommend it. Instead, we recommend:
- Integrated business logic and database logic layer
- Strictly separate the UI layer from the business/database layer
- Keep the UI layer as thin as possible, putting as much of the code as possible in the business/database logic layer
Here is the rationale for our recommendation.
At the heart of arguments in favor of 3-tier application development is the assumption that there is a significant amount of code in the database logic tier requiring database programming expertise. That is not the case with Caché, which automates most database functionality by providing “smart database objects” that automatically handle the filing and retrieval of information and a “query wizard” to construct most database queries embedded in programs. Also, the multidimensional data model results in not having to decompose a rich object into lots of relational tables, so Caché code is much simpler.
Thus in Caché there is relatively little database logic that must be written by the programmer, eliminating the need for a separate group of database specialist programmers. It also allows database objects to more accurately reflect the powerful object model; instead of there being two invoice classes (one for business logic one and one for database logic) there is only one which encapsulates all of the invoice logic. This is a much simpler model that results in more rapid programming, less coding, and easier modifications later.
However a strict separation of the user interface code, with as little of the code as possible in the user interface layer, is recommended because:
- It is easier to program additional user interfaces later.
- It provides a good separation of programming responsibilities between visual design specialists and core application programmers.
- Classes at the UI layer don’t correlate closely with business logic classes.
Another reason for having a thin UI layer is that the installation and later replacement of user interface code creates greater system management burdens than changes to the business logic, which are managed through the Caché application server.
Database Independence with Caché
Database independence is promoted within Caché through the use of Caché Storage Classes. When defining a database class, a programmer specifies which storage class should be used. This storage class specifies either direct filing into the Caché multidimensional database or into a relational database. To switch the application to use a relational database, programmers simply designate a different storage class to use (which can be accomplished as simply as redefining a single logical name for the entire application), and recompile the application.
Writing the Business and Database Logic
Caché Object Classes can be built, edited and compiled using the Caché Object Architect.
Caché has “smart database objects”, so developers do not have to write methods to load, save, or delete the objects. Caché will automatically generate those methods when the class is compiled. (In certain cases, developers may want to write custom database methods, and they may do so.)
Caché ObjectScript is an excellent language for coding the business and database logic for a database application, and InterSystems strongly recommends its use. It provides far more rapid application development than with alternate languages, and its close integration with the Caché multidimensional data server results in extremely high performance for most database applications. Its close integration with the Caché application server also makes system management, configuration changes, and the introduction of code changes into running systems much simpler.
However, it is not the only option. The Caché Object Servers can render Caché Objects as ActiveX, Java, C++, or CORBA objects for the convenience of developers who prefer to program in Visual Basic, Java, or C++.
Caché also exposes the database through SQL using ODBC and JDBC, enabling the use of programming tools based upon SQL.
Creating the User Interface
The User Interface is most commonly coded in Visual Basic, Delphi, C++, Java, or HTML. Many programmers have reported that of these choices web-based HTML user interfaces can be developed the fastest, and that C++ based user interfaces often take significantly longer to develop than the others.
Through its Object Servers, Caché is compatible with many development tools, and HTML script can be directly embedded in Caché ObjectScript.
Of special interest to those creating Web interfaces are Caché Server Pages, which allows Caché scripts to be used with off-the-shelf Web page creation tools.
For developers, the key in deciding which technology to use for the user interface is to select technology with which they are already familiar or can easily learn, and to keep the technology simple. It is easy to over complicate the technology choices in the user interface with several layers of complex technology.
Web or Client/Server Windows?
One of the most difficult choices for many is selecting whether to code the user interface for the Web or for client/server windows.
The experience of our customers indicates that if you have an existing terminal-based system which can already run on Caché, by far the fastest and simplest way to add a modern user interface is to modify it to support the Web with Caché Server Pages and embedded HTML. This often results in a retention of significant amounts of the existing code. In some respects, this is like replacing a dumb terminal with a very smart and visually sophisticated terminal. This approach generally results in a procedurally oriented application that is browser-based. However, it is often not as visually sophisticated and does not offer as much user control as a fully event driven client/server windows based application.
In writing a completely new application, the key is to maintain a strict separation between the UI layer and the business/database layer and to keep the UI layer as thin as possible. It is quite likely that in many cases supporting both client/server windows and web will ultimately be required. In general, a Web-based UI layer will often be faster to develop and have the advantage of being browser-based, and hence more widely accessible. It also is consistent with a broad market theme emphasizing the web, but client/server windows can be visually more sophisticated.
Deploying and Maintaining an Application
Hardware Platforms and Operating Systems
Caché applications typically run unchanged on a wide variety of hardware platforms and operating systems, including Windows 95/98 and NT, all major UNIX platforms, OpenVMS, and Linux.
A Caché application may be deployed in a one-tier, two-tier, three-tier, or peer-to-peer hardware configuration without reprogramming or recompiling. In very large systems there may be multiple Caché application servers as well as multiple Caché multidimensional data servers.
Usually a client connects to only one Caché application server, and it is that application server’s responsibility to obtain the data from the correct data server.
In general there is far less overhead when an application server accesses data that is on the same computer as the application server, so unless there is a real need for separate computers for the data server and application server, InterSystems recommends combining them on a single computer.
Installing an Application and Introducing Changes on a Running System
Once an application is written it must be installed and later changed. With thousands of PCs and multiple servers, this can be a real problem with most technologies.
Caché simplifies installation of application software. Caché ObjectScript code only needs to be installed on a single data server. Other computers get copies of it using DCP and cache it.
To introduce a change in a Caché ObjectScript routine into a running system, simply load the changed source code routine onto the single data server where the code resides. The source code will be automatically compiled and the applications servers will be notified they need to reload the revised routine. Of course, such changes should be introduced with caution; a process that made a call from the old version to another routine will get an error upon attempting to return to the changed routine.