Chapter 2: Caché SQL and Objects Enhancements

Table of Contents | Chapter 1 | Chapter 3 | Chapter 4
The enhancements to Caché SQL and Caché Objects are listed in the following sections. Through its ODBC compliance, Caché SQL supports third-party tools for queries, forms, report-writing, and other popular functionality.

Caché 3.2 optionally includes Version F.17 of the relational environment. F.17 is a continuation of the F relational environment which does not provide any of the Object-SQL interactions. For example, F.17 does not store Caché objects. To install F.17, you must perform a Caché for Windows custom installation or answer Yes to the appropriate prompt in the Caché for UNIX and Caché for OpenVMS versions.

New Caché SQL Documentation

The documentation installed with the Caché for Windows version includes a new Caché SQL Reference, which lists the SQL commands supported by Caché. Improvements to this document will continue after Caché 3.2 is released, and newer versions will be posted on the InterSystems Web site.

Dynamic Caché Class Reference

In Caché 3.1, the Caché Class Reference consisted of static HTML pages that were generated when a given class was compiled. The pages were stored in the CacheSys\Docs\classref folder.

For Caché 3.2, the pages are completely dynamic. There are no static HTML files; all Web pages are generated on the fly from information contained within the Caché Class Dictionary. The pages are actually generated via the %Documatic class, which means that Caché must be running for these pages to be available.

Note: If you previously had shortcuts or bookmarks to the static HTML pages, they will not work for 3.2.
Two other new features of the Caché Class Reference are:

When documenting your own classes, you can use these new keywords:

<CLASS>classname</CLASS> formats and generates a hyperlink to the class.

<PROPERTY>property</PROPERTY> generates a link to the property (in this class).

<METHOD>method</METHOD> generates a link to the method (in this class).

<EXAMPLE>lines of code</EXAMPLE> formats lines of code as an example.

Improved VB Forms Wizard

A completely rewritten version of the Forms Wizard is available. This version uses the Caché Object Factory, rather than the CacheLink control. It also includes much more functionality and produces a more attractive form.

Increase in Number of Open Objects

The maximum number of open objects has been increased to 4096 in Caché 3.2. The previous limit was 1024.

Larger Routines and Methods

The limit of 32KB on the .INT routine size, and on the .MAC generated code of a method have been removed. There are, however, some limits that are still imposed by the system:

New %Routine Class

There is a new class, %Routine, with methods for creating and handling routines.

Default Collation for %String

Prior to Caché 3.2, the default collation for %String was STRING. For Caché 3.2, it is SQLSTRING. The main difference between STRING and SQLSTRING is that SQLSTRING does not strip out punctuation characters, whereas STRING did strip out punctuation characters.

The impact on existing applications depends on how a property uses the COLLATION parameter. Assume, for example, that the Person class has a Title property whose type is %String. Prior to Caché 3.2, if no collation was specified for the Title property, then STRING collation was used. If the Title property was indexed, the index was built using STRING collation.

In Caché 3.2, no collation for the Title property causes SQLSTRING collation to be used. So if you have indexed the Title property and you did not specify collation, your index data does not agree with the collation for the property. You must therefore choose one of two options to resolve this:

So if you want to use the new default collation (SQLSTRING) where punctuation is maintained, then choose option 1 (rebuild the index). This is what people would expect from standard SQL.

If you want to continue to use STRING collation (and the existing index structures), then you need to change the property defintion and re-compile the class.

If the Title property already had specified the COLLATION parameter, then no change is needed. The current index already reflects the COLLATION.

Collation Now Applied to Equality Conditions

Prior to Caché 3.2, collation was not applied to the = operator in SQL, but it was applied to the %STARTSWITH operator. To better conform to the SQL standard, collation is applied to the = operator in Caché 3.2.
Note: Because of this equality collation feature, be aware that your 3.2 queries may return different matching data than 3.1 queries.

New String Collation Parameter

STRING collation now accepts an optional MAXLEN parameter. If specified, the collated value will be truncated to the value of MAXLEN. This parameter can be specified any place that STRING is allowed as a collation function. For example:

attribute StrAttrib1 { type = %String(COLLATION="STRING(30)"); }

and

index TruncString { attributes = Name:String(30),City:String(5),State:String; }

You can use this parameter to index on strings that exceed the 255-character limit for Caché subscripts.

New Structure for Indexes

A new keyword, Type, is implemented for indexes. The values are KEY and INDEX. A KEY index is built using only the index attributes, while an INDEX uses both the index attributes and the IDKEY attributes (or system generated ID value).

If the index is not Unique, PrimaryKey, or IdKey, then the default is INDEX, and the indexed attributes are converted according to their collation before storing in the index, with the ID stored as the last subscript. Adding the type=KEY; clause causes the index to be checked for uniqueness, converted, and stores the ID in the data.

If the index is Unique, PrimaryKey, or IdKey, then the default is KEY, and the indexed attributes are NOT converted to their collated form when stored in the index, and the ID is stored in the data. This form of index can only be used for uniqueness checking and equality constraints.

For Unique and PrimaryKey indices, adding the type=INDEX; clause causes the attributes to be converted to their collated form and stored in the index, with the ID stored as the last subscript. Note also that if any of the attributes forming the index are SQL-null, then the uniqueness check is not performed. This form of index can be used for collation-sensitive constraint checking (such as lessthan, greaterthan, startswith, etc.).

Note that all indexes created through DDL are of type INDEX.

New SQL Collation Functions

Two new SQL collation functions have been added:
SQLSTRING[(length)] — prepends a space and strips trailing whitespace, truncating the result at length characters.

SQLUPPER[(length)] — prepends a space, strips trailing whitespace, and converts to uppercase, truncating the result at length characters.

The %SQLSTRING and %SQLUPPER unary functions can be used in the same manner that %ALPHAUP, %STRING, %UPPER, and %EXACT are used in SQL queries.

Also, when defining map subscript definitions, $$SQLSTRING({Field}) and $$SQLUPPER({Field}) can be used just like $$ALPHAUP({Field}), $$STRING({Field}), or $$UPPER({Field}).

Caché SQL also now supports %SQLSTRING and %SQLUPPER in the Collation clause of a field specification in a DDL statement. For example:

CREATE TABLE MyTable (MyField VARCHAR(10) %SQLSTRING)

will create a clases with an attribute MyField which has a datatype parameter of COLLATION=SQLSTRING.

SQL View Keywords

There are two new class keywords for SQL view projection: The name of the projected view is the SQLTABLENAME value, which defaults to the class name.

Other improvements also mean that you can define a view within the same class as the table used in the view. However, if the class is a superclass, do not put the views into it, as you will get a (non-fatal) error that the viewname is not unique, when compiling the subclass(es).

You can even define a view based on another view, within the defining class. You can also make a query into a view AND a stored procedure at the same time.

A better way to define views is with the new view-class syntax, such as:

class SmallView
{
     compileafter = Patient;
     sqltablename = Females;
     view;
     viewquery =
          {
          Select ID,Name,Sex,DOB from Patient
               where Sex="F"
          }
}
This example projects the view under the name "Females".

Dynamic Queries

The following are additions/changes to SQL dynamic queries:

SQL CONTAINS Operators

When values are compared in SQL queries with the CONTAINS ([) or NOT CONTAINS (NOT [) operator, the operands are now compared in their exact format. Prior to this change, the operand values were compared in the collated format. This produced incorrect results for the CONTAINS operator.

Namespace-level Security

Caché SQL now lets you control access via ODBC at the namespace-level, so that users can be granted access rights to specified namespaces only.

Mapping of DataTypes in DDL

Caché now supports a table defining how to map each datatype from a foreign system via DDL to Caché. For example, CHAR might map to %String(MAXLEN=1) or you might change it to %ExactString(MAXLEN=1).

This table is in ^%SYS("sql","datatype").

ALTER TABLE SQL Synax

Caché SQL now supports the syntax:

ALTER TABLE <tablename> DROP CONSTRAINT <constraintname>

For example:

ALTER TABLE CustomerDemo DROP CONSTRAINT FK_CustomerDemo

DDL ALTER TABLE Change

DDL ALTER TABLE ADD COLUMN statements now work in one of the following four scenarios when the column contains a NOT NULL constraint:

1) When a field is added to a table through an ALTER TABLE <tablename> ADD COLUMN statement, and the statement specifies a NOT NULL constraint on the column, and there is no default value for the column; the statement will fail if data already exists in the table. This is because after the completion of the DDL statement, the NOT NULL constraint would not be satisfied for all the pre-existing rows. A new SQLCODE error was created to report this error:

SQLCODE=-304
Attempt to add a NOT NULL field with no default value to a table which contains data.
2) When a field is added to a table through an ALTER TABLE <tablename> ADD COLUMN statement, and the statement specifies a NOT NULL constraint on the column, and there is a default value for the column, the statement will update any existing rows in the table and assign the default value for the column to the field. For example:
     CREATE TABLE Person (
          Name    VARCHAR(35),
          DOB     DATE)
     INSERT INTO Person (Name, DOB) VALUES ('David Jones', 46639)
     INSERT INTO Person (Name, DOB) VALUES ('Kyle Jones', 56239)
     ALTER TABLE Person ADD COLUMN FavoriteColor NOT NULL DEFAULT 'Blue')
will populate the FavoriteColors column with the value "Blue" for the two rows which exist in the table prior to the addition of the FavoriteColors column.

3) When a field is added to a table through an ALTER TABLE <tablename> ADD COLUMN statement, and the statement does not specify a NOT NULL constraint on the column, and there is a default value for the column, no updates of the column in any existing rows is performed. The column value is NULL for those rows.

4) If there is no data in the table at the time of the ALTER TABLE <tablename> ADD COLUMN statement, the column is just added to the table.

QueryToTable Utility

The new QueryToTable utility allows the user to build a new table from a specified query. This is similar, but not identical, to "Temporary Tables". The utility is called by:

QueryToTable^%apiSQL(.query,table,display,.error)

where query is the SQL statement (or array) to execute, table is the name of the table to be created, display is a flag to write ouput (0=no), and error is an array of errors.

Transaction Processing and ActiveX

There are four new methods for the Caché Object factory that support Transaction Processing commands:

Note that TransactionRollBack() rolls back all the open transactions.

Factory.GetConnectionList Method

The new Factory.GetConnectionList() method lets you get a list of possible connections, to provide customized ways of choosing a connection. The method returns:
ConName1,ConString1[;ConName2,ConString2[;...

%OnSave Methods

Caché 3.2 includes the %OnBeforeSave() and %OnAfterSave() instance methods. %OnBeforeSave() is called immediately before the %SaveData() method and %OnAfterSave() is called immediately after it. They are only called if the IsModified flag is set (like %SaveData).

Both methods have an insert parameter which must be declared even if it is not used. It contains a 1 if the %Save is an Insert, and a 0 if it is an Update. The return %Status, and can cause the %Save to fail (with rollback). If %OnBeforeSave returns an error status, then %SaveData and %OnAfterSave will not be called.

The methods are run inside the transaction scope, but outside the locking scope of the %Save (i.e., before a lock is attempted, and after the lock is/may be released).

%DeleteId Method

The %Persistent class has a new method, %DeleteId(), which allows the user to delete an instance using only the ID, rather than the full OID.

Enhancements to %Populate Class

The following enhancements have been made to the %Populate utility class:

Note that the full syntax of POPSPEC at class level is:

<attribute>:<spec>,<attribute>:<spec>,...
The full syntax of POPSPEC at attribute level is:
<spec>
where <spec>=<value generator>[:<collection count>[:<key generator>]]

<value generator>  = ##class(class).method()
                   = .method()   [obj.method()]
                   = other()     [##class(%PopulateUtils).other()
<collection count> = n           [1:1:$r(n+1)]
<key generator>    = default     [##class%PopulateUtils).String(4)]
                   = ##class(class).method()
                   = .method()   [obj.method()]
                   = other()     [##class(%PopulateUtils).other()

%GetObjectId Method

The %CacheProperty class has a new public method, %GetObjectId(), which allows the user to retrieve the ID of a reference property without swizzling the object.

GROUPBY and Expressions

Caché SQL now supports expressions in SQL GROUPBY statements.

%Exists and %ExistsId Methods

Two new methods have been added to every persistent class:

%TimeStamp Class Changes

A new %OdbcToLogical() method was added to the %TimeStamp class, so that if the attribute contains only an ODBC-format date, then the time "00:00:00" is appended.

The %IsValidDT() and %Normalize() methods were also changed similarly, so that you can now set a %TimeStamp field to just a date (e.g., +$H in ObjectScript) and the system appends the time "00:00:00".

For compatibility with other major SQL databases, Caché SQL now accepts a Date as a valid entry for a TimeStamp field. TimeStamp fields can now be compared (with >, <, >=, <=, =, <>, BETWEEN) to Date values.

When comparing/inserting a Date into a TimeStamp field, an explicit conversion should be used, as specified in the SQL Standard:

insert into mytable(ts) values ({fn CONVERT({fn curdate}, SQL_TIMESTAMP)})

Support for ODBC BIT Datatype

Caché SQL now supports the ODBC datatype BIT. The %Boolean Caché datatype will now bind to SQL as an ODBC Type of BIT. Previously, it had been exposed as an INTEGER.

ID Allowed in UPDATE Statement

Caché SQL now lets you include the ID field in an UPDATE statement, but only as long as the value has not been changed.

ResultSet Changes

The ResultSet class now supports up to 16 parameters, from the previous maximum of 8. CacheObject.ResultSet also supports up to 16 parameters.

A new property has been added to the CacheObject ResultSet object. The property QHandle returns the value of the QHandle variable passed through the query, as in the following VB sample code:

Dim rs As CacheObject.ResultSet
Dim syslist as New CacheObject.syslist
Set rs = Cache.ResultSet("Class", "Query")
ok = rs.Execute("a", "b")
qh = rs.QHandle
syslist.Set (qh)
for I=1 to syslist.count
msgbox "Qhandle field" & I & "=" & syslist.item(I)
                next I

Unicode Characters Allowed in Object Identifiers

Any character in the Unicode set can now be used for Caché Object identifiers, such as classnames, methodnames, and propertynames. Specifically, characters in the range 0x100-0xFFFD can be used. For example, a classname can have Kanji characters.

RunTime Format Selection in SQL

A new compiler option is now supported:
#sqlcompile select=Runtime
Input/output conversions are dependent on the %Runtime variable. Currently, the "ODBC" value is supported (meaning ODBC conversion and "" meaning no conversion).

Locking for %CacheSQLStorage

The %AcquireLock and %ReleaseLock methods have been added to the %CacheSQLStorage class. These methods are called by the %Save() method, and call corresponding entry points in the SQL routines.

Additional SQL Identifiers

Caché SQL now supports the @ (ampersand), # (pound), and $ (dollar sign) characters in regular SQL identifiers, but not as the first character of a regular identifier. Note that Caché Object identifiers (classnames, etc.) do not support these characters, only Caché SQL identifiers.

For example, the following are now valid SQL regular identifiers:

My@Table
My#Field
My$Money
X@#$$#@
The following regular identifiers are not valid:
#Number
@Direction
$Money
This change means the Binary Modulo operator (#) can only be used with spaces between the operands and the operator. For example:
NumericField # 10
is valid, while
NumericField#10
will be treated as a single regular identifier.

When creating tables through DDL, the Identifier Translation Table is still used to translate the @, #, $, and _ characters into valid Object Identifier characters. By default, the @, #, $, and _ characters are simply stripped from the identifier to create the Object identifier. For SQL Table names and SQL Field names, these characters are preserved in the SQLTABLENAME and SQLFIELDNAME properties, but for SQL Index Names and SQL Constraint Names, these characters are modified in the resulting table definition. Caché does not support separate names for Objects and SQL for Indices and Constraints.

SQL Functions

A number of SQL functions have been added. For a complete list, see the Caché SQL Functions document.

%apiOBJ Changes

DeleteAll^%apiOBJ() no longer deletes %-classes unless the "p" flag is supplied.

ExportCDLAll^%apiOBJ() has been corrected to run on UNIX and OpenVMS platforms.

A new call, GetLastError^%apiOBJ(), returns the last error status code. Note that GetLastError^%apiOBJ() returns the last error composed by the $$$ERROR() macro, which may not be the last error returned by any function/method calls. The discrepancy comes in if someone composes an error status code and never returns that status code. For example:

set sc=$$$ERROR($$$GeneralError,"Some Error")
quit $$$OK
In this case, GetLastError() will return the $$$GeneralError even though this error is never really used in a return value. In general, you should not compose an error unless you intent to use it as a return code.

Use of Macros in SQL Code

Caché now supports the used of macro references in SQL Computed field code, SQL Trigger code, initialexpression code, and Triggered SQL Computed fields, Normalize methods, LogicalToStorage method, and so on. The macros must be defined in include files specified in the class definition or in a superclass definition.

Note that when these SQL tables are used in non-generated environments (such as programmer-written .MAC code), it is the responsibility of the programmer to include the appropriate files.

Table of Contents | Chapter 1 | Chapter 3 | Chapter 4
Top of Page