The End

Since no one really reads, this blog is over.

The house fell for once.

Lists with Caché

Caché Lists - an object oriented approach

Unlike Java, the Intersystems Caché doesn't have large API to manipulate data structures of collections (lists, queues, sets, etc). In fact, when we work with objects in Caché, we can rely only on two pre-defined structures: lists and arrays.

For Caché, lists are, as the name implies, lists of objects (instances of classes you write) r lists of basic data types (as %Integer or %Float). Each element of a list is associated with a position in it. Caché lists are very similar to Java lists (comparing with List interface, about implementation, we can not make good comparison, since Java can have lots of implementations, as ArrayList orLinkedList).

With Caché, the abstract class %Collection.AbstractList gives us the basic interface to work with lists. Adapted from docs, the code snippet below shows how to create a list, 4 elements insertions, and how to iterate the list:

; creates new list
 Set list=##class(%ListOfDataTypes).%New()
 ; here we add four elements in the list, four strings
 Do list.Insert("DArtagnan")
 Do list.Insert("Athos")
 Do list.Insert("Porthos")
 Do list.Insert("Aramis")
 ; how to iterate the list
 For i=1:1:list.Count()  Write list.GetAt(i),!

Caché arrays are a little bit different. They seem a lot like Java Map's interface. A Caché array needs, besides an element, a key to associate with it. Through this key, you can access the element in the structure. This kind of structure are useful to implement hash tables, as Java's Hashtable class.

As examples are the best way we can learn, below there is one, adapted from the documentation on class %Collection.AbstractArray, the 'interface' of Caché arrays:

; creates new array 
 Set arr=##class(%ArrayOfDataTypes).%New()
 ; put itens in array - the second parameter is the 'key'
 Do arr.SetAt("red","color")
 Do arr.SetAt("large","size")
 Do arr.SetAt("expensive","price")
 ; how to iterate arrays 
 Set key=""
 For  Set value=arr.GetNext(.key) Quit:key=""  Write key,":",value,!

(Just pay attention to SetAt method, similar to java.util.Map's put. The order of parameters is inverted in those two. As put has the key as first parameter, SetAt the key is the second.)

With both examples above, we use basic data types (%String, in both cases) as elements in lists and arrays, but we could have used objects. To use objects, we can just swap %ListOfDataTypes and %ArrayOfDataTypes for %ListOfObjects and %ArrayOfObjects.

When we use collections as classes properties, we can explicitly declare the property as an object of Array/List classes or we can declare them as below:

    ; a %String list 
    ; instead: Property Colors As %ListOfDataTypes 
    Property Colors As List Of %String

Just keep in mind that arrays and lists classes are classes of Serial type, i.e., they inherit from %SerialObject, therefore, they are 'serialized' inside other classes when they are persisted. If a list is too big, or even if the list has not that much elements, but the elements are %SerialObject too, there is a chance to occur a error.

Consider the class below:

Class Test.Bang Extends %Persistent [ ClassType = persistent ] {

Property alist As list Of %String;

ClassMethod test(amount as %Integer) 
  {
    set a = ##class(Test.Bang).%New()
    w a.%Save()
    For i=1:1:amount  {
        do a.alist.Insert(i)
    }
    w a.%Save()
  }
}

You can test calling "test method" with different values. With Caché 5.2, if you call "do ##class(Test.Bang).test(10000)", you can check the error. I just heard that newer versions, as 2007, the maximum string size was increased, so this kind of error will occur less frequently (although I didn't check this info myself).

How I miss ROWNUM sometimes...

Today I needed to calculate the average from a field from a table in Intersystem's Caché. The logic was very simple: I just neede the average of prices of last N purchases, after some date.

Oracle has the pseudo-column ROWNUM, which makes my problem very easy to be solved. I would have something like:

SELECT AVG(PRICE)
FROM PURCHASE
WHERE DATE > '01/01/01' AND ROWNUM <= 20
ORDER BY DATE DESC

In this simple example, we calculate the average of price of last 20 purchases made after day 01/01/01. Quite simple, no?

I searched for something similar in Caché, but I couldn't find it. I found some related stuff, like a variable for the number of lines a query resulted, or the TOP statement, used within SELECT. Unfortunatelly, nothing helped me much, so I had to open a cursor (or I could have used a ResultSet, if I wanted to), and iterate through the results, and calculate the average myself.

So, the query took this form:

SELECT TOP 20 PRICE
FROM PURCHASE
WHERE DATE > '01/01/01'
ORDER BY DATE DESC

See the 'TOP 20'. It that for the query I just want the first 20 records. Unfortunatelly, we can not use it on WHERE statement, so we can not use it for the AVG function.

I will not describe how I iterate through a cursor and calculate the average, since it's quite trivial, and I'm tired now.

Today, I'm done.

Execute code dinamically with Caché

...Or How to evaluate and execute an user input's expression

The scenario: with the application, an user can input some math expressions, using variables previously defined (by himself or another users). These variables might be simple or they might depend on some parameter. An example of expression is below:

(0.50 * ([VAR1(I)] / [VAR1(0)])) + (0.50 * [VAR2])

where:

• VAR1(I) is a variable, which depends on parameter "I".


• VAR1(0) is a variable (in fact, the same variable above), but depends on another parameter, "0".


• VAR2 is a simple variable.

By convention, variables are written between "[]". Most variables are in fact, historical series of values, and those parameters indicate the specific time/period to be considered.

The problem: given a time interval, I need the value the expression evaluates to. Or, with another terms, I need to execute the expression, as if it was part of the program. Some other script languages have the eval function, which takes a text (i.e., an String) and executes it as if it were program code. In this link, an example of JavaScript's eval(). With Caché Object Script, we have the command XECUTE, which does the same thing, it gets the content of an string and executes it, i.e., it does macro substitution.

Well, even with this feature, I still need to identify within the %String, each variable. To make this, I wrote a little function, which I put in a .MAC file:

(A snapshot of the function, the text source code is here.)

For this function, we pass as parameters: the expression, two characters used to delimit what is considered a variable, (in our case, the characters "[" and "]"), a %String to be place where each variable was, and another character to delimit parts of %String (something used a lot with MUMPS-like systems, to create lists - without list functions - see here some discussion.)

Using the expression above in this function, with "?" as the %String to replace the variables, and "~" as delimiting character, we get:

(0.50 * (? / ?)) + (0.50 * ?)~VAR1(I)~VAR1(0)~VAR2

The next step, we get from the system the value of each variable, and replace it within the expression. I will not show these steps, since they are very specific to our problem. After that, our expression String is something like:

(0.50 * (12.3 / 10.88)) + (0.50 * 10)

Note that the expression is mathematically correct, and the precedence of operator is explicit with the use of parenthesis. This REALLY is something to be done, since Caché DOES NOT follow math conventions of operators precedence, as I already posted.

Now, we can finally execute the expression. First, we set the command SET inside a %String, as follows:

SET someVariable = "SET result = " _ stringWithExpression

And then we call XECUTE:

XECUTE someVariable

That's it! The value of the evaluated expression is in variable "result"! Using XECUTE is very handy, but you might pay attention to not abuse it's use, since it can turn your code into a really great mess!

Uploading files with Caché and CSP

Uploading files with Caché and CSP - or... My first CSP

As I already said here (or not), my work is almost all done with Intersystems' Ensemble (and consequently with Caché, since Ensemble is built upon it). Therefore, I am not generally involved with user interfaces and these kind of stuff, but this week I needed to do a web page to upload files to my server. So, lets learn some CSP - or Caché Server Pages. In fact, I didnt reaaaally learned them, I just got the stuff to do what I needed. And I thought it'd be nice to share the experience with you ;)

As CSP Introduction page says, you have two paths if you´re willing to work with CSPs: you can write Html pages and embed them with csp tags, much like JSP (Java Server Pages) or PHP, saving them with .csp extension; or you can write a Caché's class, where you can output the html, the same approach of Java Servlets.

Within examples (in namespace SAMPLES of Caché) there is a .csp file with an example of upload (as you can guess, it is csp/samples/upload.csp). Taking this as base, I adapted the necessary to my needs.



(Upload example page, and the resulting page after an upload)

Well, as I couldn't place the source code here at blogger, in a good way, I will put an image, and let a link to source code text.

(Image with source code)

The logic behind this csp page is very simple: when you open this page, calling from a parent window, you pass the request parameter 'campo', if not, it will show you nothing. Then, the csp code checks if a file was uploaded, if not, it renders an html form with fields for upload (the form's submit points to the same csp); if the file was already uploaded, it sets the file name to an html field from the parent window who called this csp, and shows a link to close this current window.

Lets view with more details some of this.

<CSP:CLASS INCLUDES="SOMEINCFILETOINCLUDE">

Here we point to an .INC file (without the inc extension) that we will use in the page, i.e., we are including this INC file. In Caché, INC files are used to define macros (like C/C++ macros), and are very used to both implement routines and define constants.

/***********************************

<csp:if condition='($data(%request.Data("campo",1)))'>

Here we show an specific tag from csp, which means a condition, the famous 'if'. The condition is the value of attribute 'condition', of 'csp:if' tag, and it may contain COS (Caché Object Script) code, returning true or false. In our code, we are just checking if the http request (%request object) has as one its parameters a field named 'campo'. More info about %request object can be found at %CSP.Request class doc page.

/***********************************

<csp:if condition='($data(%request.MimeData("FileStream",1)))'>

Other if, but a little different. This time we do not check the http request parameter with %request.Data, but with %request.MimeData. The reason is because this time we are checking if a file was sent, using MIME.

/***********************************


File uploaded: <b>#(%request.MimeData("FileStream",1).FileName)#</b><br>
Size: <b>#(%request.MimeData("FileStream",1).Size)#</b><br>
Type: <b>#(%request.MimeData("FileStream",1).ContentType)#</b><br>


When we sent a file, %request.MimeData("PARAMETERNAME",1) return an object %CSP.BinaryStream. The above code shows on screen some info that we can get from this stream object. Note that the values to be rendered on the screen are between '#(' and ')#'.

/***********************************

<script language="Cache" runat="server">

Here we declare a code block of COS to be executed at the server. In this block, if we want something to be shown in the resulting html, we must write at the standard output, using Write command.

/***********************************

set dname = $$$SomeDirectory

Still within the block we opened above, here we create and set a value to the variable 'dname'. This value is a macro, which is indicated by the three $ ($$$SomeDirectory). This macro should be defined in some of .INC files we imported, like commented above.

/***********************************


Set stream=##class(%FileBinaryStream).%New()
Set stream.Filename= dname _ fname

do stream.CopyFrom(%request.MimeData("FileStream",1))
do stream.Flush()
do stream.SaveStream()


Here we create an stream to a file (class %FileBinaryStream), and we set it to variable 'stream'. Then we set the file's name, and use the method CopyFrom, to copy the content of the stream sent through http, to the file.

/***********************************

<csp:else>

Here we finish the 'if' block, and start the 'else' block.

/***********************************

<form enctype="multipart/form-data" method="post" action="este_arquivo_csp.csp">
Arquivo: <input type=file size=30 name=FileStream>
<p>

<script language="Cache" runat="server">
New bytes
Set bytes="<input type=""hidden"" name=""campo"" value=""" _ %request.Data("campo",1) _ """ > "
Write bytes,!
</script>

<p> <ul><input type="submit" value="Upload file"></ul> <p>
</form>

The 'else' block has the definition of an html form, to upload the file to the server. Note that inside de html form definition, we mix some Caché code, to place a hidden field within the form, to keep the value of parameter 'campo', which was passed. This is one solution, another one (maybe more 'elegant') would be use sessions, but as this is very simple case, I didn't check how to manage sessions with CSPs (but of course, you can explore it yourself).

/***********************************

As this post got bigger than I expected, I will finish it here. Within CSP file there are other features (as some javascript, and COS directory apis), but I will not comment them now. Besides, as I simplified the csp a lot, and I removed all stylish stuff. For you to use it, you will have to change some things, like the import of .inc file.

That's all folks.

Some curious stuff from Caché

As Intersystem's Caché Object Script is a "typeless language", sometimes it produces curious results, especially if you, reader, is more used to math notations than "stuff" of IT.

As an example, lets open a Caché's terminal window, and declare two variables*:

USER>set x = "5.10"

USER>set y = "5.1"

USER>write x
5.10
USER>write y
5.1

Remembering that Caché, (as good and old C) treats 1 (one) as TRUE, and 0 (zero) as FALSE*:

USER>write x = y
0
USER>write x > y
0
USER>write x >= y
1




It all means that (x = y) is false, (x > y) is also false, but (x >= y) is true! Or, "x" is not equals to "y", and "x" is not greater than "y", but "x" is greater than or equal to "y", hohohoho 8-)

Lets give some thoughts about it:

• The first statement (x = y) is false because in this case, Caché assumes an iguality test between two Strings, and String "5.1" is different from String "5.10"
  
  
• The second statement (x > y) is false, not because Caché compares String's lengths (x < y also returns false), but because when the operator ">" is used, Caché dynamically casts the variables' values to numeric values, and because "x" IS NOT GREATER THAN "y" ("x"'s value is numerically EQUAL TO "y"'s value, since 5.1 = 5.10), the overall result is false.
  
  
• Well, by now you might already have figured out why (x >= y) returns true, right? When using "<", "<=", ">", ">=" operators, Caché tries to cast the variables to numeric values, and because numerically "5.1" IS EQUALS to "5.10", the expression (x >= y) is true.
  

All this might confuse some people not used to programming, just as Caché's operators precedence, which I already talked about.

* Some tips: the "set" command is used to initialize/assign a value to a variable, and the "write" command (just as it says) writes to the current output device, in our case, the terminal window.

Ensemble Best Integration Software in the market? Not really...

Some weeks ago I was searching something with Big G (I don't remember exactly what), and I found this news: Report recognizes Ensemble as best Integration Software of the market (the news is in portuguese, and I just adapted the title). This really surprised me! I'm not saying Ensemble is a bad software, but the fact is, that it is faaar from being what I would like to work with (OK, there are parts of it that are reaaaally terrible).

I followed the link, and after I have read the page, I just realized that this, as many news calls, is (at least) incomplete. Lets to the facts: A report really pointed this 'leadership' of Ensemble. However, the report aims only for american Healthcare IT business, which is much more restrict than the term 'of the market' means.

The company which published this report is KLAS, which in their words "is the only research and consulting firm specializing in monitoring and reporting the performance of healthcare’s information technology (HIT) vendors". The report is not completely available (at least, not free at the site), but you can get a sample of it here.

As it was done, the result of this report didn't surprised me that much. Caché (and therefore, Ensemble, which is built upon it) is a very strong player in the market of Healthcare IT, inherited from MUMPS. Although Intersystems doesn't like to mention it. Caché is an implementation of MUMPS (with some new features, but the 'spirit' is the same).

IMHO, if this report was made not focusing this specific market, Ensemble would not be that well. I don't have data to confirm this, so if someone knows some reliable data, tell me! (but please, I don't want those 'white papers' from their own company, which is probably written by marketing division)

Web Services with Caché/Ensemble - The Quest - chpt 3

And this is the end of trilogy...

As I said at the end of last post, the chapter 2, when I was writing the SOAP-Body manually, the requisition message was correctly understood by the Web Service. However, for some unknown reasons, the response message was not returning correctly to my code in Caché Object Script. At first, I thought it wasn't working at all, but after I analyzed the TCP stream with a sniffer, I realized that the Web Service was returning correclty. The problem was with Caché/Ensemble, who didn't understand the answer message.

Wondering if the problem wasn't with my modifications, I tested all the WS Client classes without any modifications, but the error was the same. Conclusion: all work so far was for nothing. So, after analyzing Caché's source, and through some little clues the error on screen, I realized that Caché was giving error because it tried to instantiate a class based in WS answer message, but it failed:


(an example of a response message, informing the request was correctly processed)

By then, tired of navigating through source code and bad written documentation, I decided to went down: since I the Caché's %SOAP package could not work fine with the given Web Service, I was going down in the protocol stack, to make it work. Caché doesn't have 'native' classes to deal straight with HTTP (at least, I didn't find any), but Ensemble has an HTTP "Adapter".

(One parenthesis: Adapters in Ensemble are pieces of software which have the function of connecting Ensemble with other systems/technologies. The documentation on this Adapters is really bad, as an example, in this page listing the Adapters, the HTTP is not even listed, and a lot of Adapters there, simply doesn't exist here, in my installation. Freaking...)

After a little testing, I discovered that I was not able to use HTTP Adapter with the Web Service, because this Adapter doesn't let me set a header SOAPAction, in HTTP headers, which is mandatory for this Web Service to work.

As I couldn't use HTTP classes, I went one step below in protocol stack, and use TCP. Ensemble's TCP Adapter is nothing more than a wrapper for Caché's Socket classes, something I already talked about in two previous posts, here (about client sockets) and here (about server sockets).


(protocol stack - almost at IP level =P )

The good thing in using TCP is that you have control over everything, of the HTTP headers, the SOAP XML message, etc... The bad thing is that in order to use it, besides you must understand a great part of all this stuff, you have to write all manually. But as the Caché/Ensemble tools availables were just giving me headache (and errors), I had not much choice. XD

Take a look how it ended (more or less, here I simplified some stuff):



Finally, with a horrible code and using some anti-patterns, all worked. And all that with Ensemble, which, here, some call "Integrator"... I wonder if it wasn't...

Web Services with Caché/Ensemble - The Quest - chpt 2

And the quest continues...

I had based my implementation of all internal stuff of Ensemble
at previous Web Service/WSDL the third-party company gave me. (If you wanna know what stuff I mean, it's BPEL e Ensemble's "Business Operations".)

With the previous WSDL, the Caché's WS Client wizard generated all the structures as classes, i.e., if I had to send an structure "Person", I would have a generated class "Person", with its properties "Name", "Age", etc..., and the method's parameter would be an instance of this class. However, with the new WSDL, the wizard (I dont know why) generated the methods with each property of the structure as a separated parameter. If before it was:

updatePerson(p as Person)

Now it generated:

updatePerson(name as %String, age as %Integer, ...)

And OF COURSE it broke down all my previous work. For me not to have to refactor all other classes, I thought it would be better to change manually the classes generated by the wizard, adapting them for the classes I had.

So, I continued my quest, changing the Web Client class generated:

(before)

(after)

Well, as naive as someone can be, I thought just this would work. I compiled and tested... and didnt work. To not extend the description of my struggle, I found out that with my changes, it was not writing correctly the Body of the SOAP Envelope, as the Web Service expected, with the namespaces correctly declared.

Looking for some solution, I saw the property "WriteSOAPBodyMethod" from the class %SOAP.WebBase, which is superclass of %SOAP.WebClient. This property allows us to define a method to write a custom Body for a SOAP message.

Using it, we have:

(writing custom SOAP-Body - advice: do it as last solution)

With this little "trick", I finally got all my other code intact and working, and it really called the Web Service correctly. But... there was a catch, always a catch...

The quest was not over yet...

To be continued.

Web Services with Caché/Ensemble - The Quest - chpt 1

I have been creating SOAP Web Services with Caché/Ensemble for some time now, and it is very easy, it is as easy as use an annotation with Java, the difference is that you have to extend a class (not a big problem, since Caché supports multiple inheritance) besides annotating the method you want to expose as a service. So, I thought that it would be nice and easy to use Caché/Ensemble as a Web Service client. I was wrong.

It all started when I finally got in hands the WSDL describing some services from a software the company had purchased. These services were needed to allow ours systems to integrate with the new software (which is almost a black box to us).

Since I had made some tests creating Web Service clients for another services and WSDLs, I was pretty confident it would be easy. Too wrong, I was, as Master Yoda would say. Using the WS Client generator Wizard, I just got an error screen. Unfortunately, as you can see below, Caché/Ensemble didn't give me quite nice and enough information, so I got some doubts: the WSDL I got from the third-party software-house was broken/wrong/f***ed up, or Caché just doesn't understand some stuff? (At this point, both hypothesis had the same probabilites to me.)


(Screen with WS client generator wizard error)

So, I thought I'd test the Web Service using soapUI, a program made with Java, that allows you to test SOAP Web Services. I talked about it in last post.

Using soapUI, I could not only create the correct structure of SOAP messages, but also execute the service. This led me the conclusion that there were something wrong with Caché's wizard. As I am a curious person, I tried to figure out what was causing trouble in that WSDL. After half day lost, comparing it with other WSDLs and navigating into specs, the answer finally arised: the request message had two elements, when it usually have just one (usually the message object itself):


(the piece of WSDL which was causing me trouble)

I just erased the second element tag from the message, and the wizard worked! I felt relief, since in this WSDL there were other methods (which didn't cause me problems), and the wizard just stopped when it encountered an error. Unfortunately, my relief didn't last long...

With an older version of this Web Service (one that was given to me during the third-party company development), Caché's WS Client wizard worked well, so I had all my implementation based in it. Of course I was expecting some changes, but I didn't thought there were so many (and so big)... I was wrong, very wrong...

To be continued...