Talking to Authorize.net using RPG XML on the IBM i

So, for the last few weeks I’ve been working on adding credit card handling to a legacy RPG application with a static HTML website over the top. The application code itself was written in the eighties and has evolved through a number of different versions of RPG (a mix of RPG2, RPG3, RPG400 and even a liberal splash of RPGLE) so I’ve had an enjoyable task of taking a bunch of old code, re-factoring to RPGLE/free and then adding some funky web services. Talking to Authorize.net using RPG XML over the web is not as hard as it sounds.

Now I’ve added the Authorize.net CIM functions and windows to my website pages, I can manually test the payment process. Now, the authorize.net documentation is not for noobs and makes an awful lots of assumptions, so implementing this technique on IBM i (AS400) using RPG, without a modern PHP Server on the front, is not as clear as it seems. After reading everything I could on the various SOAP, XML, JAVA, RUBY techniques – I’ve opted for the XML route.

The next step is setting up the XML conversation to this lovely IBM i System talk to Authorize.net. After two days of googling, reading, studying, prototyping, re-prototyping, going down dead ends, trying sample code that doesn’t work and repeating moaning and smacking my forehead with a wooden paddle.

So I want to

1. Create the GetTokenFromAuthorize.net.XML from within my RPG program
2. Run the XML so its talks to authorize.net and gets a Token (the unique ID for this payment)
3. Update my HTML page with the Token returned from the previous step
4. Display the Page and let glorious payment be received

I think I finally had a message from the Gods.

I made my first coffee this morning, sat at mission control and *you have mail* and in my inbox I see a fresh article from IBM SYSTEMS MAG entitled:

Generating XML the XMLi Way

In a Nutshell – This article overviews an opensource IBM i application that automates creating XML documents from within RPG programs. So, for my first step in the process [1] GetTokenFromAuthorize — I can either simply generate an XML string in my program and write it to the IFS or I could use this XMLi rather nifty piece of open source to do it so much more gracefully.

I think I’m going with easy option first… for proof of concept and to make sure our IBM i system can *talk* to Authorize dot net

Then its rollout the XMLi solution… very slick piece of opensource.

I’ve spent weeks trying to find any kind of documented examples on how to do this with IBM i operation system and IBM RPG or CL languages… so rest assured further blogs and code examples will be coming. Just check back after Christmas and look for any blogs tagged with Authorizenet

Anyways, less blogging and more reading, head scratching and researching.

Just for my reference it says this:

Back in 2011, we wrote an article called “There’s an RPG App For That!” There, we mentioned the XMLi package, an open-source toolkit from Larry Ducie for the generation of XML. We said at the time that we planned on doing a full report on this tool, and here it is—or at least a first installment. A little later than planned but hopefully you’ll agree that this is a case of better late than never.

The XMLi package offers two different methods for generating XML:

• The first, known as XMLi1, provides a set of APIs that allow you create an XML document in memory, and then either write it to the IFS or utilize it directly as you would need to do if you were building the XML to pass to a Web service.
• The second, not surprisingly known as XMLi2, uses a templating system, which not only contains the XML skeleton but can also include SQL to retrieve the data to be built into the document. This approach is probably harder to learn, but often much simpler to modify to meet changes in the XML document or data selection requirements. As you’ll see when you study XMLi’s supplied examples, it’s an incredibly powerful approach.

The XML Document

To demonstrate how XMLi works, we’ll generate the same XML document using each of these methods in turn. The document is a very simple one but hopefully will illustrate the basic principles for you. Here’s an extract:

<Customers> 
 <RecordCount>12</RecordCount> 
 <Customer ID="938472"> 
 <Name>Henning</Name> 
 <Address> 
 <Street>4859 Elm Ave</Street> 
 <City>Dallas</City> 
 <State>TX</State> 
 <Zip>75217</Zip> 
 </Address> 
 </Customer>
 <Customer ...

Since just about every IBM i system we’ve seen has the QIWS library loaded, we are using the PC Support Customer File – QCUSTCDT from that library as the source of the data for these tests.

Notice that the document consists simply of the root element Customers, followed by a record count and then the Customer element, which will be repeated once for each row in the selected data. We will be using SQL for the data retrieval and this will be equally simple consisting of:

select count(*) from QIWS.QCUSTCDT where STATE = 'TX' 

to retrieve the count of the number of customers and

select CUSNUM, LSTNAM, STREET, CITY, STATE, ZIPCOD 
 from QIWS.QCUSTCDT
 where STATE = 'TX' 

to retrieve the data to be incorporated into the document.

In order to simplify the examples, we’ve hard coded the selection value. In reality, we’d parameterize this to allow other states to be selected, but handling parameters in XMLi templates, while not difficult, is beyond the scope of this article and we will cover it in a subsequent piece.

Using the XMLi1 API Method

As you’ll see, the RPG code is not complex, but in the case of an XML document with a large number of elements it can become rather tedious to code.

Beginning at (a) in the following code, we identify the binding directory supplied with XMLi to make it easy for the compiler to locate XMLi’s subprocedures. This way a simple CRTSQLRPGI (or CRTBNDRPG if we weren’t using embedded SQL) can be used to compile the program.

(b) includes the XMLi prototypes in the program and (c) defines xmlData, the field to hold the generated XML document, followed by xmlFilename, the name of the file to which it will be written.

(a) H BNDDIR('XMLILIB/XMLI') DftActGrp(*no)

(b) /include XMLILIB/QRPGLESRC,XMLI_H

 // Declare field definitions for use by SQL
 d customer E DS ExtName('QIWS/QCUSTCDT')

 // Work Variables
 d recordCount s 5i 0
 d endOfData c '02000'
(c) d xmlData s 10000a Inz
 d xmlFilename s 128a Varying
 d Inz('/Partner400+
 d /CustomersE1.xml')

In the following logic, the call to xmli_useVariable() at (d) identifies the variable in which XMLi is to build the XML document. You can either specify a variable as we’ve done, or have XMLi provide “managed memory” automatically for you.

We next (e) set the format for the generated XML. The PRETTY option causes the XML to be indented to make it easier to read and is a good option to use when testing. For production purposes, you might want to use SIMPLE (no indentation). Other options are available when using the XMLi2 templating approach.

 /Free
(d) xmli_useVariable(xmlData);

(e) xmli_setFormat(XML_FORMAT_PRETTY);

 Exec SQL
 select count(*)
 into :recordCount
 from qiws/qcustcdt
 where STATE = 'TX';

Now we begin the process of generating the actual XML. At (f), we call the xmli_openTag() API to generate the <Customers> tag. Then (g) calls xmli_addElement() to generate the complete <RecordCount> nnn </RecordCount> element.

The actual SQL directives (h) set up the cursor so that we can subsequently loop through the result set. The actual loop begins at (i) in the next code example.

(f) xmli_openTag('Customers');

(g) xmli_addElement('RecordCount': %char(recordCount));

(h) Exec SQL
 declare customerCursor cursor for
 select CUSNUM, LSTNAM, STREET, CITY, STATE, ZIPCOD 
 from QIWS.QCUSTCDT
 where STATE = 'TX';
 Exec SQL
 open customerCursor;

Now that all the setup is complete, we can loop through the result set and build the body of the XML document. This process begins at (j) where we create the opening tag for the <Customer> complex element. Since the Customer element includes the attribute ID, we need to call xmli_addAttribute() to add the attribute value (k). xmli_addAttribute always adds the attribute to the last element opened, so the sequence is important here.

We then proceed to add all of the individual simple elements before finally at (l) using the xmli_closeTag() API to close out the Address and Customer elements.

Once all rows in the result set have been processed, we can proceed to close the Customers element (m) and then call the API xmli_writeToFileWithVar() to write the XML document that has been built in the xmlData variable to the IFS file named in the xmlFilename variable.

(i) DoU SQLSTATE = endOfData;
 Exec SQL
 fetch next
 from customerCursor
 into :CUSNUM, :LSTNAM, :STREET, :CITY, 
 :STATE, :ZIPCOD;

 If SQLSTATE <> endOfData;
(j) xmli_openTag('Customer');
(k) xmli_addAttribute('ID': %char(cusnum));
 xmli_addElement('Name': LSTNAM);
 xmli_openTag('Address');
 xmli_addElement('Street': STREET);
 xmli_addElement('City': CITY);
 xmli_addElement('State': STATE);
 xmli_addElement('Zip': %editc(ZIPCOD:'X')); 
(l) xmli_closeTag('Address');
 xmli_closeTag('Customer');
 EndIf;
 EndDo;

 Exec SQL
 close customerCursor;

(m) xmli_closeTag('Customers');

 xmli_writeToFileWithVar( xmlFilename
 : xmlData
 : XML_ENCODING_UTF8);

That’s all there is to it. Effectively, we use API calls to open tags (xmli_openTag), add attributes to them (xmli_addAttribute), close them (xmli_closeTag) and to write complete elements (xmli_addElement).

A number of other APIs are available; see the documentation for examples.

In a previous article “Using CGIDEV2 for Generating XML,” we demonstrated a method for generating XML using CGIDEV2’s templating constructs. XMLI2 takes that concept to a whole new level. Let’s start by taking a look at the template. Those of you familiar with XML will recognize it as being very similar to an xsl transform (XSLT). It looks a little scary at first, but once you understand the basics, it’s really quite straightforward. Luckily, the XMLi package includes a dozen or so worked examples and the documentation describes their function in detail.

(A) <xmli:template xmlns:xmli="http://www.sourceforge.net/xmli"
 ccsid="1208" format="pretty">
(B) <Customers> 
(C) <xmli:run-sql name="custCount" 
 statement="select count(*) from QIWS.QCUSTCDT
 where STATE = 'TX'"> 
 </xmli:run-sql> 
(D) <xmli:for-each> 
(E) <RecordCount><xmli:value-of select="custCount.1" />
 </RecordCount> 
 </xmli:for-each> 

The first directive (A) identifies the CCSID and format for the resulting XML.

The next thing you see is the <Customers> tag (B). Because it does not begin with “xmli:” (i.e., it’s not in the xmli namespace) XMLi simply passes it through to the output stream.

Next comes the run-sql directive (C) that will execute our first SQL statement. We name the result set “custCount” so that we can reference the individual columns in subsequent operations.

Next at (D) we begin a for-each loop that will allow us to iterate across the result set. The construct for-each is available in many modern languages and sets up an automatic loop across all of the rows in the result set. This is much simpler than hard coding a loop as we had to in the XMLi1 example. Of course, in this particular instance there is only a single row – the count.

(E) The RecordCount tags will simply be passed through to the output as before. The actual value of the count is included in the output courtesy of the value-of directive. custCount.1 refers to the first (and in this case only) column in the row.

Now that you’ve seen the basics, the rest of the template should hopefully be a little easier to follow.

(F) <xmli:run-sql name="custRow" 
 statement="select CUSNUM, LSTNAM, STREET, CITY, STATE, ZIPCOD
 from QIWS.QCUSTCDT where STATE = 'TX'"> 
 </xmli:run-sql> 
(G) <xmli:for-each> 
(H) <Customer ID="${custRow.1}" > 
 <Name><xmli:value-of select="custRow.2" /></Name> 
 <Address> 
 <Street><xmli:value-of select="custRow.3" /></Street>
 <City><xmli:value-of select="custRow.4" /></City> 
 <State><xmli:value-of select="custRow.5" /></State> 
 <Zip><xmli:value-of select="custRow.6" /></Zip>
 </Address> 
 </Customer> 
 </xmli:for-each> 
 
 </Customers> 
(I) <xmli:write-to-file 
 path="'/Partner400/CustomersE2.xml'" />

At (F), we execute the main SQL directive and associate the name custRow with it. We then loop over the results (G).

Earlier when we incorporated the count in the document (E), we were able to use the value-of construct. In this instance (H) however, we can’t because this time, we need to embed a value in the middle of an XML tag, rather than as the value between tags as we did before. XMLi supports this by using the ${ } sequence to wrap code that XMLi will interpret and replace. In this case, we’re just using it to retrieve a value from the result set. In future articles, we’ll discuss other uses for this construct.

Once we drop out of the for-each loop (I), we can then write the generated XML to the stream file.

The RPG Program

Now that you understand the template, let’s take a look at the RPG code that executes it.

The first thing you’ll notice about this version is the simplicity of the RPG code. So simple in fact that, in a case like this, we could have just run the supplied command RUNXMLI and not written a single line of RPG code – the template would do all the work. In fact, the command would even allow you to specify the output file name as a parameter.

Beginning at (J), we define the name of the template we’ll be using. We had to resist the temptation to use the new free form dcl-s syntax for this D spec – it makes it so much easier to type long initialization values! However, we don’t want to give the impression that XMLi only runs on V7 systems – it will run on all systems V5R4 and later.

 H DFTACTGRP(*No) BNDDIR('XMLILIB/XMLI')

 /include XMLILIB/QRPGLESRC,XMLI_H

(J) d templateName s 128a Varying
 d Inz('/Partner400/+
 d TemplateE2.xml')

 /Free
 // Load and run the template...
(K) xmli_loadTemplate('CUST' : templateName);

(L) xmli_runTemplate('CUST');

 // Unload the template
(M) xmli_unloadTemplate('CUST');

 *InLR = *On;

The real work begins at (K) where we tell XMLi to load the template file. Notice that we associate the template with a simple name (CUST) and it’s this name that we use at (L) to cause the template to be executed. The same name is used at (M) where we inform XMLi that we have finished with this template and it can be unloaded from memory.

The association of a name with a template is not just to simplify subsequent function calls. It’s important because XMLi allows for up to 256 templates to be active at any given time and this feature allows you to build complex documents easily by combining multiple templates. However, that’s well beyond the scope of this article.

Amazingly Powerful

XMLi is a great tool kit that can really simplify the process of generating XML. In particular, the templating system is amazingly powerful – we haven’t even begun to scratch the surface in this article. For example, in addition to the functions provided by the basic package, you can add your own coding to provide additional custom processing within the template. One feature that the author Larry Ducie has implemented allows for the template to not only generate the XML document but to email it. Of course, this principle could be expanded to include calling a Web service, FTPing the file and more.

The XMLi package can be downloaded from sourceforge.net/projects/xmli/. In addition to the save file of the XMLi library, it includes multiple examples and comprehensive documentation to explain their operation.

Take a look – a little study could save you many hours of work.