TCP/IP

Transmission Control Protocol over Internet Protocol (TCP/IP) is the most commonly used communication protocol in use today. It provides reliable delivery of a stream of data over interconnected networks. This reliable transport provides the basis for a wide variety of application-level protocols, including Hypertext Transfer Protocol (HTTP) for web pages and web services, Simple Mail Transfer Protocol (SMTP) for email, File Transfer Protocol (FTP) for copying files, Telnet for terminal connections, and many other standard and custom protocols.

The basic design of most TCP/IP application protocols is a server that is waiting for client connections. When a client connects to the server and establishes the reliable bidirectional TCP/IP stream, the application protocol defines the order and content of messages between the client and the server.

SSL/TLS

Secure Sockets Layer (SSL) was the predecessor of the current Transport Layer Security (TLS). TLS uses certificates and cryptographic protocols to secure data that is transported over TCP/IP. The standard TCP/IP applications on IBM i (HTTP, FTP, and Telnet) and the IBM i host servers can be configured to use TLS.

To use TLS for TCP/IP communications, server certificates must be generated, signed by a certificate authority (CA), and associated with the application. The CA can be the local CA that is created on the IBM i server, an internal enterprise CA, or an external public CA. The advantages and disadvantages of each of the options are described below:

The certificates that are required for TLS are created and managed through the Digital Certificate Manager (DCM) web interface. The IBM i information center provides detailed instructions for each of the configuration steps. The specific certificates and configuration steps that are required are described below:

HTTP/HTTPS

HTTP is the protocol that is used by web servers and web browsers. IBM HTTP Server for i (based on the open source Apache web server) can be used to serve static Hypertext Markup Language (HTML) files that are stored in the integrated file system (IFS) or in physical files, or images, and other file types (such as PDF) stored in IFS. In addition, it can serve dynamic pages in which the content is generated by programs that are written in any of the ILE programming languages. The interface between the web server and the programs follows the Common Gateway Interface (CGI) standard with some additions specific to IBM i, including the conversion between the ASCII character set that is commonly used for web pages and the EBCDIC character set that is used on IBM i.

A simple CGI program can be written in ILE RPG by creating the ILE RPG prototypes for a few C functions and using them to retrieve information about the request and send the dynamically generated reply.

The following program (Example 5-1) creates a simple web page that contains the query string (the part of the Uniform Resource Locator (URL) after the?).

ctl-opt dftactgrp(*no) bnddir('QC2LE') actgrp(*new);
// Include the C prototypes
/copy cgipr
// Variable to hold the query string envirnoment data
dcl-s envval varchar(200);
// Retrieve the query string (the part of the URL after the ?)
envval = %str(getenv('QUERY_STRING'));
// The CGI interface requires that the content type be specified.
// Content types that start with "text" will be converted to ASCII if
// the server is configured for conversion.
// The HTTP header is separated from the HTML content with a blank line,
// so two CRLF strings are appended to the line.
printf('Content-Type: text/html' + CRLF + CRLF);
// Start the HTML output
printf('<html><head><title>Simple Web</title></head>');
// Complete the HTML output which includes the request query string
printf('<body><h1>Hello, %s</h1></body></html>' +CRLF:envval);
// The generated HTML will be sent to the browser.
return;

The following prototype include (Example 5-2) contains the functions that are needed for the
program

// Simple prototype for C printf function that handles from
// zero to five string substitution values
d printf pr 10i 0 extproc('printf')
d fstring * value options(*string)
d str1 * value options(*nopass:*string)
d str2 * value options(*nopass:*string)
d str3 * value options(*nopass:*string)
d str4 * value options(*nopass:*string)
d str5 * value options(*nopass:*string)
*************************************
* Prototype for C function to retrieve an environment variable
*************************************
d getenv pr * extproc('getenv')
d varname * value options(*string)
*************************************
* EBCDIC Carriage return/line feed that will convert to ASCII properly
*************************************
d CRLF c x'0D25'

Although a program that produces simple HTML can hardcode the output, this process becomes cumbersome for more complex pages. One no-charge tool that can be used to map field data into HTML templates is CGIDEV2.

An HTTP server is configured by using the IBM Web Administration for i web interface. That interface provides the tools and wizards that are necessary to set up quickly a web server on IBM i. The configuration of a secure HTTP server (HTTPS) also requires the use of the DCM tool.

FTP/FTPS

The FTP client and server applications on IBM i provide a standard way of transferring files between heterogeneous systems. The IBM i implementations of this protocol include the capability of converting data from standard IBM i physical files (EBCDIC with fixed-length records) to and from the ASCII variable length record stream files common on other platforms. Although the FTP client is often used as an interactive application, it can be used in scripted or batch applications by creating input and output physical files and overriding the FTP input and output to those files. The instructions for doing this are available at the IBM i information center.

FTPS is an extension to FTP that uses TLS to provide secure transmission of the data. FTPS is enabled on the server by assigning a certificate in DCM and using the CHGFTPA command to enable the option. FTPS support on the client is specified by the SECCNN parameter.

SSH/SFTP

Secure Shell (SSH) is a protocol for providing secure communication between systems. It can be used for remote sign-in, remote command execution, file transfers, and for providing secure TCP/IP socket tunnels. The SSH File Transfer Protocol (SFTP) is often used for secure file transfer between heterogeneous machines.

The SSH protocol does not use digital certificates for authenticating keys. Instead, it uses manual verification on the first connection and stores this confirmation for subsequent connections. Another significant difference between FTPS and SFTP is that FTPS provides various automatic data conversion options, and SFTP simply transfers the data without conversion. If SFTP is used for transferring data from IBM i physical files, it is necessary to copy the data to and from IFS with the CPYTOSTMF or CPYFRMSTMF commands with the appropriate conversion options.

Sockets

Although there are various application-level protocols for communicating over Internet Protocol networks, there is often a need for data communication that does not match one of the standard protocols because of either application or performance requirements. The TCP/IP socket interface can be used to write custom interfaces to standard protocols, to write interfaces to protocols that are not supported by standard IBM i applications, or to write
interfaces to custom protocols.

One important property of a TCP/IP socket connection is that it is a stream connection. This means that while the bytes are received in the same order that they are sent, there is no guarantee that they are received in the same blocks as they were sent. For example, if 20,000 bytes are sent in a single operation, they are not necessarily received in a single operation even if the receive function is provided with a large enough buffer. The application-level protocol must provide some mechanism to delimit messages or records that are sent over the stream. The most common mechanisms are providing a length before the message data or delimiting the messages with a delimiter character. It is easier to program the receive function if the length option is used because the message data does not have to be examined for the delimiter character.

This example calls a method to receive a four-byte length field from a socket.

d socket s 10i 0
d length s 10i 0
d buffer s 1000
d rc s 10i 0

rc = FillReceive(socket:%addr(length):%size(length));
if rc = 0;
rc = FillReceive(socket:%addr(buffer):length);
endif;
...

The following code is the code that receives a four-byte value from a socket request.

*************************************
* Receive a specific number of bytes from a socket
*************************************
p FillReceive b export
*************************************
d FillReceive pi 10i 0
d socket 10i 0 value
d buffer * value
d size 10i 0 value
d*
d length s 10i 0
/free
dow size > 0;
length = recv(socket:buffer:size:0);
if length < 0;
return length;
endif;
buffer += length;
size -= length;
enddo;
return 0;
/end-free
p FillReceive e

Server applications
Generally, a TCP/IP socket server application listens on a specific port (which is identified by a number up to 65535) for a client to connect. When a client connects, the server accepts the connection, receives the client requests, and sends the appropriate responses. It is often the case that the server must be able to handle multiple clients simultaneously. In those cases, it is necessary for the server to use multiple processes or threads. A technique that is commonly used in example code for other platforms is to create a process for each incoming client request. This does not scale well on IBM i because of the amount of processing that is necessary to start a job. A better technique is to start multiple jobs to process client requests and distribute the incoming client requests to those jobs. There are various techniques to do this task. One simple and reliable technique takes advantage of user queues to allow a dispatcher job to find an available processing job and give a client connection to that job.

Dispatcher job flow
The following list provides an outline of the process for a dispatcher job flow

1. Create a LIFO user queue.
2. Submit a pool of processing jobs.
3. Open a listening socket.
4. Do the following until server shutdown request occurs:
a. Wait on accept() for an incoming connection
i. Receive a process job identifier from a user queue.
ii. If none is immediately available, submit another processing job and wait on a queue
for a job identifier.
iii. Use the give descriptor() function to send a connection to a processing job.
b. Notify the processing jobs of the server shutdown.

Processing job flow
The following list provides an outline of the process for a processing job flow:

1. Do preconnection work (open files and other actions).
2. Do the following until a server shutdown request occurs:
a. Send a job identifier on a user queue.
b. Wait on the take descriptor() for a connection.
c. Do the following until a protocol defined close occurs:
i. Receive a message from a client.
ii. Process a request.
iii. Send a response to a client.
d. Close the connection.

The use of a LIFO user queue causes the most recently used job to be used for a new connection, which increases the chance that it is still active in memory. The dispatcher job can also limit the number of processing jobs. In that case, the dispatcher just holds the client connection until one of the processing jobs becomes available.

Client applications
In general, client applications are simpler than server applications because they do not have to handle multiple simultaneous connections. The client application opens a socket connection to a server, sends and receives the application defined messages, and then closes the connection.

SSL/TLS programming
There are two sets of APIs that can be used to write SSL/TLS sockets programs: an older set that start with SSL_ and the more current GSKit set. The GSKit APIs are recommended for new applications.

Useful socket declarations and prototypes
The following example (Example 5-5) contains a number of coding examples where various socket declarations and prototypes are defined.

*
* Constant values
*
d AF_INET c 2
d AF_INET6 c 24
d SOCK_STREAM c 1
d SO_KEEPALIVE c 25
d SOL_SOCKET c -1
d TCP_NODELAY c 10
d IPPROTO_TCP c 6
d SOCKOPT_SIZE c 4

*
* Error values
*
d EADDRNOTAVAIL c 3421
d EIOERROR c 3101

*
* Structures
*
d sockaddr_in ds qualified template
d sin_family 5u 0
d sin_port 5u 0
d sin_addr 10u 0
d sin_zero 8
d hostent ds qualified template
d h_name *
d h_aliases *
d h_addrtype 10i 0
d h_length 10i 0
d h_addr_list *
d h_addr_list ds qualified template
d address *
d timeval ds qualified template
d tv_sec 10i 0
d tv_usec 10i 0
d fdset ds qualified template
d fdes 10u 0 dim(7)
*
* Prototypes
*

d socket pr 10i 0 extproc('socket')
d addr_family 10i 0 value
d type 10i 0 value
d protocol 10i 0 value
d inet_addr pr 10u 0 extproc('inet_addr')
d addr_string * value
d setsockopt pr 10i 0 extproc('setsockopt')
d sock_desc 10i 0 value
d level 10i 0 value
d option_name 10i 0 value
d option_value 10i 0 const
d option_length 10i 0 value
d gethostbyname pr * extproc('gethostbyname')
d host_name * value
d connect pr 10i 0 extproc('connect')
d sock_desc 10i 0 value
d dest_addr * value
d addr_len 10i 0 value
d select pr 10i 0 extproc('select')
d max_desc 10i 0 value
d read_set likeds(fdset) options(*omit)
d write_set likeds(fdset) options(*omit)
d except_set likeds(fdset) options(*omit)
d wait_time likeds(timeval) options(*omit)
d send pr 10i 0 extproc('send')
d sock_desc 10i 0 value
d buffer * value
d buffer_length 10i 0 value
d flags 10i 0 value
d recv pr 10i 0 extproc('recv')
d sock_desc 10i 0 value
d buffer * value
d buffer_length 10i 0 value
d flags 10i 0 value
d sclose pr 10i 0 extproc('close')
d sock_desc 10i 0 value
d geterrno pr * extproc('__errno')
{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}
>