Communicating with another application

Simile version 6.11 introduces the ability for a submodel to incorporate communication with a separate application during model execution. The purpose of this feature is to enable systems to be modelled where individual components in different modelling environments must run in combination, and each system has its own means of managing execution. For instance, one might use Simile to create a farm-scale or larger model of an agricultural system, but want to use existing models in another tool such as APSIM for the various crops and soils in the system

 

It is possible to implement such communication using externally-supplied procedures as described in the previous section, but supplying a procedure that carries out communication with another process rather than implementing the submodel itself. However, doing communication this way would require the procedure to be manually re-written each time a change was made to the format of data being passed back and forth while developing the combined model. Also, in the case of a multi-instance submodel, the external procedure is called once for each instance in sequence, meaning it is hard to avoid problems if the remote process is running the instances in some other order.

Implementation of the connection

For these reasons, we have built a remote connection interface specification into Simile itself, and this can be selected as a third option, after normal execution or use of external procedure, in the Submodel Properties dialogue. Communication is via named pipe in Windows, or Unix socket (which is the closest equivalent to a Windows named pipe) in MacOS or Linux. Note that the commands in c# to operate on named pipes work unchanged on Unix sockets in these systems, allowing remote models implemented in c# to be cross-platform. Selecting remote connection allows the modeller to enter the name of the connection, which can be anything in Windows but must be an available filesystem location in MacOS or Linux.

The dialogue for specifying the remote interaction has two other fields:

• Command to start. This is a system command that is called to start the remote model each time the Simile model is initialized or reset. This can be used if it is possible to run the remote model from the command line, in order to save the modeller from having to start the remote model via its own interface at this point. Default is 'none', meaning no command is issued.
• Time unit. At the start of the simulation, Simile works out the offset between the times as represented on each system, but if the unit is something other than 'day' on the remote system it must be entered here. e.g. if you enter 'year' while the Simile time unit is default or 'day', then the Simile model will advance 365 units for each time unit in the remote model. 

When the model is initialized, Simile opens a connection as the server at the given location. The remote system can then connect as a client. If the submodel is multiple-instance, Simile will accept one client connection for each instance. These can all be made to the same server location. The two systems then exchange data over the pipe as the models run, as described in the next section. It is up to the remote system to close the pipes at the end of its run, which signals Simile also to pause. Resetting the Simile model will cause it to issue the start command again and accept a new connection for each submodel instance. Exiting the Simile model (e.g., before initializing an updated version) will delete the pipe or socket. Here is a description of the server operation in pseudocode:

create server pipe
while {model initialized or reset} {
  issue start command if set
   foreach {submodel instance} {
      accept connection
      foreach {submodel dimension} {
         read integer index from connection
      }
      assign connection to submodel instance
      read real remote time from connection
      time offset = remote time - simile time
   }

   foreach {time step} {
      evaluate inputs from rest of model
      foreach {submodel instance} {
         if {simile time + time offset >= remote time} {
            write real (simile time + time offset + time step) to connection
            for (n=1; n<=inputs; ++n) {
               write input n to connection as appropriate type
            }   
            flush connection
         }
      }
      foreach {submodel instance} {
         if {simile time + time offset >= remote time} {
            if (remote system has closed connection) {
               signal model execution pause
               delete submodel reference to connection
            } else {
               read real remote time from connection
               for (n=1; n<=outputs; ++n) {
                         read output n from connection as appropriate type
               }
            }
         }
      }
      evaluate rest of model from outputs
   }
   delete any remaining connections
}
delete server pipe

Data transfer protocol

The values to send and receive via the pipe are specified in the same manner as the values to pass to an external procedure as described in the last section, except the numerical part of their caption determines the order in which they are sent to or received from the pipe.

Data passes along the pipe in raw binary format, one byte for a boolean, four for an integer or eight (double precision) for a floating-point value. These sizes correspond to what is sent automatically in c# by the BinaryWriter Write() method, according to the datatype of its argument, and what is received in c# by the BinaryReader ReadBoolean(), ReadInteger() and ReadDouble() methods respectively. If a data value, or an index for the communication submodel instances, is a member of an enumerated type then that value is passed as a string in a format compatible with the c# ReadString() method, i.e., a byte giving the string length followed by that number of bytes for its characters.

When a remote system connects as a client implementing one instance of a multi-instance submodel, it must first identify which submodel instance that connection is for. To do this it sends one integer for each dimension of the submodel, giving its index. For a single-instance submodel, no indices are needed.

Next the remote system must send its notion of the time at initialization, as a floating-point value. This is taken by Simile to be equivalent to the "Time at reset" specified in the run control. Time units are assumed to be the same for the systems. This is also the time at which the first interaction is to take place.

Multiple instances

There are two ways of having multiple instances of an external process communicate with Simile as multiple instances of a submodel. Firstly, the parent model containing the externally implemented submodel might already be multiple-instance. In this case, the start command will be issued once for each instance of the parent model, after which Simile will wait for a single client connection, which will be used for the submodel in that parent instance. In this case the remote process does not need to send any indices when connecting. The start command can include arguments that are specified by model components in the parent submodel whose captions are of the form 'paramn' where n is an integer, in much the same way as inputs to external code are specified. Typically, one of these components would be set to the index of the parent model instance to let the remote process know which submodel instance to start. 

Alternatively, the externally implemented submodel itself can be set multi-instance. In this case the start command will be issued only once, and it is up to the remote process to make a client connection to the pipe for each submodel instance. When a connection is made, a value or set of values must be sent by the remote process providing the index or indices of the submodel instance corresponding to that connection. The connections for each instance within a single parent instance can be made in any order. It is possible to combine these methods, and have multiple parent model instances each with multiple submodel instances implemented by separate pipe client connections, but such complexity is unlikely to be needed.

Interactions during model execution

When Simile executes, if the model time is equal or greater than the time at which the next interaction is expected (which will be true at reset, see above) it carries out an interaction. Components in the submodel with captions starting with input1, input2 etc will have their values sent to the remote system, and those with captions starting output1, output2 etc will be read from the remote system. The naming may seem back to front, but it corresponds to the system used for submodels implemented as a hand-coded c++ procedure, where input1, input2 etc correspond to inputs to the procedure.

Simile starts by sending values. First, a floating-point value is sent, representing the earliest time at which the next interaction could occur (typically the current time plus one time step). This has been adjusted to fit the remote system's notion of time as determined from the initial exchange. Then each input value (to remote system) is sent, in the order of their numbers, the datatype according to their units in Simile. If any of the inputs are arrays, all their values are sent as a block. All these, including the time, are sent to each client instance from its Simile submodel instance. After they are all sent, Simile starts receiving values. First it reads a floating-point value, representing the earliest time (in the remore system's notion of time) at which the next interaction can occur. Then it reads a value of the appropriate type for each output value (from remote system) in order. Again if an output value is an array, a block of data is read for all the values. All the reads are then repeated for each further submodel instance if there are multiple instances.

Error codes

The interface management uses numerical codes as in the user-defined stop function to signal when something has gone wrong. These cause messages to appear as popup boxes or in the log tab of the run control pane, according to the preference settings. They will typically contain text like "There was a user-defined exit: 71". Here is an explanation of the more common codes:

Error code	Meaning	Remedy
70	Server could not be created	Check pipes exist on your system
71	Server could not be added to filesystem	Delete anything already present in that location
72	Server cannot listen for clients	Reduce number of simultaneous connections
73	Client cannot connect	Ensure client connection type is compatible
76	Failed to write time to pipe	Remote model should always read a set of data after writing one
77	Failed to read time from pipe	No problem, remote model has exited normally
78	Failed to read submodel indices from pipe	Make sure remote model is sending an index (integer or string) for each submodel dimension

How to implement a remote system

The interface allows two approaches to designing the interaction:

1. Serial execution. Simile runs until it sends the input values from the connection submodel, then it waits while the remote system processes those values and returns its outputs to the connection submodel, then Simile completes the time step while the remote system waits for the next set of values.
2. Both systems run at once. Whichever arrives first at the time specified by the other for the next interaction will send values to the pipe (this does not block execution; the values are buffered by the OS) then wait until the other does the same, after which both can read values and continue in parallel to the next interaction point.

There is a simple trade-off here; in the serial case, a value in Simile can be passed to the remote system, affect an output from the remote system and have that affect another value in Simile all in one time step, and this can happen every time step if the step is the same in each system. In the parallel case, overall execution may be faster on multi-processor systems, but the effect of a change to a remote model's input will not be seen in its outputs until after two interactions have taken place.