Auto module initialization with OmniRpcModuleInit API

By using OmniRpcModuleInit API, when modules of a remote executable program are activated, the call initialize function is enabled automatically. So, you can write programs efficiently for master/worker programs which require worker initialization.

What's auto module initialization.
Programming with OmniRpcModuleInit API

What's auto module initialization

Auto module initialization is a function which calls the initialization method automatically when modules of a remote executable program are invoked.

By using OmniRpcHandle API, you can write efficient programs which keep data on a remote executable program, but you should write the program which is used by the remote executable program. We only specify the remote functions which are specified by OmniRpcCall at the start, the OmniRPC system executes and runs appropriate remote executable programs for requests. But we cannot know which remote executable programs are allocated by the OmniRPC system, so it is impossible to set the data beforehand.

In some OmniRPC master/worker programs, the master and worker share common data. Workers calculate its data with different parameters which are taken from the master. Auto module initialization is convenient in cases like this. In OmniRPC, if there is a function named "Initialize" in a module, "Initialize" is called automatically when a new remote executable program is activated for an other function's RPC call. Common data are set in initialization, and it can be efficient to reuse this data when function calls are called by real varied parameters. The bigger the costs of setup, the bigger are the effects.

Programming with OmniRpcModuleInit API

For example, we will use a program to calculate the appearance of 10 sorts of strings (at a maximum of 10 characters) from a string which is 10000 characters in size. The sequential version may be written as follows.

#include <stdio.h>
#include <string.h>

char data[10000];  /* data to be searched */
char str[10][10];  /* string to be compared */
int occurrence[10];  /* array to record occurrence */

/* prototype */
void count_occurrence(char *data,char *str, int *r);

int main(int argc, char *argv[])
{
    FILE *fp;
    int i;

    if((fp = fopen("data","r")) == NULL){
	fprintf(stderr,"cannot open data file\n");
	exit(1);
    }
    fread(data,10000,1,fp);
    fclose(fp);
    
    if((fp = fopen("strings","r")) == NULL){
	fprintf(stderr,"cannot open strings file\n");
	exit(1);
    }
    for(i = 0; i < 10; i++)
	fscanf(fp,"%s",str[i]);
    fclose(fp);

    for(i = 0; i < 10; i++)
	count_occurrence(data,str[i],&occurrence[i]);
    
    for(i = 0; i < 10; i++)
	printf("string(%i,'%s') occurrence=%d\n",i,
	       str[i],occurrence[i]);
    
    exit(0);
}

void count_occurrence(char *data,char *str, int *r)
{
    int i,len,count;
    len = strlen(str);
    count = 0;
    for(i = 0; i < 10000-len; i++){
	if(strncmp(&data[i],str,len) == 0) count++;
    }
    *r = count;
}

To parallize this program with OmniRpcCallAsync, we calculate count_occurrence in the remote executable program. The IDL file may be like the example shown below.

Module count_occurrence;

Define count_occurrence(char IN data[10000],char IN str[10], int OUT
r[]) Calls "C" count_occurrence(data,str,r);

Link this with the count_occurrence function. and register it. In the client program, we change a call of count_occurrence to a call of OmniRpcCallAsync API.

   
int main(int argc, char *argv[])
{
    FILE *fp;
    int i;
    OmniRpcReqeust reqs[10];

    OmniRpcInit(&argc,&argv);

    ... /* input data */

    for(i = 0; i < 10; i++)
    	reqs[i] = OmniRpcCallAsync("count_occurrence",
	                      data,str[i],&occurrence[i]);
   OmniRpcWaitAll(10,reqs);
			          
    ... 
   OmniRpcFinalize();
   exit(0);
}

In this case, there is the problem that data are sent for each RPC call on the remote executable module. This data is unchanged, so it is efficient to reuse the data sent on the remote executable program side.

By using OmniRpcModuleInit API, the client program sends the data in the initialization of the remote executable programs, and sends only the strings which are searched with OmniRpcCall. We define the IDL file as follows.

Module count_occurrence;

Globals {
#include <string.h>
char data[10000];
}

Define Initialize(char IN input_data[10000])
{
     memcpy(data,input_data,10000);
}

Define count_occurrence_each(char IN str[10], int OUT r[]){
    count_occurrence(data,str,r);
}

In the client program, initialization is described.

int main(int argc, char *argv[])
{
    FILE *fp;
    int i;
    OmniRpcReqeust reqs[10];

    OmniRpcInit(&argc,&argv);

    ... /* input data */

    OmniRpcModuleInit("count_occurrence",data);

    for(i = 0; i < 10; i++)
    	reqs[i] = OmniRpcCallAsync("count_occurrence_each",
	                      str[i],&occurrence[i]);
   OmniRpcWaitAll(10,reqs);
    ... 
   OmniRpcFinalize();
   exit(0);
}

We specify the necessary data in the initialization with OmniRpcModuleInit. The initializations are indeed taken when the necessary remote executable modules are activated. So, it is important not to write in the area addressed by the pointer variables in initialization.