- Ninf-G: a GridRPC system on the Globus toolkit. - 25.1 INTRODUCTION. - Currently, the Globus Toolkit [1] serves as the ‘de facto standard’ lower-level substrate for the Grid. - However, the Globus Toolkit alone is insufficient for programming on the Grid. - The abstraction level of the Globus, being a lower-level substrate, is terse and primitive in a sense. - this is fine for the intended use of the Toolkit, but nevertheless higher-level programming layers would be absolutely necessary for most users. - one would certainly program with higher-level parallel programming systems such as MPI and OpenMP, rather than by using the raw TCP/IP socket interface of the operating system.. - Ninf and NetSolve are implementations of the GridRPC [6] programming model, pro- viding simple yet powerful server-client-based framework for programming on the Grid.. - For the remainder of this chapter, we discuss the implementation of the Ninf-G system, and demonstrate the usage with a simple parallel programming example with the Ninf-G, along with its performance. - In Section 25.2 we briefly introduce the Globus Toolkit. - in Section 25.3 we discuss the overall design of GridRPC, and how each feature can be mapped on to lower-level Globus Toolkit features. - Section 25.4 outlines the implementa- tion of the Ninf-G, while Sections 25.5 and 25.6 illustrate a typical usage scenario along with its performance evaluation. - Section 25.7 will conclude and hint at future directions.. - 25.2 GLOBUS TOOLKIT. - The Globus Toolkit is a collection of modules that provides standardized lower-level features for implementing a distributed system on the Grid. - Table 25.1 covers the services provided by Globus, in which each service can be used independently when needed. - We give brief descriptions of the most relevant modules for GridRPC.. - Table 25.1 Globus services. - 25.3 DESIGN OF NINF-G. - 25.3.1 GridRPC system. - Application programmers write task-parallel client programs using simple and intuitive GridRPC APIs that hide most of the complexities involving Grid programming. - Client program basically invokes Remote Executables to request computation to be done on the server. - GridRPC system generally consists of the following four components:. - Each component consists of the user’s main program and the GridRPC library.. - Information service: Information service provides various information for the client com- ponent to invoke and to communicate with the Remote Executable component.. - 25.3.2 Ninf-G implementation on the Globus toolkit. - More specifically, Ninf-G employs the following components from the Globus Toolkit as shown in Figure 25.1.. - GRAM : Serves the role of the server in the old Ninf system.. - GASS : Redirects stdout and stderr of the GridRPC component to the client console.. - 25.3.3 API of Ninf-G. - Interface information. - Figure 25.1 Overview of Ninf-G.. - Table 25.2 GridRPC API principal functions. - Wait for completion of the session specified by the sessionID.. - Wait for completion of one of the RPCs invoked by grpc − call − async beforehand.. - serves as one of the reference implementations of the GridRPC API. - 25.3.4 Server side IDL. - In order to ‘gridify’ a library, the Ninf library provider describes the interface of the library function using the Ninf IDL to publish his library function, which is only manifested and handled at the server side. - Figure 25.2 An example of Ninf IDL file.. - As an example, interface description for the matrix multiply is shown in Figure 25.2, in which the access specifiers IN and OUT specify whether the argument is read or written within the Gridified library. - In this example, the value of N is referenced to calculate the size of the array arguments A, B, C. - In addition to the interface definition of the library function, the IDL description contains the information needed to compile and link the necessary libraries. - 25.4 NINF-G IMPLEMENTATION. - We now describe the implementation of the Ninf-G in more detail.. - 25.4.1 ‘Gridifying’ a library or an application using GridRPC. - ‘Gridified’ in the following manner on the server side (we note again that no IDL handling is necessary on the client side, as opposed to traditional RPC systems such as CORBA):. - For this purpose, Ninf-G adds the lines as shown in Figure 25.3 into the ‘grid-info-resource- ldif.conf’ file.. - 25.4.1.1 Generating the LDIF file. - Both embody interface information (in XML), pathname of the remote executable, and the signature of the remote library.. - Figure 25.4 shows a sample LDIF file generated from an IDL file in Figure 25.2. - 25.4.1.2 Registration into the MDS. - Figure 25.3 Addition to the grid-info-resource-ldif.conf.. - dn: GridRPC–Funcname=sample/mmul, Mds–Software–deployment=GridRPC–Ninf–G. - Mds-Software-deployment: GridRPC–Ninf–G GridRPC–Funcname: sample/mmul. - Figure 25.4 LDIF file for matrix multiply.. - 25.4.2 Performing GridRPC. - Now we are ready to make the actual Ninf-G GridRPC call that can be broken down into the steps as shown in Figure 25.1.. - The client requests interface information and executable pathname from the MDS.. - The remote executable connects back to the client using Globus-I/O for subsequent parameter transfer, and so on.. - 25.4.2.1 Retrieval of interface information and executable pathname. - The client retrieves the interface information and executable pathname registered within the MDS using the library signature as a key. - The retrieved information is cached in the client program to reduce the MDS retrieval overhead.. - 25.4.2.2 Invoking the remote executable. - The client invokes the remote executable (done by Ninf-G via the Globus GRAM), spec- ifying the remote executable path obtained from the MDS and a port address that accepts the callback from the remote executable. - this is because the party that owns the proper Globus proxy certificates derived from the client user certificate can connect to the port.. - 25.4.2.3 Remote executable callbacks to the client. - The remote executable obtains the client address and the port from argument list and con- nects back to the client using Globus-I/O. - Subsequent remote executable communication with the client will use this port.. - 25.5 USAGE SCENARIO. - As a sample application, consider computing an approximation of the value of pi using a simple Monte Carlo method. - 1 The Monte Carlo method generates a huge amount of random numbers, converts them into meaningful input parameters, performs some calculations using the parameters, and statistically processes the results of the calculation to arrive at a meaningful result. - For this example, we assume an environment shown in Figure 25.5. - The Client Program looks up the GIIS server as an MDS server, and performs GridRPC onto the GRAM server on each host.. - 25.5.1 Setup remote executable. - To setup the servers, the Monte Carlo calculation is defined as a standard C function, and an IDL description of the function is also defined. - Figures 25.6 and 25.7 show the function and its IDL description, respectively. - 25.5.2 Client program. - The client program performs GridRPCs onto the servers in parallel using the asynchronous invocation API. - The core fragment of the client program is shown in Figure 25.8, demon- strating the relative ease of parallel application construction in Ninf-G.. - Figure 25.5 Usage scenario environment.. - 1 This is one of the simplest Monte Carlo applications. - Figure 25.6 Monte Carlo PI trials.. - Figure 25.7 IDL for the PI trial function.. - Synchronize on the result return. - Figure 25.8 Client PI program.. - 25.6 PRELIMINARY EVALUATION. - 25.6.1 Experimental setup. - Although most of the network paths between two sites go through fast National backbones, the state of the network upgrading was in flux and there still remained some 10 Mbps bottlenecks at the time of the measurement.. - Figure 25.9 shows the setup and FTP throughput and ping latency between the clients and the server. - 25.6.2 Results and discussion. - Table 25.3 shows the MDS query cost and the GRAM invocation cost. - Figure 25.9 Experiment setup.. - Table 25.3 MDS lookup cost and GRAM invocation cost. - Figure 25.10 Ninf-G throughput.. - Here, we observe that both MDS and GRAM costs are independent of the network latency between the client and the server. - Figure 25.10 shows the data throughput difference between the clients and the server in both the LAN and WAN environments. - 25.7 CONCLUSION. - Thanks to the Globus Toolkit, Ninf-G now provides a secure and interoperable GridRPC framework, compared to its predecessor. - Modular design of the Globus Toolkit. - allowed us to conduct step-by-step deployment of the components, facilitating productive development, verifying the claim made by the Globus development team.
Xem thử không khả dụng, vui lòng xem tại trang nguồn hoặc xem
Tóm tắt