- By ‘standard’ and ‘Grids’, we specifically mean Grids based on the common practice and standards coming out of the Global Grid Forum (GGF) (www.gridforum.org)..
- That is, these suites of software have provided the implementation of the Grid functions used in the IPG and DOE Science Grids..
- A general design approach that allows a decentralized control and deployment of the software,.
- Several functions were added to PBS over the course of the IPG project in order to support Grids..
- the Grid monitoring and event framework of the Grid Monitoring Architecture Working Group (WG) [20.
- Nevertheless, the software of the prototype production Grids described in this chapter is provided primarily by the aforementioned packages, and these provide the context of this discussion..
- This chapter recounts some of the lessons learned in the process of deploying these Grids and provides an outline of the steps that have proven useful/necessary in order to deploy these types of Grids.
- In the opinion of the author, there is a set of basic functions that all Grids must have in order to be called a Grid: The Grid Common Services.
- These constitute the ‘neck of the hourglass’ of Grids, and include the Grid Information Service (‘GIS.
- The Grid Forum’s Grid Monitor Architecture (GMA) [29] addresses one approach to Grid events, and there are several prototype implementations of the GMA (e.g.
- This sort of Grid also facilitates/encourages the incorporation of the supercomputers into user constructed systems..
- however, their execution is largely independent of the other jobs in the collection..
- 1 A proxy certificate is the indirect representation of the user that is derived from the Grid identity credential.
- The manager must stay alive while the jobs are running on the remote Grid resource in order to keep track of the jobs as they complete.
- Events can also be generated by the Grid remote job management system signaling various sorts of things that might happen in the control scripts of the Grid jobs, and so on..
- Co-scheduling is essential for this to be a generally useful capability since different ‘parts’ of the same program are running on different systems..
- knowledge, these implementations are not Grid services because they do not make use of the Common Grid Services.
- In particular, the MIPCH-G2 use of the Globus I/O library that, for example, automatically provides access to the Grid Security Services (GSS), since the I/O library incorporates GSI below the I/O interface..
- That is, it could drive a distributed simulation in which some of the computational resources are under the control of the user – for example, a local cluster – and some (the Glide- in) are scheduled by a batch queuing system.
- SRB/MCAT provides capabilities that include uniform remote access to data and local caching of the data for fast and/or multiple accesses.
- SRB provides a uniform interface by placing a server in front of (or as part of) the tertiary storage system.
- Access control in SRB is treated as an attribute of the dataset, and the equivalent of a Globus mapfile is stored in the dataset metadata in MCAT.
- GridFTP provides many of the same basic data access capabilities as SRB, however, for a single data source.
- However, much of the emphasis in GridFTP has been WAN performance and the ability to manage huge files in the wide area for the reasons given in the next section.
- An effective technique for improving access speeds and reducing network loads can be to replicate frequently accessed datasets at locations chosen to be ‘near’ the eventual users.
- (Most of the aforementioned projects already maintain their own style of metadata catalogue.) The European Union DataGrid project provides a similar service for replica management that uses a different set of catalogue and replica management tools (GDMP [59.
- A common situation is that a whole set of simulations or data analysis programs will require the use of the same large reference dataset.
- However, another service that is needed in this situation is a network cache: a unit of storage that can be accessed and allocated as a Grid resource, and that is located ‘close to’ (in the network sense) the Grid computational.
- The Grid Resource Information Service (GRIS) runs on the Grid resources (computing and data systems) and handles the soft-state registration of the resource characteristics.
- One of the most important contributions of Grids to supporting large-scale collabo- ration is the uniform Grid entity naming and authentication mechanisms provided by the GSI..
- In the PKI authentication environment assumed here, the CA policies are encoded as formal documents associated with the operation of the CA that issues your Grid iden- tity credentials.
- The nature of the policy associated with identity certificates depends a great deal on the nature of your Grid community and/or the VO associated with your Grid.
- Hopefully the sites of interest already have people who are (1) familiar with the PKI CP process and (2) focused on the scientific community of the institution rather than on the administrative community.
- The GGF is working on a standard set of CPs that can be used as templates, and the DOE Science Grid has developed a CP that supports international collaborations, and that is contributing to the evolution of the GGF CP.
- One of the important issues in developing a CP is the naming of the principals (the.
- 3 Much of the work described in this section is that of Tony Genovese (
[email protected]) and Mike Helm (
[email protected]), ESnet, Lawrence Berkeley National Laboratory..
- This is the model of the DOE Science Grid CA, for example, and it is intended to provide a CA that is scalable to dozens of VO and thousands of users.
- The architecture of the DOE Science Grid CA is indicated in Figure 5.2 and it has the following key features..
- The Root CA (which is kept locked up and off-line) signs the certificates of the CA that issues user certificates.
- With the exception of the ‘community’ Registration Man- ager (RMs), all RMs are operated by the VOs that they represent.
- (The community RM addresses those ‘miscellaneous’ people who legitimately need DOE Grid certificates, but for some reason are not associated with a Virtual Organization.) The process of issuing a certificate to a user (‘subscriber’) is indicated in Figure 5.3..
- critical components of the CA.
- Using certificates issued by your CA, validate correct operation of the GSI [72], GSS libraries, GSISSH [62], and GSIFTP [73] and/or GridFTP [28] at all sites..
- Therefore, managing the contents of the mapfile is the basic Globus user authorization mechanism for the local resource..
- It is also being incorporated into some of the Globus tools.
- Some of the most important of these are the functions associated with job initiation and management on the remote computing resources.
- Development of the PBS batch scheduling system was an active part of the IPG project, and several important features were added in order to support Grids..
- It actually creates a queue that ‘owns’ the reservation.
- The Globus deployment team should be familiar with the install and operation issues and the system admins of the target resources should be engaged..
- Grids present special challenges for system administration owing to the administratively heterogeneous nature of the underlying resources..
- 5.7.12 Take good care of the users as early as possible.
- One of the scaling/impediment-to-use issues currently is that extant Grid functions are relatively primitive (i.e.
- This hides some of the ‘low-level functionality’.
- The MyProxy service provides for creating and storing intermediate lifetime proxies that may be accessed by, for example, Web-based portals, job schedulers, and so forth, on behalf of the user.
- The impact of the emerging Web Grid services work is not yet clear.
- however, it is the opinion of the author that this will in no way obviate the need for the Grid common services.
- Credit is also due to the intellectual leaders of the major software projects that formed the basis of IPG and the DOE Science Grid.
- The SRB/MCAT team is led by Reagan Moore of the San Diego Supercomputer Center.
- Bill is currently head of the Computing Division at Los Alamos National Laboratory..
- One reviewer, in particular, made extensive and very useful comments, and that review is the basis of the Abstract..
- The GridLab project is currently running, and is being funded under the Fifth Call of the Information Society Technology (IST) Program..
- NERSC is one of the largest unclassified scientific supercomputer centers in the US.
- This white paper will explore PKI technology of the ESnet community.
- This paper will provide a project overview of the immediate requirements for the DOE Science Grid PKI support and cover the long-term project goals described in the ESnet PKI and Directory project document..
- Legion sits on top of the user’s operating system, acting as liaison between its own host(s) and whatever other resources are required.
- Jobs can be submitted to any of the platforms of a UNICORE GRID and the user can monitor and control the submitted jobs through the job monitor part of the client..
- The Grid Monitoring Architecture working group is focused on producing a high-level archi- tecture statement of the components and interfaces needed to promote interoperability between heterogeneous monitoring systems on the Grid.
- The Anatomy of the Grid: Enabling Scalable Virtual Organizations, Foster, I., Kesselman, C..
- Virtual Observatories of the Future, Caltech, http://www.astro.caltech.edu/nvoconf/..
- The primary elements of the GSI are identity certificates, mutual authentication, confidential communication, delegation, and single sign-on..
- The Globus Toolkit’s implementation of the GSI adheres to the Generic Security Service API (GSS-API), which is a standard API for security systems promoted by the Internet Engineering Task Force (IETF)..
- The request is sent to the gatekeeper of the remote computer.
- The executable, stdin and stdout, as well as the name and port of the remote computer, are specified as part of the job request.
- The job manager handles the execution of the job, as well as any communication with the user..
- The current GMA specification from the GGF Performance Working Group may be found in the documents section of the Working Group Web page..
- Many of the components of the DMF have already been prototyped or implemented by the DIDC Group.
- The project has six main partners: CERN – The European Organization for Nuclear Research near Geneva, Swiss.
- ESRIN – the European Space Agency’s Centre in Frascati (near Rome), Italy.
- NIKHEF – The Dutch National Institute for Nuclear Physics and High Energy – Physics, Amsterdam, and.
- JiPANG performs uniform higher-level management of the computing services and resources being managed by individual Grid systems such as Ninf, NetSolve, Globus, and so on.
- The middleware components include Condor-G, DAGMAN, GDMP, and the Globus Toolkit packaged together in the first release of the Virtual Data Toolkit..
- MPICH-G2 is a Grid-enabled implementation of the MPI v1.1 standard.
- The metacomputing effort used for the simulations linked 3 of the top 10 largest supercomputers in the world.
- Overview of the Grid Security Infrastructure (GSI), Globus Project, 2002, http://www-fp.globus.org/security/overview.html..
- Many of the terms and concepts used in this description of the GSI come from its use of public key cryptography.
- Instead, they will store copies of the most relevant portions of the data set on local storage for faster access.
- Replica Management is the process of keeping track of where portions of the data set can be found..
- Instead, they will store copies of the most.
- relevant portions of the data set on local storage for faster access.
- The CPS is a statement of the practices, which a certification authority employs in issuing certificates..
- ESnet is funded by the DOE Office of Science to provide network and collaboration services in support of the agency’s research missions..
- Each resource provider verifies that the holder of the community credential represents that.
- Low overhead is an important requirement for such tools, thereby we evaluate efficiency of the monitoring itself.
- Using NetLogger for Distributed Systems Performance Analysis of the BaBar Data Analysis System, Tierney, B., Gunter, D., Becla, J., Jacobsen, B.
- The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Inte- gration, Foster, I., Kesselman, C., Nick, J.
- we focus here on the nature of the services that respond to protocol messages.
- This project involves, first, development of the basic techniques required to achieve coexis-.
- HotPage enables researchers to find information about each of the resources in the NPACI computational Grid, including technical documentation, operational status, load and current usage, and queued jobs..
- New tools allow you to – obtain a portal account on-line – personalize your view of the status bar.
- The architecture and security of the MyProxy system are described in detail.