Sun Grid Engine
GridEngine Logo | |
---|---|
Developer(s) | Sun Microsystems in association with the community |
Stable release | 6.2u5 / December 22, 2009[1] |
Operating system | Cross-platform |
Type | Grid computing |
License | SISSL |
Website | http://gridengine.sunsource.net |
Sun Grid Engine (SGE), previously known as CODINE (COmputing in DIstributed Networked Environments) or GRD (Global Resource Director),[2] is an open source batch-queuing system, developed and supported by Sun Microsystems. Sun also sells a commercial product based on SGE, also known as N1 Grid Engine (N1GE).
SGE is typically used on a computer farm or high-performance computing (HPC) cluster and is responsible for accepting, scheduling, dispatching, and managing the remote and distributed execution of large numbers of standalone, parallel or interactive user jobs. It also manages and schedules the allocation of distributed resources such as processors, memory, disk space, and software licenses.
SGE is the foundation of the Sun Grid utility computing system, made available over the Internet in the United States in 2006,[3] later becoming available in many other countries.
Features
Features new in version 6.2
- Advance reservation
- Array job interdependencies
- Enhanced remote execution (without using external rshd/rlogind/sshd processes)
- Multi-clustering [4]
- Daemons managed by the Service Management Facility on Solaris
- Pseudo TTY (pty) support for interactive jobs
- Job Submission Verifier (client-side and server-side job verification)
- GUI Installer and SGE Inspect
- Topology-aware scheduling and thread binding
- Hadoop integration, Amazon EC2 integration for cloud computing
Other features of SGE include:
- Multiple advanced scheduling algorithms allow powerful policy-based resource allocation
- Cluster queues
- Job and scheduler fault tolerance - Grid Engine continues to operate as long as there is one or more hosts available
- Job checkpointing
- Job arrays and job tasks
- DRMAA (Job API)
- Resource reservation
- XML status reporting (qstat and qhost), and the xml-qstat web interface
- Parallel jobs (MPI, PVM, OpenMP), and scalable parallel job startup with qrsh [5]
- Usage accounting
- Accounting and Reporting COnsole (ARCO)
- parallel make: distmake, dmake (Sun Studio), and SGE's own qmake
- FLEXlm integration [6] and multi-cluster software license management with LicenseJuggler [7]
Platforms
SGE runs on multiple platforms, including:
- AIX
- BSD - FreeBSD, NetBSD, OpenBSD
- HP-UX
- IRIX
- Linux
- Mac OS X
- Solaris
- SUPER-UX
- Tru64
- Windows via SFU(Interix) or SUA(Microsoft Windows Services for UNIX) (as execution hosts only)
- Z/OS (in progress)
Cluster architecture
A typical Grid Engine cluster consists of a master host, and one or more execution hosts. Moreover, multiple shadow masters can be configured as hot spares, which take over the role of the master when the original master host crashes.
Support and training
Sun provides support contracts [8] for the commercial version of Grid Engine on most UNIX platforms and Windows. Professional services, consulting, training, and support are also provided by Sun Partners. [9] Sun partners with Georgetown University to deliver Grid Engine administration classes.[10] The Bioteam runs short SGE training workshops that are 1 or 2 days long.[11]
Users can obtain community support on the Grid Engine mailing lists.[12]
Grid Engine Workshops were held in 2002, 2003, 2007, and 2009 in Regensburg, Germany.[13]
Prominent users
Notable deployments of SGE include:
- Sun Grid
- the TSUBAME supercomputer at the Tokyo Institute of Technology,[14] which was number 7 on June 2006 TOP500 list.
- Ranger at the Texas Advanced Computing Center (TACC). Ranger has 62,976 processor cores in 3,936 nodes and a peak performance of 504TFlops. [15] [16] Ranger is currently the 6th most powerful TOP500 supercomputer. [17]
- San Diego Supercomputer Center (SDSC)
- Geophysical Fluid Dynamics Laboratory (NOAA GFDL)
History
In 2000, Sun acquired Gridware, Inc. a privately owned commercial vendor of advanced computing resource management software with offices in San Jose, Calif., and Regensburg, Germany.[18] Later that year, Sun offered a free version of Gridware for Solaris and Linux, and renamed the product Sun Grid Engine.
In 2001, Sun made the source code available,[19] and adopted the open source development model. Ports for Mac OS X and *BSD were contributed by the non-Sun open source developers.
Other Grid Engine based products
- Sun Constellation System
- Sun Visualization System
- Sun Compute Cluster [20]
- ClusterVisionOS Distribution
- Rocks Cluster Distribution
- EGEE [21]
- Univa UD's UniCluster Express
- BioTeam's iNquiry
Add-on software
A number of SGE add-ons are available:
- Solaris Cluster integration [22]
- Service Domain Management module in order to meet service level objectives
- Transfer-queue Over Globus (TOG). Globus does not officially support SGE, but the London e-Science Center has created a package and provides instructions on how to configure a Globus Toolkit server so that it can submit jobs for execution on a local Sun Grid Engine installation.
- JOb Scheduling Hierarchically (JOSH)
References
- ↑ Announcing Sun Grid Engine 6.2 Update 5
- ↑ "A Little History Lesson". Sun Microsystems. 2006-06-23. http://blogs.sun.com/templedf/entry/a_little_history_lesson.
- ↑ "World's First Utility Grid Comes Alive on the Internet". Sun Microsystems. 2006-03-22. http://www.sun.com/smi/Press/sunflash/2006-03/sunflash.20060322.1.xml.
- ↑ "Hedeby Project home". Sun Microsystems. http://hedeby.sunsource.net. Retrieved 2008-01-25.
- ↑ "Long delay when submitting large jobs (mailing list message)". Sun Microsystems. http://gridengine.sunsource.net/servlets/ReadMsg?listName=users&msgNo=9446. Retrieved 2007-12-25.
- ↑ "Olesen-FLEXlm-Integration". wiki.gridengine.info. http://wiki.gridengine.info/wiki/index.php/Olesen-FLEXlm-Integration. Retrieved 2007-12-25.
- ↑ "LicenseJuggler". wiki.gridengine.info. http://wiki.gridengine.info/wiki/index.php/LicenseJuggler. Retrieved 2007-12-26.
- ↑ "Sun Store Grid Engine Entitlement Purchase". Sun Microsystems. http://store.sun.com/CMTemplate/CEServlet?process=SunStore&cmdViewProduct_CP&catid=115672. Retrieved 2008-03-03.
- ↑ "Sun Grid Engine 6 Partners". Sun Microsystems. http://www.sun.com/software/gridware/partners/index.xml. Retrieved 2007-12-14.
- ↑ "Advanced Sun Grid Engine Configuration and Administration Class". Sun Microsystems. http://blogs.sun.com/templedf/entry/advanced_sun_grid_engine_configuration. Retrieved 2007-12-14.
- ↑ "Training". The Bioteam Inc.. http://blog.bioteam.net/category/training/. Retrieved 2008-03-24.
- ↑ "Grid Engine Mail Lists". Sun Microsystems. http://gridengine.sunsource.net/maillist.html. Retrieved 2008-01-23.
- ↑ "Grid Engine Workshops". Sun Microsystems. http://gridengine.sunsource.net/workshop.html. Retrieved 2009-12-26.
- ↑ "Sun N1 Grid Engine Software and the Tokyo Institute of Technology Super Computer Grid". Sun Microsystems. http://www.sun.com/blueprints/0607/820-1695.html. Retrieved 2007-11-16.
- ↑ "TACC > HPC Systems". The University of Texas at Austin. http://www.tacc.utexas.edu/resources/hpcsystems/#ranger. Retrieved 2007-12-13.
- ↑ "More Ranger Facts and Figures". Sun Microsystems. http://blogs.sun.com/marchamilton/entry/more_ranger_facts_and_figures. Retrieved 2008-02-12.
- ↑ "TOP500 List - June 2008". TOP500.Org. 2006-06-18. http://top500.org/list/2008/06/100.
- ↑ "Gridware's resource management software increases efficiency and productivity in compute-intensive technical computing environments". Sun Microsystems. 2000-07-24. http://www.sun.com/smi/Press/sunflash/2000-07/sunflash.20000724.3.xml.
- ↑ "Sun Microsystems makes SUN GRID ENGINE software available to open source community". Sun Microsystems. 2001-07-23. http://www.sun.com/smi/Press/sunflash/2001-07/sunflash.20010723.1.xml.
- ↑ "Sun Compute Cluster Solution". Sun Microsystems. http://www.sun.com/servers/hpc/computecluster/index.jsp.
- ↑ "Sun Grid Engine, a new scheduler for EGEE middleware". Imperial College. 2000-12-29. http://pubs.doc.ic.ac.uk/egee-sge-integration/.
- ↑ "Installing and Configuring Sun Cluster HA for Sun Grid Engine". Sun Microsystems. 2008-02-15. http://docs.sun.com/app/docs/doc/819-3064/cacjgdbc?a=view.
See also
- Sun Ops Center - Sun's datacenter automation tool
- Open Grid Forum
External links
Search Wikimedia Commons | Wikimedia Commons has media related to: Sun Grid Engine |
- gridengine: Home - Official project page
- Sun Grid Engine - Official page for the commercial version
- Sun Grid Engine 6.0 Collection - Documentation Collection for Grid Engine 6.0
- Sun Grid Engine 6.1 Collection - Documentation Collection for Grid Engine 6.1
- Sun Grid Engine Information Center - Documentation Collection for Grid Engine 6.2
- gridengine.info - Unofficial Grid Engine resources blog
- wiki.gridengine.info - Unofficial Grid Engine resources wiki - including information about integration of applications
- "Understanding the differences between Grid Engine 5.3, 6.0 and Sun N1 Grid Engine 6 (N1GE 6)"
- xml-qstat project page - Web based status monitoring of Grid Engine 6.x systems
- Good Tips - several blog entries on getting started with Grid Engine
- Grid Engine-Globus Toolkit adapter
- BinBase Cluster System - an API to access the SGE from java and run java based jobs on the cluster.
- Schedule::SGE - An API to access the SGE using Perl
ar:Sun grid engin es:Sun Grid Engine it:Sun Grid Engine ja:Sun Grid Engine ru:Sun Grid Engine zh:Sun Grid Engine
If you like SEOmastering Site, you can support it by - BTC: bc1qppjcl3c2cyjazy6lepmrv3fh6ke9mxs7zpfky0 , TRC20 and more...