On the EDGE On the EDGE

«Back

XtremWeb-HEP 8.1.2 released

EDGI project partner CNRS from the Laboratoire de l'Accélérateur Linéaire in Orsay, France has released the XtremWeb-HEP 8.1.2 version.

Major points in this version include:

  • Correction: inter node communication thanks to Jason Maassen from the Netherlands eScience Center in Amsterdam
  • Correction: REST interface
  • New feature: the server embeds Web service available through HTTPS

A. Corrections:

  • A.1 on server side, the scheduling improved: hosts.incomingconnection is now taken into account to ensure a server like job is not sent to a volunteer resource where incoming connections are not allowed
  • A.2 on server side, the scheduling improved: restoring a feature from XtremWeb 1.8.0 by INRIA: "expected host" so that the end user can specify at submission time the expected volunteer resource to run the job. Note that if the expected host does not request a job or does not match the requirements (OS, CPU, ...), the job is never computed.
  • A.3 the DG to SG bridge is now compatible with shared applications so that one can use VirtualBox on EGI resources
  • A.4 some bugs corrected on client side regarding application registration and task management
  • A.5 a bug corrected on temporary file usage: they are now cleaned at software exit
  • A.6 the REST interface has been completely rewritten. The <xwhep></xwhep> root element MUST be used. Ex:

a/ to send a new application: https://server/sendapp/?XWPARAM=<xwhep version="8.0.2-flyinggrid"><app name="pouet" /></xwhep>

b/ to retrieved registered applications: https://server/getapps

  • A.7 a bug corrected in HTTP data download
  • A.8 SmartSocket usage corrected; inter node communications fully functional now

B. New features:

  • B.1 the creation of new Live CD can be customized
  • this is done by providing several optional files:
  • > a text file 'user.packages' may be provided to install custom packages: this file must contain a space separated packages list
  • > a text file 'user.hostname' may be provided to customize the LiveCD hostname: this file must contain the host name only
  • > any package files are installed on the fly (*.rpm or *.deb, depending on the LiveCD OS)
  • B.2 the client can now be used to create SmartSockets end point, independently of any task
  • this may be useful to create SmartSockets tunnel from a running VM to the client PC
  • > e.g. mount the client FS inside the running VM
  • B.3 a new Mac OS X package "xwhep.vworker" to deploy the middleware inside a VM
  • B.4 the server contains an embedded Web service interface available through HTTPS

C. Known Bugs:

  • C.1 xtremweb.gmond.pl does not scale
  • C.2 the scheduler is not fair
  • C.3 there may be some concurrent access problems leading to some errors

There may be two concurrent access problems

  • on server side, DB access may lead to inconsistency between works and tasks tables

This could certainly be solved using SQL transactions

But we would then need MySQL >=5

There are sime issues with transactions with hsqldb

  • on worker side, file access if a worker is configured to run jobs in parallel (default one thread per CPU core)

CNRS has observed that these concurrent access problems may lead to up to 6% erroneous jobs. Until further notification, CNRS considers this rate acceptable.

More information is available at http://www.xtremweb-hep.org/spip.php?rubrique15

EDGI EDGI

EDGI will develop middleware that consolidates the results achieved in the EDGeS project concerning the extension of Service Grids with Desktop Grids (DGs) in order to support European Grid Initiative (EGI) and National Grid Initiative user communities that are heavy users of Distributed Computing Infrastructures (DCIs) and require an extremely large number of CPUs and cores. EDGI will go beyond existing DCIs that are typically cluster Grids and supercomputer Grids, and will extend them with public and institutional Desktop Grids and Clouds. EDGI will integrate software components of ARC, gLite, Unicore, BOINC, XWHEP, 3G Bridge, and Cloud middleware such as OpenNebula and Eucalyptus into SG-DG-Cloud platforms for service provision and as a result EDGI will extend ARC, gLite and Unicore Grids with volunteer and institutional DG systems.

EDGI will develop DG-Cloud bridge middleware with the goal to get instantly available additional resources for DG systems if the application has some QoS requirements that could not be satisfied by the available resources of the DG system. EDGI will improve Desktop Grid middleware (BOINC and XtremWebHEP-E) in order to handle QoS requirements and the SG-DG bridge middleware in order to support data-intensive applications. EDGI will deploy a production infrastructure that integrates ARC-, gLite- and Unicore-based Grids with Desktop Grids based on the bridge middleware developed in EDGI.

The production EDGI infrastructure will also enable the dynamic, on-demand extensions of the connected Desktop Grids with Cloud resources. As such EDGI users can benefit of the versatile and flexible eco-system provided by EDGI. The EDGI production infrastructure will be offered as service for EGI and NGI user communities. It will also serve as a demonstration for NGIs to extend their eco-system with Desktop Grids and Clouds. EDGI will establish a European Desktop Grid federation to coordinate DG-related activities in Europe both for solving technical issues as well as to attract volunteer DG resource donors by disseminating results of the EDGI and EGI-related projects. The European Grid Federation and EDGI will work in strong collaboration with EGI, EMI, NorduGrid, Unicore Forum and interested NGIs.
 

International Desktop Grid Federation International Desktop Grid Federation

EDGI supports the http://desktopgridfederation.org/. The International Desktop Grid Federation brings together Desktop Grid operators, developers, organisations and companies working with Desktop Grids and intergration with other infrastructures through EDGeS Bridge Technology.

About EDGI About EDGI

EDGI is a project managed by the EDGI consortium.

EDGI is supported by the FP7 Capacities Programme under grant agreement nr RI-261556.

                                                                                                                                  

© EDGI Website and EDGI: EDGI Consortium 2010 - 2011 - 2012.