• mi-cts08.doc


  •   
  • FileName: cts08.pdf
    • Abstract: our efforts to develop a capability, coupled with the Earth. System Grid (ESG), for sharing an entire executable. workspace of visualization among collaborators. ... Further enabled by the recent creation of the Earth. System Grid [1,14], as a universal global repository of ...

Download the ebook

Web Enabled Collaborative Climate Visualization in the Earth System Grid
Wesley Kendall, Markus Glatter, Jian Huang Forrest Hoffman and David E. Bernholdt
University of Tennessee, Knoxville Oak Ridge National Laboratory, Oak Ridge
E-mail: {kendall, glatter, huangj}@cs.utk.edu Email: {hoffmanfm, bernholdtde}@ornl.gov
Figure 1. Illustration of our web-enabled data-intensive visualization system.
ABSTRACT 1. INTRODUCTION
The recent advances in high performance computing, The recent advances in high performance computing,
storage and networking technologies have enabled storage and networking technologies have enabled
fundamental changes in current climate research. While fundamental changes in modern climate research. An
sharing datasets and results is already common practice ever-closer integration of observational data with
in climate modeling, direct sharing of the analysis and simulations of an unprecedented scale, both in terms of
visualization process is also becoming feasible. We report resolution and size of ensembles of runs, is already under
our efforts to develop a capability, coupled with the Earth way. Further enabled by the recent creation of the Earth
System Grid (ESG), for sharing an entire executable System Grid [1,14], as a universal global repository of
workspace of visualization among collaborators. datasets from different scientific models, climate science,
Evolutionary history of visualizations of research findings and climate modeling in particular, has become
can also be captured and shared. The data intensive fundamentally collaborative in a widespread manner.
nature of the visualization system requires using several Many previously unapproachable problems can now be
advanced techniques of visualization and parallel studied. In a way, today's globally collaborative climate
computing. With visualization clients implemented research represents a central trend that is taking place in
through standard web browsers, however, the ensuing all areas of computational sciences. Scientists not only
complexity is made transparent to end-users. We share datasets and research results, but more importantly
demonstrate the efficacy of our system using cutting edge they can now also share their research processes.
climate datasets.
In this work, we describe a prototype system that enables
sharing large-scale visualization of climate simulations in
KEYWORDS: Large Data Visualization, Climate an executable form. By exchanging a few compact
Modeling, Distributed Collaborative Visualization. specification files and URLs (for instance, via email),
geographically separated scientists can asynchronously
execute and view the same visualization workspace for
collaborative scientific insights. The same specification The rest of the paper is organized as follows. Section 2
can be applied to other datasets for comparative study. contains a brief discussion of the related background of
our work. The technical components for handling data
Scientists can also edit and tweak the specifications in intensive visualization are provided in Section 3. Our web
ways that reveal additional scientific findings and browser based visualization workspace is described in
efficiently communicate with their colleagues about the Section 4. In Section 5, we present a sample workflow
modified visualization. An email thread, with the using our system.
evolving versions of visualization specifications as
attachments (or the same content published via web 2. Background
blogs), is a natural way to document and share a
collaborative research process that has lasted over a long Our driving application is to visualize results from global
period of time. This computing interface not only makes it coupled climate-carbon cycle model simulations produced
easier to reproduce the visualization results on other by different models. We draw our research motivation
datasets, but also in essence provides a possibility to from the current thrust to generate accurate simulations of
critique and disseminate a practice of science. the global carbon cycle that model the interactions and
feedbacks between the terrestrial biosphere and the
The data intensive nature of collaborative climate climate system. The Carbon-Land Model Intercomparison
visualization presents many challenges that computer Project (C-LAMP, www.climatemodeling.org/c-lamp [5]
scientists must resolve. However, the technical was initiated to allow the international scientific
complexities must be made transparent to end users as community to thoroughly test and intercompare such
much as possible. In our system, we have implemented terrestrial biogeochemistry models through a set of
our user interfaces universally through web browsers. carefully crafted experiments. Well-defined metrics have
Users are not required to install any additional software been established for comparison of model results against
beyond those that are standard with all web browsers. best-available observational datasets, and models are
graded on their scientific performance with respect to
Since in essence our collaborative visualization system these metrics. Visualization tools and diagnostics are
and the ESG are both designed to support the global particularly helpful in uncovering model biases and
community of climate research, let us try to distinguish discovering ways for improving individual models.
the role that each system plays from a high level point of
view. To date, the ESG's primary function and benefit has The visualization aspect of this task is very demanding for
been to make data available to a much larger and more several reasons. First, there are a large number of
general community than had previously been possible. A variables involved in each simulation. Exacerbated by the
key goal of the next generation ESG architecture is to need to study multiple runs in a cohesive manner, the
serve as a point of integration coupling data access with combinatorial space that needs to be explored, even just
routine analyses and visualizations, helping to reduce the task of studying two variables from two simulation
download volumes and user-side storage and processing runs, is overwhelming. For instance, scientists already
requirements. have some understanding of how net ecosystem exchange
relates to net primary productivity. Does the relationship
The ESG team is working to integrate with some exist as expected in a peta-scale simulation, maybe across
"standard" climate analysis and visualization modules, different time spans of different runs? This model of
such as the Live Access Server [10], but the development investigative study and the need of high interactive rates
of novel visualization techniques and tools is beyond the present a challenge for large data visualization.
scope of the ESG project, rather, they are expected to be
developed through external collaboration. Our work is Second, a very desirable feature in data intensive
one early example of this concept in action. Collaborative visualization tools in this field is for the tools to be
server-side visualization is novel for the ESG community, available on-demand. In climate research, exhaustively
where most of the ESG server-side processing use cases iterating over all possible combinations is prohibitively
focus on the scientific needs of individual users. With the expensive. One can only use the domain expertise as a
idea of sharing analysis and visualization results in a guide to practically sample a small portion of the entire
model similar to social networking just starting to gain combinatorial space. This process of scientific research
attention in the field [13], no existing efforts can provide often involves an element of spontaneity. Hence it is
such a level of collaboration for data intensive preferable not to require advanced reservation before each
applications. These large data applications are the norm use. We do not know of any previous systems that can
for our system and the ESG, both in terms of the amount support such spontaneous use of visualizations by
of data that must be handled and the complexity that is geographically separated concurrent users. Thankfully,
involved in the scientific process of research. powered by the ESG, it is now feasible to investigate how
on-demand visualization capabilities can be developed for reduce the local computing and storage requirements for
large-scale climate modeling research and tested in a real- many users. The work presented in this paper can be seen
world environment. A brief overview of the ESG follows. as an example of the server-side processing the next-
generation ESG plans to support. Because of community
2.1. Earth System Grid interest in the next generation of biogeochemistry models,
our work has focused on visualization of the data held in
The Earth System Grid (ESG, www.earthsystemgrid.org) the C-LAMP [5] archive, which is physically located at
[1, 14] has been developed as a tool to make the large ORNL, but the technique described is broadly applicable.
volumes of climate simulation data more easily
discoverable and accessible to a larger community of 2.2. Collaborative Parallel Visualization
researchers interested not only in the simulations using Heterogeneous Resources
themselves, but also in understanding the impacts of
climate change, and possible technological and policy Researchers in distributed graphics pioneered the creation
strategies for their mitigation. The ESG collaboration of distributed collaboration capabilities involving 3D
includes seven US research laboratories (ANL, LANL, graphics. The resulting technology is already in
LBNL, LLNL, NCAR, NOAA/PMEL, and ORNL) and a widespread use by distributed systems of virtual reality,
university (USC/ISI), and is sponsored by the US networked games and visualization [2, 9]. Unfortunately,
Department of Energy, Office of Science. when it comes to data intensive applications, involving
terabyes (TB) to petabytes (PB) datasets by today's
standards, we still do not have a widely accepted solution
that could scale up both in terms of the amount of data
that can be handled and the number of concurrent users
that can be supported.
Large-scale data intensive applications dictate the use of
parallelism. However, computing platforms that are
capable enough to handle large datasets are still far from
being a commodity. Even with a large-scale parallel
machine available, it still takes time to develop
capabilities based on non-primitive use of parallelism.
This problem is particularly acute when it is impractical
to provision a homogeneous large-scale visualization
facility for dedicated use by a collaborative project. With
today's project needs evolving faster than before, the issue
of not having the right kind of parallel platform promptly
available is a rather fundamentally limiting issue.
Figure 2. Portal Architecture of the ESG. In this regard, it is very appealing to alternatively consider
using distributed heterogeneous resources as the common
When climate simulation results are published into the platform to support on-demand parallel visualization. This
ESG, metadata is extracted providing a detailed would allow one to efficiently use all resources available,
description of the simulation context and the variables regardless of their geographical location. On top of this
represented in the datasets. ESG users can browse or platform, visualization can be implemented according to
query the metadata catalog through a web portal to the software-as-a-service model and allow fast evolution
discover datasets of interest, and then request their and adaptation. In a way, this approach has been well
retrieval from the archive site that holds them, most often studied by researchers in networked games and virtual
at the computer center where the simulation was environments [12] and grid computing [3]. In those
originally run (see Figure 2). applications, each user's own personal computer is an
individual piece of heterogeneous resource. However, to
Among the key goals of the next generation of the ESG assemble data intensive visualization services with fault-
architecture, currently under development, are much more tolerance, high scalability and high performance, we still
widely distributed deployment, and integration of analysis have significant road blocks to overcome, due to the data-
and visualization capabilities on the server side in order to intensive aspect and investigative mode of usage that
minimize the amount of data that has to be delivered to restricts orchestrating user interaction according to
users over (relatively slow) wide-area networks, and to planned scripts.
As we have discovered through extensive previous typical hardware maintained by ESG. Climate-carbon
research in large-scale distributed visualization, cycle model results are queried, subsetted and retrieved
distributed collaborative visualization systems that do not from high-performance systems maintained by the ESG.
include fault-tolerance and heterogeneity as primary The data are transformed, partitioned and replicated on
design goals would be very hard to use, even just to carry standard Internet computing nodes for fault-tolerance and
out experiments mimicking real-world scenarios for later use in on-demand parallel visualization. In a way,
multiple concurrent users [6]. This is because, in a real the nodes in our visualization system can be considered as
scenario, users join or leave a collaborative session on a set of function-rich data caches. A set of visualization
different schedules. The client software used by servers manages the parallel visualization process
geographically separated users could frequently be turned (illustrated as blade servers). The colored arrows illustrate
on and off. To assume that a resource, regardless of type parallel visualization results computed on heterogeneous
(e.g. graphics, computation, storage, and a guaranteed nodes are being assembled by the schedulers, which then
level of network bandwidth), would always be available, forward the results to the web browser running on the
even with prior reservation, could quickly render a system client workstation. In the client's visualization workspace,
impractical. We have discovered through previous the final visualizations are displayed for viewing,
research that the primary factor is that the parallel annotation and interaction.
scheduling algorithm must be designed to consider
heterogeneity [7]. A set of concurrent users can switch on the visualization
clients through their web browser to work in their
Although collaborative visualization has been a traditional individual visualization workspaces. The visualization
topic of research, its focus has been mostly about a operations initiated by each workspace are computed in
synchronous mode of collaboration. In other words, all parallel and orchestrated by a set of fault-tolerant and
participants are assumed to come online at the same time heterogeneity-aware visualization schedulers. The most
and work on the same subject. With the advent of Web generic visualization operation in our system centers
2.0 technology, especially represented by the introduction around the notion of query-based visualization. After
of ManyEyes [13], focus in many fields has now been discussing some details of how to implement this
shifted to study how geographically separated users can technique in a scalable manner, we follow with a
asynchronously collaborate by means such as community- discussion of how to integrate parallel query-based
shared annotations and blogs. It would be particularly visualization using heterogeneous computing resources.
interesting and impactful if such asynchronous mode of
collaboration can be inherently supported by data 3.2. Parallel Query-Based Visualization
intensive visualization applications.
Decadal to century time scale climate simulations
In summary, while distributed collaboration is a concept typically output a large number of two- and three-
that has been pioneered, reinvented in the past and is dimensional variables at regular intervals, usually
already in widespread use by millions of current Internet monthly. While identifying features is difficult, it is
users, the main technical hurdle for users to effectively actually intuitive and practical to select a subset of the
collaborate on tera- to peta-scale datasets is still a data from within that high-dimensional value/variable
fundamental one. It is not a matter of simply addressing a space to obtain a qualitative understanding of the overall
few tradeoffs. We must develop creative technologies as results [11]. This approach is not hard to implement if the
well as creative ways to use current technologies to meet resulting dataset is small enough to be stored entirely in-
this challenge. As an attempt, our system is described in core, but to handle datasets sized at hundreds of gigabytes
Section 3 using climate visualization as an example. and beyond, more sophisticated solutions are required.
3. PARALLEL VISUALIZATION Our solution [4] involves designing specialized scalable
visualization data servers with large-scale parallelism. It
3.1. Overview indexes general data items, including vertex, voxel,
particle, or a general tuple describing height, weight and
Our prototype system inherently utilizes heterogeneous gender. Due to this design consideration, it is independent
computing resources. All nodes involved can be of grid type. The core data structure is a heavily
distributed by nature; in fact, each node is uniquely optimized M-ary search tree to support parallel
identified by its IP address. Any regular personal visualization. The tree structure as metadata only amounts
computer can serve as a node used in a session of parallel to ~1% the size of the whole dataset, which can exist
computing. Figure 1 illustrates the concept. The high entirely on external hard drives and can be compressed.
performance computers (lower-right corner) represent the Our method only decompresses the parts of the data (and
caches the decompression results) that are used by the
user. The compression rates vary from dataset to dataset. memory of all data servers be sufficient to hold the entire
For typical datasets, we saw 20x compression rates, while preprocessed data in core. However our implementation
on highly turbulent or noisy datasets, we could obtain as remains operable even if this requirement is not met.
low as 4x compression. Using a conservative compression
rate of 4 times as a benchmark, a typical mid-sized cluster 3.3. Fault-Tolerant Job Scheduling
could already support parallel visualization of a
dynamically queried dataset sized at 1 TeraByte (TB). Each data server operates as an independent parallel
process. Server processes will be replicated on multiple
The M-ary tree serves the role of meta data to guide a nodes for scalable performance as well as fault-tolerance.
search process. Assuming the balanced M-ary search tree Considering each request to any one of the servers as a
is of depth N, a practical combination could be M = 256 task to be completed in parallel, the scheduler of the
and N = 3; in all situations, M should always be parallel tasks is crucial for our system.
significantly larger than N. This is one of the primary
differences between this search data structure and The scheduler we use was designed to achieve scalability
previous data structures, such as interval tree, k-d tree, while considering heterogeneity and fault-tolerance [7].
quadtree and octree. Due to the large branching factor, M, Our scheduler implements co-scheduling of computation
the M-ary search tree requires little storage space. Actual and replication, by adaptively measuring node
data items are not stored in the tree, but in a linear list performance to dynamically assign visualization tasks and
sorted by a key function. The leaf nodes of the tree store direct runtime data movements. This is necessary because
pointers to the respective records in a sorted linear list. the nodes used for parallel computation will be
heterogeneous, non-dedicated nodes. Geographically
M-ary search trees have been widely used, for instance in distributed users access the system with competing goals
the form of B-trees as in database systems. We have of obtaining computing resources.
discovered that conventional methods to access records
by traversing B-trees to be too expensive, both in terms of Our scheduler needs to discover fast processors on the fly,
the large number of addressing operations and caching assign as many tasks to them as possible and avoid being
performance. To address this, we use a novel method to stalled by slow or faulty nodes. We devised three generic
accelerate range searches in an M-ary tree, optimized mechanisms for this need: (i) a dynamically ranked pool
specifically for multivariate datasets [4]. of nodes, (ii) a two-level priority queue of tasks and (iii) a
competition avoidant task assignment scheme. Each
Data items are partitioned into groups by round-robin processor is ranked by its estimated time to process a task
assignment according to high-dimensional space filling of unit size. This measurement roughly reflects
curve [8] order in attribute space. By using this type of performance of processors delivered to the experiment. It
data partition to distribute data amongst all visualization is updated adaptively by computing a running average of
data servers, we can achieve a nearly optimal load- measured processor performance, with older
balance for almost all kinds of queries. An M-ary search measurements given an exponentially decreasing weight.
tree is then used to manage the data on each server.
A two-level priority queue maintains unfinished tasks.
For a user of our system, e.g., an application scientist, it is The higher priority queue (HPQ) contains tasks that are
often of interest to interact with the dataset of a simulation ready to be assigned and the lower priority queue (LPQ)
to verify correctness or to explore newly formulated contains tasks that have been assigned to one or more
hypotheses. To this end, one should be able to navigate processors but not finished. Initially, only the first w tasks
not only in time and space, but also in the high- are placed in the HPQ, where w is the size of task window.
dimensional attribute space. Additionally, every server in It controls the degree of parallelism and number of tasks
our system accepts queries in the same way that web that can be finished out-of-order. In most cases, w is
servers function. This allows parallel servers to operate greater than the number of available processors so that
persistently, while the client can choose to dynamically every processor can contribute. However, w is decreased
start, stop or simply shutdown. in cases of severe resource contention as a processor
"back off" strategy. Each task in the HPQ is keyed by the
Our approach is relatively easy to deploy on networked minimum unit task processing time of all nodes holding
commodity computers, whether they are clustered or not. the required data partition. This priority ranks new tasks
The necessary number of parallel data servers depends on by their likelihood of being finished by a fast node. Each
the size of the dataset. As long as more computers are task in the LPQ is keyed by its estimated waiting time.
available to function as data servers, our system could be Tasks in both the HPQ and LPQ are sorted by their keys
scaled to handle larger and larger datasets. For in decreasing order. Figure 3 shows an illustrative
performance reasons, it is desirable that the combined
example of how the scheduler operates in general, and a is held by 3 server nodes, and each server node holds 3
snapshot of the window-based task assignment. pieces of data. In other words, assuming a 1 TB dataset
has been partitioned for hosting by 100 data server, each
When a parallel visualization starts, tasks in the HPQ are server holds 30 GBs of the data (before compression).
sequentially assigned to available processors and demoted
to the LPQ. In case of failure, the task in the LPQ is Any visualization query is parsed and issued first to only
promoted back to the HPQ so that other processors can one server holding data that are needed. Secondary
take it over. When tasks are completed, every available requests are only issued when a result is not returned by a
node will be directly assigned the first task in the HPQ server before a timeout period has expired. The processes
that it can handle. In this way, slow processors do not for different visualization queries take place in parallel. It
compete for tasks with fast processors to ensure fast is typical for each user's visualization workspace to
processors are assigned as many tasks as possible. If a include a handful of different visualizations, while several
processor is not assigned a task in the HPQ, the first task users can use the system concurrently.
in the LPQ that it can handle is considered. This is the
slowest task among all unfinished tasks that the processor
can assume. Thus, multiple processors can work in
parallel on the same unfinished task. If any instance of the 4. WEB-ENABLED INTERFACE
duplicated tasks is done, other instances are aborted.
All the visualization results are viewable through a
standard web browser, together with an information
visualization viewer that monitors the parallel run. The
web interface was designed using Flex Builder 2 and
Action Script 3. To create a generalized framework for
any web visualization, the web interface is set up by an
XML configuration file. This configuration file specifies
what controls are to be shown on the interface along with
the properties of the controls and the visualization server
(hostname and port). Each control is given a separate
smaller window inside the web console, giving users their
own desktop for organizing the visualization.
Once the controls for the visualization are set up on the
web interface, it then connects to the visualization server
with the Action Script 3 Socket library. Using our own
visualization server API, the visualization server can relay
messages to the web interface using notification messages.
This message transmission procedure is set up in the
Figure 3. Our Heterogeneous Scheduler. configuration file. The user has complete control of which
components of the web interface the messages are sent to.
There are also situations where moving data from a
slower processor to a vacant faster processor may be To allow the components of the web interface to interact
beneficial. To ensure that the overhead of data movement with the visualization server, the visualization server must
does not exceed the benefit that we might gain, bandwidth register callback functions for any component of the
information between processors needs to be acquired. We interface that might interact with it. For example, if a
then leverage a multi-source partial download scheme query comes from a video viewer on the web interface,
with deadline to drive data movement between nodes. the user would register a specific callback function, and
any queries from the video viewer with that ID would be
3.4. System Integration sent to the function in a special generalized format that is
easily parsed by another function in the API. This API
Currently data replication is our only method of fault- and the configurable web interface combined allow any
tolerance for addressing failing processors as well as user to create their own web visualizations without any
overloaded processors. Each data server node holds a knowledge of web or socket programming. It also allows
portion of the entire dataset to be visualized. Through the user to plug in any back-end system, regardless of a
experimentation, we have found a replication factor of k = cluster or a home desktop, to their web visualization.
3 to be sufficient for most situations and not cause too
much operating overhead. In this case, each piece of data
4.1 Data Selection Interface with ESG visualization of the captured one. Since these captures are
still associated with the individual sessions, multiple
In the absence of more flexible interfaces to the metadata scientists could be in a web visualization session at once
and data, which are currently under development by the sending each other these URLs to simulate a live, real-
ESG team, we have implemented our prototype system as time collaboration of the same visualization.
a post-processing client to the ESG. A web-based client
(shown in Figure 4, top-left) utilizes the the FTP-based 5. TYPICAL USE CASES & DISCUSSIONS
access provided to the C-LAMP archive to allow the user
to browse and select the data to be downloaded to our We use the screen capture shown in Figure 4 (bottom) to
system. When submitting their request, users are asked to demonstrate a typical use of our visualization system. Our
provide their name, affiliation, and email address. When current user interfaces illustrated in Figure 4: (top-left) for
the download is complete, they receive an email with their selecting datasets from ESG; (top-right) for specifying
session ID, password, and a link to process the data. The which subsets of the selected datasets are to be further
password provides a simple mechanism to restrict control processed for visualization and analysis; and (bottom) is a
of the processing of the data to the requesting user, but screen capture of the web-enabled visualization interface
anyone will be able to analyze the data in a given session. in operation. This example shows that the user selected C-
In the next-generation ESG architecture, we anticipate LAMP runs from the ESG and chose variables such as
that it will be possible to provide the user with a seamless CO2 concentrations, net ecosystem exchange (NEE), net
interface, including integration with the ESG's primary productivity (NPP), and vegetation temperature
authentication and authorization model to allow more (TV). The time span of interest is from 1800 to 2005. A
control over access to the analysis results if desired. user could either examine how each variable evolves over
time in a 2D movie, or do the same with the (first order)
4.2 Data Processing & Linking with Data derivative of each variable.
Servers
In addition, we allow the user to study pairs of variables.
After the files have been transferred, data processing It may be interesting, for instance, to visualize the first
starts for visualization purpose. The user goes to a web derivative of NEE vs. the first derivative of NPP. If both
address, enters their session ID and password, and then derivatives are of the same sign, we assign the colors blue
browses/selects the variables in the NetCDF files. As (both positive) and red (both negative), while green and
illustrated in Figure 4 (top-right), the variables are added yellow are used for cases where the two derivatives are of
to a cart similar to how the previously mentioned FTP opposite signs. This is useful for revealing patterns in
client worked. Once the users select all the variables that which the two variables, NEE and NPP, vary in tandem of
are of interest to them, the data is then sent to the data each other. The two variables could be given a different
servers and processed. A configuration file for the web time lag as well. For instance, have the time sequences of
interface is automatically generated, and a link to the web NEE start from January 1950 and NPP start from January
visualization is sent to the user. The user may then query 1980. Similarly, we can also study the monthly pattern
the variables chosen in the data processing stage. Any variations between NEE from 1900-1920 vs. NEE from
person may go to this link and analyze the data. 1980-2000.
A user can create an unlimited number of analyses. Each
4.3 Collaboration through the Web Interface analysis is given a separate floating viewport. All
viewports can be moved around in the workspace in the
Each web visualization has a specific session ID that is web browser. This design allows the user to selectively
assigned to it when created. Users can type in the URL to position viewports in neighboring positions for closer
the web interface along with the session ID as a parameter comparisons. For an intuitive control by our domain users,
in the URL and this will bring up the visualization. Along each viewport essentially acts as a movie viewer, with its
with each interface is a control that allows users to leave own set of controls including play, pause, and time tick.
comments along with the previous comments that have In addition, at the bottom of the browser, there are
been left. Since the interface is connected to a background overriding play and pause buttons for the whole
server via sockets, this allows the comments to be updated workspace. This design allows each viewport to be set to
in real time. If multiple users are running the same session, a different starting point and to be started and paused in a
they will all see any new comments posted w


Use: 0.4119