Canned Visualizations

Abstract

Details

Download

Project

Aah, you wanna know more? All right, here we go!

Problem setting

Visualize this: You have access to some high performance computer and you use it for, say, computational fluid dynamics simulations. The computation runs in batch mode, i.e. you tell the machine what to compute, the machine stores it away in its queuing system, and when the other people's jobs are done it does your job. And after some hours to a few days have passed, you get an email that your job is done and the 300GB of data it produced are available in /tmp for you to view. So you sit down in front of your favourite graphics workstation, fire up your favourite data visualization tool and realize that you have a problem.

It will take you several hours to download the whole data set from off the host. It will just bust your hard disk. And it definitely won't fit into memory. So what you need is some visualization tool that runs on the host.

About all the big data visualization systems will do that for you. But you want to view your data interactively. Turn it on the screen with your mouse. Fiddle with, say, some isovalue until you see the features you wanted to see. But your host account only is for batch access.

Concept

So what we do is to split the visualization pipeline into two parts. The head needs quick access to the big chunk of data and maybe to the big iron's computing power, too, so this part will have to run in batch mode on the host. The tail of the vis pipeline ends on your very screen, and you would like this part to run with interactive response time.

A nice point to split the two is between creation and rendering of the visualization geometry. That means, the host does everything except painting the triangles onto the screen. This of course implies that there are triangles to paint. You can easily extend this concept to other geometry objects, but direct volume rendering doesn't fit in here too well.

Communication between the two parts is naturally asynchronous. So what we lose is the ability to interactively change the parameters that are used by the host part. That includes isovalues! But we can alleviate this constraint by prechoosing a whole set of parameter values. Given that the creation of a single visualization geometry is very fast compared to the ``real'' computation, creating a few dozens of them won't slow things down too much. So now we cannot take any value for our parameter but only one of the prechosen set. But we can still make it look like dragging a slider until the picture looks right.

The System

The system consists of two parts, one running on the HPC host as part of the simulation run, and one running on the workstation or PC interactively. No, actually the server side isn't done yet. All that exists is a first version of the client side viewer. But let's ignore that for now.

The server side creator performs the actual visualization algorithms and emits a set of geometry descriptions. For now we use the Open Inventor's .iv file format for that. But when file sizes get bigger, that won't be suitable any more. So we think about some view dependent accessible progressive mesh. But that's dreams of the future.

The client side viewer gets the files and loads them into a plain vanilla Inventor viewer widget. It can read the files from local disk (ah, well), it can transparently read them from an ftp server (qt gives us that one for free), and it can suck them through an ssh login connection (not that transparently) as this is the only way to access our HPC. It recognizes sets of files and loads them as a batch (say, the airfoil's geometry, a coordinate system and the stream ribbons), and it recognizes parametrized sets (say, the same isosurface for different isovalues) and displays only one of them at a time, offering a slider to ``tweak'' the parameter.