Image Processing for Space & Earth Observations
Multisource & Multispectral Image Processing via Bayesian Inference
MIV team ICUBE

CNRSUnistra

Development of a visualization tool enabling large IFU data cubes upload, navigation and interpretation
Title : CubeVisualization
Keywords : Visualization, massive data sets, hyperspectral data, IFU
Funding : ANR Dahlia
Start date : 2009-01-01
End date : 2013-01-31
  • People
  • Summary
  • Description
  • Bibliography
  • Links
  • Download
  • Uncertainty representation
  • People

    Participants

    Collaborators

    Summary

    Visualizing multi-dimensional observation data sets is a common challenge for observational sciences and crucial for Integral Field Spectroscopy (IFS) in astronomy. For the specific case of IFS data cubes, like MUSE, the size of the observation requires ad-hoc tools to be able to upload, visualize and navigate inside the 2D+lambda cube. This work package aims at :
    • providing a viewer that enables to see the IFU data cube and navigate along the observational axes: position (x,y) and wavelength lambda;
    • allowing for intelligent browsing by linking to observational metadata, provided along with the data cube, e.g. variation of the Line Spread Function (LSF) along the wavelength axis, variance data corresponding to the observed data, etc...
    • allowing for comparison of original data with analysis results, available in heterogeneous formats: extracted sources lists, sky background statistical features along position and wavelength axis, fusion-combined data;
    • comparing visually several data cubes with each other.

    Description

    Key objectives

    This project aims at providing a visualization tool for IFU (Integral Field Unit) data cubes and their associated parameters (variance, psf,...), either by re-using existing viewers or by designing and developping a specific application. The huge amount of data in an hyperspectral image (up to 2.4Go for MUSE data cubes) requires the development of new tools for visualizing and browsing the cube. Indeed, most of the current 2D/3D viewers are not suitable because the visualization of large hyperspectral data (up to 4000 bands) requires specific design requirements and processings:
    • parameter and meta-data visualization: hyperspectral images are often available with parameters and meta-data (whose size might be the same as the image) that should be taken into account during the visualization process. For instance, using variance or PSF information (on each sensor location) helps for signal interpretation, average spectra or image extraction helps to analyse trends in the data set.
    • processing features for data exploration: the application could provide a set of dedicated algorithms that perform intermediate views (variance weighted averaged spectrum, background substracted spectrum or image, averaged image on a wavelength interval, etc...) and offer innovative viewing modes. Advanced spectral analysis functionalities developped in the context of VO dedicated tools, like VOSpec or SPLAT should be available through this tool thanks to the SAMP protocol ;
    • user-friendly interface: while viewing complex images, the user needs a proper cursor reference in the coupled spatial and spectral domains in order to facilitate data exploration as well as a logical intuitive user interface;
    • memory-efficient: the simultaneaous visualization of several hyperspectral images requires the use of a strategy that minimizes the amount of allocated memory while providing fast access to the data ;
    • multiplatform support: Unix, Windows, MacOS... The scientific community uses a whole range of operating system. The software must be as portable as possible ;
    • extensibility - plugins: as this tool might be used for different kind of visualizations and analysis, it is useful to provide a convenient way for users to add new capabilities to the tool.
    Moreover, this visualization software must support a set of basic features generally covered by most data cube visualization softwares (IFU, radio, interferometry) dedicated to the visualization of IFU data:
    • loading and saving 3D data in various formats: FITS extension, HDF, etc...
    • simple display functions: for spectrum, image plane, polychromatic maps, interpolation...
    • contrast enhancement via selection on the luminosity range ;
    • spectral browsing of hyperspectral images ;
    • mean operator on both spatial and spectral frames ;
    • source catalog overlap and comparison.
    Design of the software for the visualization of hyperspectral data cubes
    Design properties expected for a visualization software for hyperspectral data cubes

    QuickViz

    QuickViz is a Java prototype for such a datacube viewer. It has been designed as a plug-in application of the Aladin VO application which already provides multi-view, multiband images handling catalog overlap and interoperability with other VO applications. QuickViz is currently developed and maintained by the PASEO team by Matthieu Petremand and Mireille Louys with the help of Pierre Fernique at the CDS (for the interaction with Aladin). It is dedicated to the visualization of hyperspectral cubes such as IFU to be obtained with the MUSE instrument (ANR Dahlia) where the variance information is available for each sensor location. Nevertheless, any kind of hyperspectral images may be supported. QuickViz may also run as a standalone application (without Aladin). In addition, QuickViz provides processing algorithms, specific representation modes for variance or uncertainties and a useful interaction with Aladin (catalogs overlap, calibration functions, header management, 2D display...) giving access to a set of scientific features in use in the astronomical community. Indeed, the design strategy of QuickViz has been thought in order to maximize the extensibility of the plug-in. Users can then add:
    • processing algorithms acting on spectra (mean, weighted mean, convolution...) ;
    • processing algorithms acting on image frames (mean, colored composition, frame extraction...) ;
    • basic visualization modes (error bars, histogram...) ;
    • advanced visualization modes (animation, interaction, popup window...) ;
    For each of these modules, management, display and execution are totally supported by QuickViz.
    The following sections introduce the main features of QuickViz.

    Spectral extraction

    If you click on the image plane in the Aladin frame, the spectrum corresponding to the mouse location in the image is extracted from the data cube, added to the spectrum stack of QuickViz, displayed on the spectrum panel and a position tag is shown on the Aladin's image plane too. If your cube does not provide a FITS standard compliant spectral calibration, QuickViz will ask you, only once, to fill in missing calibration values. In the same way, an astrometrical calibration can be defined from the Image menu of Aladin (Astrometrical calibration entry). Spectra cannot be extracted from Aladin if your data don't provide an astrometrical calibration.
    Each newly extracted spectrum replaces the previous one in the spectrum stack to simplify its management. If you want to permanently store a spectrum, press the key K (for Keep) while selecting the spectrum, set it a name and validate. This single-spectrum extraction mode can be disabled in the Extraction menu (Single extraction entry).
    As the spectrum stack grows, kept spectra can be simultaneously displayed (shift + left-click on spectra you want to add (resp. remove) to (resp. from) the selection) even if they haven't been extracted from the same cube: the spectral scale is updated according to the current spectrum selection whereas spectra having incompatible units are not selectable. In addition, right-clicking on a spectrum in the stack displays a popup menu allowing you to:
    • change the display color for the spectrum;
    • rename or erase the spectrum;
    • view or edit the spectral calibration parameters (see below).
    All these actions (except for Rename) can also be triggered from the shortcut toolbar located above the spectrum stack.
    At last, when you let your mouse pointer linger over a spectrum in the stack, a tooltip appears and provides relevant spectrum information.


    Extraction and superposition of spectra

    Panel interactions: zoom, spectral navigation, selection and pan

    Once you have extracted spectra of interest, four panel interactions, available in the shortcut toolbar, can be enabled by left-clicking and dragging the mouse on the spectrum panel:
    • spectral navigation (accelerator key: C): a calibrated cursor allows to navigate through the cube frames, i.e. the cube frame displayed in Aladin is updated according to the location of the cursor;
    • zoom (accelerator key: Z): a calibrated zoom window precisely defines both spectral and spatial zoom ranges. Moreover, double left-clicking on the panel resets the zoom factor, as activating the Reset zoom entry from the View menu;
    • selection (accelerator key: S): spectral range selections (joint or not) can be defined over spectra and thus restricts the spectral domain where a specific algorithm is applied (see below). Once you have defined a selection, left-clicking on one of its boundaries allows to either resize it (by dragging the mouse) or open a pop-up menu to delete it or edit its properties (color, range...). QuickViz also provides some basic features that can be applied on a selection (available through Selection menu and toolbar shortcuts). You can then:
      • create a new selection area;
      • select the whole panel;
      • reset a selection;
      • merge the selection areas;
      • chop the spectral axis into a given number of chunks of the same size (very useful for RGB compositions for instance);
      • save and restore selections;
    • pan (accelerator key: P): move around your data without changing the zoom factor;
    In addition, depending on the current enabled interaction, others might still be available through right or middle (if any) mouse buttons. For instance, if the cursor is enabled, the right mouse button controls the zoom window whereas the middle one allows to define selections. At any time, let your mouse pointer linger over an interaction button shortcut to get more information.


    Zoom and spectral selection


    An interactive zoom is also available from the shortcut toolbar (accelerator key: R) and might be enabled together with other interactions. Once activated, moving the mouse over the panel displays its zoomed version into a circle or square (if rounded windows are not supported on your operating system) shaped window. The zoom factor is controlled by the mouse wheel. This feature is useful to focus on a specific area while keeping an eye on the whole panel.


    Interactive zoom

    Spectrum properties

    At the bottom of the application frame, a toolbar displays parameter values that are related to the current visualized spectra like minimum and maximum values, mean, standard deviation, name, coordinates of the cursor position... By right-clicking on this toolbar, you can customize the displayed information and copy them into clipboard.


    Customizing the toolbar

    Managing the spectral calibration

    Whenever you right-click on a spectrum in the stack, a popup menu appears that allows you to manage its spectral calibration ("Edit calibration" or "View calibration" menu entries). If your spectrum has been loaded from Aladin, you will only be able to view the calibration. But, if it comes from the output of an algorithm (see below), you can update its spectral calibration (unit, reference, delta...).


    Managing the spectral calibration

    Multithread loading

    As QuickViz is intented to deal with large hyperspectral data cubes, the spectrum extraction may be time-consuming! That's why QuickViz implements a multithread loading scheme that allows to still use the application while spectra are loading. Moreover, QuickViz keeps you aware of the progression of the extraction process by displaying a progress bar in the spectrum list.


    Multithread loading

    Visualization modes

    Spectra can show very packed line patterns and therefore can't be simply drawn with simple lines, crosses or points as any other functions, especially when you superimpose spectra. QuickViz allows to customize the way spectra are displayed by introducing various visualization modes. These drawing modes are accessible via the upper shortcut toolbar.


    Two different modes of visualization: bars and lines

    Variance extraction and visualization

    When you handle a hyperspectral cube, you may have access to its variance values in the form of another hyperspectral cube. QuickViz provides a way for the joint extraction of spectra and associated variance values. This extraction mode is enabled by selecting the Spectrum + Variances entry in the Extraction menu. Thus, for the next spectrum extraction, QuickViz tries to automatically detect the variance cube associated to your data (based on extension names within the same FITS file, i.e. DATA for data cubes and STAT for variance cubes). If it failed, QuickViz asks you to manually select the variance cube to be associated to your data. Once both cubes have been linked, variance values are extracted for each spectrum and displayed, along the spectral axis, with a dedicated visualization mode named error covering. Another mode (error bars) is also available through the shortcut toolbar.


    Visualization of variance values along the spectral axis: error bars and error covering


    QuickViz also allows you to visualize your data together with variance values across the field of view (see link for details) in the form of a video. This beta feature can be tested from the View menu by clicking on the 2D variance entry. Once the variance visualization window is opened, you can select a cube's frame, for which you want to visualize the variance movie, by exploring a cube thanks to the cursor in QuickViz. A left-click on the Play button (in the shortcut toolbar of the movie window) starts the video and the next cursor move in QuickViz automatically stops it and selects a new frame (unless the Lock button is checked). At any time, while the video is playing, you can:
    • stop the video by left-clicking on the Stop button in the shortcut toolbar;
    • zoom in on the frame by right-clicking and dragging the mouse on the video panel;
    • reset the zoom factor by double right-clicking on the video panel;
    • duplicate the video panel to focus on specific objects (zooming is also available on duplicated panels) by left-clicking on the Duplicate button in the shortcut toolbar;
    • set the FPS (Frames Per Second) rate from the shortcut toolbar.



    Visualization of variance values across the field of view: main frame and duplicated panels

    Algorithms

    QuickViz allows to run many algorithms on a set of spectra, all along the spectral axis or on a restricted set of spectral selections over this set. Algorithms are sorted into two groups:
    • frame algorithms performing on image frames: mean, RGB composition, sub-cube extraction... As these algorithms often provide an output image, this one can be loaded and displayed in Aladin. Moreover, the resulting image might be automatically connected to the input plane in order to be used as a guide image, i.e. a spatial summary map of the datacube. In the extraction mode, each click on this image leads to the extraction of a spectrum from the input image. Frame algorithms are executed on cubes from which selected spectra have been extracted and NOT from currently displayed cubes in Aladin (that might differ);
    • spectrum algorithms applied on the spectrum values: mean, sum, convolution, variance-weighted mean (if variance values are available)... Output results of these algorithms are mainly spectra that are added to the spectrum stack.


    Frame algorithms



    A specific visualization mode, using 3 selections for a color composition average image

    Multiview

    You can easily add (resp. remove) spectrum panels to (resp. from) QuickViz thanks to the Panels submenu located in the View menu. This feature allows you to compare spectra without having to superimpose them. In multiview mode, all processings related to panels (like algorithm execution, spectrum extraction, unit change...) will be applied on the selected one. A panel is selected whenever you interact with it and then becomes surrounded by a light blue border. Keep in mind that spectra can be shared between panels.


    Multiview mode
    Another useful feature is to link panels together. The spectral cursor and the zoom window will be synchronized together and displayed at the same time on linked panels. Thus, you can focus on an interesting area in a first panel while keeping an eye on the whole spectrum in a second panel. The Link submenu is available in the Panels submenu.


    Linking panels

    Shape extraction (still in beta version)

    Aladin provides a way for the drawing of shapes on cube's frames. QuickViz uses this feature by providing a shape extraction that consists in computing the averaged spectrum (without taking variances into account for now even if they are available) over the shape. For the time being, QuickViz can only deal with circular areas.
    To define a circular selection, just left-click on the tag button in Aladin's toolbar and draw a circle over a frame by left-clicking and dragging the mouse. When you release the mouse button, the averaged spectrum will be computed and displayed in QuickViz. Moreover, you can resize and move the circular shape over the frame, leading to the extraction of a new averaged spectrum whenever you have finished to modify the size or the location of the circle. At last, for now, the averaged spectrum is not yet updated while resizing or moving the shape and a new averaged spectrum is systematically created.


    Circle extraction

    Bibliography

    • M. Petremand, L. Michel, M. Louys: "Visualization and logical binding of hyperspectral data using QuickViz and Saada" - Proceedings of Astronomical Data Analysis Software and Systems XX (ADASS) conference, Boston, ASP Conference Series, Nov. 2010 - pdf
    • M. Petremand, C. Collet, A. Jalobeanu, V. Mazet, F. Salzenstein, M. Louys: "New bayesian fusion scheme and visualization tool for astronomical hyperspectral data cubes" - Astronomical Data Analysis VI (ADA), Monastir, Tunisia, May 2010 - pdf - poster

    Links

    Download

    Current Release - QuickViz v1.50 (10/01/2012)

    • requires Aladin v7.000 (please update Aladin if needed: update link)
    • Interaction with SPLAT available through SAMP: right-click on a stacked spectrum and select Send to SPLAT to send it to SPLAT via SAMP. This option is only available if SPLAT is running

    Previous Releases

    • See change logs for details : links

    Current Limitations

    For the time being, QuickViz is still under development. Several features are not completely available and require additional developments to be fully finalized. The current proposed version of QuickViz has the following limitations:
    • QuickViz is fully supported by Aladin v7.000 but newer versions might not be compatible. Make sure using Aladin v7.000 for the best user experience;
    • for now, adding your own algorithms or visualization modes is not yet allowed (but this feature is already implemented);
    • QuickViz has not been fully tested on Mac OS. You may experience little bugs or defects while using QuickViz on this operating system.

    Download & Installation

    • Download and install (or update) the latest version of Java: Java SE Runtime Environment (JRE) (Java 5 is required) ;
    • download Aladin v7.000: Aladin ;
    • the execution of Aladin is realized as follows:
      under Linux/Windows/MacOS, enter the following command in the command line (or create a shortcut on your desktop): java -Xmx1024m -jar AladinBeta.jar where 1024 stands for the amount of RAM allocated to Aladin (in Mo). For a 32-bit architecture, the maximum amount of allocable memory is set to almost 1.6Mo. With a 64-bit architecture, this threshold becomes obsolete. For instance, 1024Mo are sufficient to load 4 cubes of 1.2Go in Aladin ;
    • download the latest release of QuickViz: QuickViz ;
    • copy the QuickViz plugin file into the plugin directory of Aladin. To know the Aladin plugin directory on your computer, just run Aladin and open the Plugin manager (menu Tools - Plugins - Plugin manager) and look for the Plugin directory text area.
      By default, this directory is set to $USERDIR$/.aladin/Plugins where $USERDIR$ stands for the user directory depending on your operating system ;
    • after restarting Aladin, launch QuickViz using the Aladin menu Tools - Plugins ;
    • Warning: if your data cube does not define any astrometric calibration, you have to create a default calibration (menu Image - Astrometric calibration in Aladin) by left-clicking on the Create button, otherwise you won't be able to extract spectra from it ;
    • QuickViz can be directly run from the command line together with Aladin with the following command: java -Xmx1024m -jar AladinBeta.jar -script="quickviz"

    Questionnaire

    Please, fill in our questionnaire and help us to improve QuickViz: link

    Uncertainty representation

    Uncertainties are a significant and valuable output of the fusion algorithm and are available at each location in the reconstructed cube. In addition, variances induced by the acquisition process are also provided for each observation session. The visualization of the behavior of uncertainties along spatial and spectral axes is essential as it may guide further investigations towards image areas or pixels having low uncertainties (e.g. for astrometry and photometry computations). Both coarse and fine views are interesting: a general overview allows to quickly localize high varying regions whereas pixel by pixel representation offers high resolution interpretation. The joint visualization variance/intensity data has been investigated along the spectral axis thanks to visualization modes available in QuickViz but the simultaneous spatial representation remains unsolved and problematic. Indeed, straightforward static visualizations such as colored or transparent compositions fail and instead tend to entangle both information. We have then selected two approaches based on a dynamical viewing process which is the most striking to the human eye.

    The first strategy consists in using the third dimension z so that pixels (represented by 3D objects like spheres or cubes) oscillate more or less quickly around the (x,y) plane according to their variance: the highest the variance, the quickest the oscillation. The number of video frames to end a cycle is thus inversely proportional to the variance value. Such a 3D movie is relevant to instantaneously visualize high variance areas in spectral bands but requires the choice of an appropriate projection angle for an efficient perspective view. In addition, when the image size increases, it becomes more difficult to clearly distinguish swinging pixels. Thus, this representation should be preferred for limited regions or for the visualization of covariance values around a neighborhood. Two video examples obtained on simulated data (for testing purpose only) are available:
    • the first one (link) shows a 30x30 2D frame where variances of middle pixels in the image have been set to higher values than other ones. On this video, the eye is directly focused by this high variance region ;
    • the second one (link) shows a 100x100 2D frame with the same high variance region on the top left corner of the image. The more the size of the image is, the more the uncertainy representation becomes difficult because of 3D projection ;

    In the second visualization, a map is created where the intensity of the pixel varies with the corresponding variance measurement. Thus, high variance positions will change their intensity quicker than low variance ones. Contrary to the first strategy, oscillation amplitudes are not spatially constant but are taken in the interval [Yp - x*sigmap;Yp + x*sigmap] where Yp is a pixel intensity with its associated standard deviation sigmap and x is a real positive number. Compared to the 3D method, this one is particularly well-suited for large images because an intensity variation is less confusing than a height one and more attractive to the human eye, i.e. one can easily identify general trends in the image as illustrated in the following movie where variances of the top left corner of the image have been set to high values (link).

    The evaluation and validation of both methods with the help of the astronomical community is still in progress.

    Webmaster (C. Collet), (M.Louys)
    © IPSEO Group 2005-2016 | WebSite Info & Credits
    Last update: Dec 8, 2016
    IPSEO
    Home
    Research projects
    Highlights-Results
    Softwares

    People
    Collaborations

    Local Admin