Visual Data Mining for Exploration of EO Images Archives
Teleaga, Delia1; Buican, Manole1; Cucu-Dumitrescu, Catalin1; Serban, Florin1; Datcu, Mihai2
1Advanced Studies and Research Center, ROMANIA; 2DLR, GERMANY

The article presents a prototype of a Visual Data Mining (VDM) tool the so-called DataVis3D tool - aiming at interactively and efficient browsing and understanding of the structure of large data sets of EO imaging products. One of its main applications will be in EO Payload Data Ground Segment (PDGS) of the German satellite mission TerraSAR-X. However, the tool could operate with any type of EO imaging products, such as Sentinel 1 and 2 missions and other similar missions (Spot, Pleiades, Landsat, etc.).

The volume of EO archives is very large today and is rapidly increasing. It is impossible for specialists to manually process all the information. Also, the quality of direct handwork is still superior to automatic/semiautomatic processes in data mining. For example, human decision in segmentation or classification issues is more accurate and even faster than the machine decision. Understanding and using the semantics of an image or set of images is a very time consuming feature for software. In this context, combining the accuracy of human decision and the force of computer vision, the result would be synergetic.

Visual Data Mining (VDM) is an emerging discipline, partially synonymous with Visual Analytics (VA) and sharing some terminology and methods with Computer Vision (CV). Through VDM the specialists can perform better reasoning and make informed decisions with respect to large, complex, even dynamic, data and situations. The basic nature of the data that VDM deals with is usually visual (images of all sort, satellite scenes, radar scenes, magnetic resonance images, time series of images, photos, movies etc.), but data can also be strictly numerical or literal (natural language texts). Visualizing the data must not be a simple graphical representation, but a technique to increase the informational entropy of a message [Chen 2008, Chen 2010]. Another impulse for developing VDM techniques is the ever growing volume of image data bases. The specialists in remote sensing and image processing need tools, preferably automatic ones, for dealing with terabytes of images that need to be organized, understood and used [Datcu 2003, Datcu 2005].

The DataVis3D tool is based on a hierarchical information representation. In a first step the EO images are partitioned in patches with the size adapted to capture meaningful contextual information. Experimentally the optimal size was found to be of ca. 200 pixels, independently on the image resolution. Patches are indexed by extracting a relevant parameter set and this is represented in a descriptor. Generally the descriptors capture different properties, e.g. radiometric, phase, or geometry, thus will not describe the same information content. A library of specific descriptors for multispectral and SAR is used: it comprises Gabor features, Multispectral descriptors, and SIFT. Thus, the whole archive is represented in the n-dimensional space of the extracted features, each patch being a point.

For visualization, a projection of the n-dimensional space in the 3-dimensional one is performed. Two methods are used: Laplacian eigenmaps and Stochastic Neighbor Embedding.

Laplacian eigenmaps is a nonlinear data reduction algorithm based on spectral techniques. The main idea here is to represent the data as a low dimensional curve in a high dimensional space. The representation of this curve is done with a graph which is built in such a way that data point are the nodes of this graph and the edges are based on K- nearest neighbor technique.

Stochastic Neighbor Embedding is a probabilistic approach that aims to preserve the neighborhood identities in the low dimensional representation of data. Here, the Gaussian distribution is used for each data point i to compute the probability of having data point j as its neighbor. The same construction of neighborhood is also done in a low-dimensional space and the goal is to match these two constructions as much as possible.

DataVis3D is implemented as a software prototype able to exhibit graphical representation; it will be not the usual geographic overlapping of images, but a 3D spatial grouping, by means of their visual resemblance. Every image or tile is reduced to a small but still comprehensive icon, which will be used only for formal representation in the final projective map. In Figure 1, a screenshot of the application shows its main features: zooming and rotation navigation in the 3d space of extracted features, together with a selection tool based on a user-defined sphere (left main screen, Figure 1); the selected tile may be projected and visualized on Google Earth (right screen, Figure 1) and, vice versa, by selecting an area in Google Earth, the corresponding tiles may be visualized in the features space. Figure 1 Screenshot of the DataVis3D tool, running on a set of about 6000 tiles obtained by dividing a Landsat scene into 100 x 100 pieces. On the left main screen, their representation in the 3d space of the extracted features is displayed. In this space, zooming and rotation navigation is provided, together with a selection tool based on a user-defined sphere. The selected tiles may be projected and visualized on Google Earth.


  • C. Chen - An Information-Theoretic View of Visual Analytics, IEEE Comp. Graph. Appl. 28, issue 1, 18, 2008.
  • M. Chen, H. Jäenicke - An Information-theoretic Framework for Visualization, IEEE Trans. Viz. Comp. Graph. 16, 1206, 2010.
  • M. Datcu, H. Daschiel, A. Pelizzari, M. Quartulli, A. Galoppo, A. Colapicchioni, M. Pastori, K. Seidel, P.G. Marchetti, S. D'Elia - Information mining in remote sensing image archives - system concepts, IEEE Trans. Geo. Rem. Sens. 41, 2923, 2003.
  • M. Datcu, K. Seidel - Human-centered concepts for exploration and understanding of Earth observation images, IEEE Trans. Geo. Rem. Sens. 43, 601, 2005.
  • M. Buican, C. Cucu-Dumitrescu, F. Serban, D. Teleaga - A 3D Visual Data Mining Application, Image Information Mining Conference: Knowledge Discovery from Earth Observation Data, DLR, Oberpfaffenhofen, Germany, 2012.