A rainbow for everyone

Matteo — Fri, 24 Feb 2012 01:36:19 +0000

Traffic lights for everyone

Stephen Westland of Colour chat recently posted about a clever new LED traffic light developed in Japan. Here’s my tweet with the link to Westland’s original blog post:

Traffic lights for everyone: colourware.wordpress.com/2012/02/09/tra…
—
Matteo Niccoli (@My_Carta) February 20, 2012

I really like the idea of making a traffic light that works for everyone: for people with full color vision and people with color vision deficiencies. In fact, I think we should do the same with our color palettes. Why do I say that?

A rainbow for everyone

Take a look at Figure 1 below. This is a map of the Bouguer Gravity (terrain corrected Bouguer Gravity to be precise) in Southern Tuscany, colored using a rainbow palette. I intentianally left out the colorbar. For a moment ignore the sharp gradient changes at the yellow and cyan color (that is one of the topics of my upcoming series “The rainbow is dead…long live the rainbow!”). Can you tell which color is representing high values an which low? If you have used a mnemonic like ROY B GIV and can tell that highs are towards the Southwest and lows towards the Northeast, then you are right and you also have full color vision, just like me. Great, because this post is exacly for us, the “normals”.

Figure 1

Take now a look at Figure 2: this is exacly the same map, but I run a simulation to show us normals how it would be seen by a person with Deuteranopia, a form of color vision deficiency (I run the simulation using Vischeck plugin for ImageJ, see the related contents section). Can you tell now which is high and which is low? I certainly can, but I’d be wrong. Using topography as a methafor, what I see is a nice yellow mountain ridge, from which the surface slopes down to shades of brown, and then white, on either side. There’s a nice big blue lake in the the Northeast but as far as I’m concerned there’s also a low in the South.

Figure 2

In my upcoming seires “The rainbow is dead…long live the rainbow!” i will tackle these topics in depth and I will also propose some new rainbow-like color palettes that are better for everyone, no matter how many and how well you see colors. Stay tuned!

What Words Can’t Convey: Illustrations Promote Scientific Understanding

Matteo — Wed, 15 Feb 2012 21:33:24 +0000

Reblogged from Tomorrow's Ecology:

Humankind has long used illustrations to represent complex ideas and concepts. When words just can’t quite convey our meaning images often becomes necessary. From cave paintings to modern-day microscopy our ability to understand and influence our environment has relied on our evolving ability to create read images. To quote the National Science Foundation (NSF): Some of science’s most powerful statements are not made in words. From the diagrams of DaVinci to Rosalind Franklin’s X-rays, visualization of …

This is a fantastic initiative.

http://www.nsf.gov/news/special_reports/scivis/challenge.jsp

Filed under: Visualization

Visualization tips for geoscientists – Matlab

Matteo — Fri, 10 Feb 2012 02:15:45 +0000

Introduction

In my last post I described how to create a powerful, nondirectional shading for a geophysical surface using the slope of the data to assign the shading intensity (i.e. areas of greater slope are assigned darker shading). Today I will show hot to create a similar effect in Matlab.

Since the data set I use is from my unpublished thesis in Geology, I am not able to share it, and you will have to use your own data, but the Matlab code is simply adapted. The code snippets below assume you have a geophysical surface already imported in the workspace and stored in a variable called “data”, as well as the derivative in a variable called “data_slope”.

Method 1 – with a slope mask and transparency

Some time ago I read this interesting Image Processing blog post by Steve Eddins at Mathworks on overlaying images using transparency. I encourage readers to take a look at this and other posts by Steve, he’s great! That particular blog post gave me the idea to use transparency and the slope to create my favorite shading in Matlab.

In addition to the code below you will need normalise.m from Peter Kovesi‘s website, and to import the color palette cube1.

%% alpha transparency code snippet
black = cat(3, zeros(size(data)), zeros(size(data)), ...
    zeros(size(data)));             % make a truecolor all-black image
gray=black+0.2;                     % make a truecolor all-gray image
alphaI=normalise(data_slope);       % create transparency weight matrix
                                    % using data_slope

imagesc(data);colormap(cube1);      % display data
hold on
h = imagesc(gray);                  % overlay gray image on data
hold off
set(h, 'AlphaData', alphaI);        % set transparency of gray layer using
axis equal;                         % weight matrix
axis tight;
axis off;

And here is the result in Figure 1 below – not bad!

Figure 1. Shaded using transparency

Method 2 – using Shaded Pseudo Color

You will need shadedpcolor.m, a function by Jody Klymak available on the Matlab File Exchange, and again normalise.m and cube1.

%% shadedpcolor code snippet
 figure;
 shadedpcolor(xe,ye,data,(1-normalise(data_slope)),[-5.9834 2.9969], ...
      [0 1],0.55,cube1,0);
 axis equal; axis off; axis tight
 shadedcolorbar([-5.9834 2.9969],0.55,cube1);

The result is in Figure 2. This really looks good. I favor this over the code above it now all the time because it allows greater flexibility (for example shading brightness adjustment), and because it creates a shaded colorbar for you.

Figure 2. Shaded using shadedpcolor

Method 3: 3D

I am trying to write some code to use transparency to apply he shading with the surf command. Another option to generate a real 3D effect much like the one shown in the last post in Surfer I use imageshiny.m, a function by Gordon Cooper available on the Matlab File Exchange. This function will create both a 2D map like the ones above and a 3D map. The 3D map for the gravity data is shown below in Figure 3.

Notice that for imageshiny to work properly you will have to replace all instances of imshow with imagesc in the code if you do not have the Image Processing Toolbox.

Figure 3. Plotted and shaded using imageshiny

It is actually very interesting to look at the ingenious solution the author of imageshiny came up with to assign the shading. When you run imageshiny, each pixel in the surface is assigned simultaneously color and shading. This is done by generating a new matrix of HSV triplets (refer to wikipedia for a review of HSV and HSL) using the data directly: the hue H is assigned with the data (normalized to the interval 0.5 to 1 so as to translate in the color range green to red); saturation S and value V are both assigned using the slope (imageshiny calculates it for you on the fly). The new matrix HSV is then converted to a matrix of RGB triplets, and the latter is used to color the surface when using the surf command. Here below is the relevant code, modified to facilitate reading (data* is the normalized data, slope* is the calculated slope, to distinguish from the variables in my code snippets above):

HSV(:,:,1)=1-data*; HSV(:,:,2)=1-slope*; HSV(:,:,3)=slope*;
RGB=hsv2rgb(HSV);
figure; clf;
surf(data,RGB);axis off; axis tight; shading flat; view(5,85);

I am currently experimenting to see if I can make this function work with different hue combinations.

Contours

I am still working at figuring out the right approach and write my own code to drape contours over the surface in 3D. This is actually possible using the Matlab function contour3m, but you need the Mapping Toolbox.

Filed under: Color, Geophysics, Image processing, Matlab, ImageJ, et cetera, Tutorial, Visualization Tagged: data, display, gravity, horizontal derivative, Matlab, pseudo-3D, residual anomaly, slope, visualization

Visualization tips for geoscientists: Surfer

Matteo — Thu, 26 Jan 2012 02:03:56 +0000

I have been thinking for a while about writing on visualization of geophysical data. I finally got to it, and I am now pleased to show you a technique I use often. This tutorial has shaped up into 2 independent posts: in the first post I will show how to implement the technique with Surfer, in the second one with Matlab (you will need access to a license of Surfer 8.08 or later, and Matlab 2007a or later to replicate the work done in the tutorial).

I will illustrate the technique using gravity data since it is the data I developed it for. In an upcoming series of gravity exploration tutorials I will discuss in depth the acquisition, processing, enhancement, and interpretation of gravity data (see [1] and [4]). For now, suffice it to say that gravity prospecting is useful in areas where rocks with different density are laterally in contact, either stratigraphic or tectonic, producing a measurable local variation of the gravitational field. This was the case for the study area (in the Monti Romani of Southern Tuscany) from my thesis in Geology at the University of Rome [2].

In this part of the Apennine belt, a Paleozoic metamorphic basement (density ~2.7 g/cm³) is overlain by a thick sequence of clastic near-shore units of the Triassic-Oligocene Tuscany Nappe (density ~2.3 g/cm³). The Tuscan Nappe is in turn covered by the Cretaceous-Eocene flish units of the Liguride Complex (density ~2.1 g/cm³).

During the deformation of the Apennines, NE verging compressive thrusts caused doubling of the basement. The tectonic setting was later complicated by tensional block faulting with formation of horst-graben structures generally extend along NW-SE and N-S trends which were further disrupted by later and still active NE-SW normal faulting (see [2], and reference therein, for example [3]).

This complex tectonic history placed the basement in lateral contact with the less dense rocks of the younger formations and this is reflected in the residual anomaly map [4] of Figure 1. Roughly speaking, there is a high in the SE quadrant of ~3.0 mgal corresponding to the location of the largest basement outcrop, an NW-SE elongated high of ~0.5 mgal in the centre bound by lows on both the SW and NE (~-6.0 and ~-5.0 mgal, respectively), and finally a local high in the N.W. quadrant of ~-0.5 mGal. From this we can infer that in this area can infer that the systems of normal faults caused differential sinking of the top of basement in different blocks leaving an isolated high in the middle, which is consistent with the described tectonic history [2]. Notice that grayscale representation is smoothly varying, reflecting (and honoring) the structure inherent in the data. It does not allow good visual discrimination and comparison of differences, but from the interpretation standpoint I recommend to always start out with it: once a first impression is formed it is difficult to go back. There is time later to start changing the dispaly.

Figure 1 – Grayscale residual anomalies in milligals. This version of the map was generate using the IMAGE MAP option in Surfer.

OK, now that we formed a first impression, what can we try to improve on this display? The first thing we can do to increase the perceptual contrast is to add color, as I have done in Figure 2. This is an improvement, now we are able to appreciate smaller changes, quickly assess differences, or conversely identify areas of similar anomaly. Adding the third dimension and perspective is a further improvement, as seen in figure 3. But there’s still something missing. Even though we’ve added color, relief, and perspective, the map looks a bit “flat”.

Figure 2 – Colored residual anomalies in milligals. This version of the map was generate using the IMAGE MAP option in Surfer.

Figure 3 – Colored 3D residual anomaly map in milligals. This version of the map was generate using the SURFACE MAP option in Surfer.

Adding contours is a good option to further bring out details in the data, and I like the flexibility of contours in Surfer. For example, for Figure 4 I assigned (in Contour Properties, Levels tab) a dashed line style to negative residual contours, and a solid line style to positive residual contours, with a thicker line for the zero contour. This can be done by modifying the style for each level individually, or by creating two separate contours, one for the positive data, one for the negative data, which is handy when several contour levels are present. The one drawback of using contours this way is that it is redundant. We used 3 weapons – color, relief, and contours – to dispaly one dataset, and to characterize just one property, the shape of gravity anomaly. In geoscience it is often necessary, and desireable to show multiple (relevant) datasets in one view, so this is a bit of a waste. I would rather spare the contours, for example, to overlay and compare anomalous concentrations in gold pathfinder elements on this gravity anomaly map (one of the objectives of the study, being the Monti Romani an area of active gold exploration).

Figure 4 – Colored 3D residual anomaly map in milligals. Contours were added with the the CONTOUR MAP option in Surfer. Figure 5 – Colored 3D residual anomaly map in milligals with lighting (3D Surface Properties menu). Illumination is generated by a point source with -135 deg azimuth and 60 deg elevation, plus an additional 80% gray ambient light, a 30% gray diffuse light, and a 10% gray specular light.

The alternative to contours is the use of illumination, or lighting, which I used in Figure 5. Lighting is doing a really good job: now we can recognize there is a high frequency texture in the data and we see some features both in the highs and lows. But there’s a catch: we are now introducing perceptual artifacts, in the form of bright white highlight, which is obscuring some of the details where the surface is orthogonal to the point source light.

There is a way to illuminate the surface without introducing artifact – and that is really wanted to show you with this tutorial – which is to use a derivative of the data to assign the shading intensity (areas of greater gradient were assigned darker shading) [5]. In this case I choose the terrain slope, which is the slope in the direction of steepest gradient at any point in the data (calculated in a running window). The result is a very powerful shading. Here is how you can do it in Surfer:

1) CREATE TERRAIN SLOPE GRID (let’s call this A): go to GRID > CALCULUS > TERRAIN SLOPE

Result is shown in Figure 6 below:

Figure 6 – Terrain slope of residual anomaly. Black for low gradients, white for high gradients. Displayed using IMAGE MAP option.

2) CREATE COMPLEMENT OF TERRAIN SLOPE AND NORMALIZE TO [1 0] RANGE (to assign darker shading to areas of greater slope. This is done with 3 operations:

i) GRID > MATH> B=A – min(A)

where min(A) is the minimum value, which you can read off the grid info (for example you would double click on the map above to open the Map Properties and there’s an info button next to the Input File field) .

ii) GRID > MATH> C=B /max(B)

iii) GRID > MATH> D= 1-C

Result is shown in Figure 7 below. This looks really good, see how now the data seems almost 3D? It would work very well just as it is. However, I do like color, so I added it back in Figure 8. This is done by draping the grayscale terrain slope complement IMAGE MAP as an overlay over the colored residual anomaly SURFACE MAP, and setting the Color Modulation to BLEND in the 3D Surface Properties in the Overlay tab. I really do like this display in Figure 8, I think it is terrific. Let me know if you like it best too.

Finally, in Figure 9, I added a contour of the anomaly in the Gold Pathfiners, to reiterate the point I made above that contours are best spared for a second dataset.

In my next post I will show you how to do all of the above programmatically in Matlab (and share the code). Meanwhile, comments, suggestions, requests are welcome. Have fun mapping and visualizing!

Figure 7 – Complement of the terrain slope. White for low gradients, black for high gradients. Displayed using IMAGE MAP option.

Figure 8 – Complement of the terrain slope with color added back.

Figure 9 – Complement of the terrain slope with color added back and contour overlay of gold pathfinders in stream sediments.

GOODIES

Did you lie the colormap? In future series on perceptually balanced colormaps I will tell you how I created it. For now, if you’d like to try it on your data you can download it here:

cube1 - generic format with RGB triplets;

Cube1_Surfer - this is preformatted for Surfer with 100 RGB triplets and header line. Dowload the .doc file, open and save as plain text, then change extension to .clr;

Cube1_Surfer_inverse – the ability to flip color palette is not implemented in Surfer (at least not in version 8) so I am including the flipped version of above. Again, dowload the .doc file, open and save as plain text, then change extension to .clr.

ACKNOWLEDGEMENTS

I would like to thank Michele di Filippo at the Department of Earth Science, University of Rome La Sapienza, to whom I owe a great deal. Michele, my first mentor and a friend, taught me everything I know about the planning and implementation of a geophysical field campaign. In the process I also learned from him a great deal about geology, mapping, Surfer, and problem solving. Michele will make a contribution to the gravity exploration series.

NOTES

[1] If you would like to learn more about gravity prospecting please check these excellent course notes.

[2] Niccoli, M., 2000: Gravity, magnetic, and geologic exploration in the Monti Romani of Southern Tuscany, unpublished field and research thesis, Department of Earth Science, University of Rome La Sapienza.

[3] Moretti A., Meletti C., Ottria G. (1990) – Studio stratigrafico e strutturale dei Monti Romani (GR-VT) – 1: dal Paleozoico all’Orogenesi Alpidica. Boll. Soc. Geol. It., 109, 557-581. In Italian.

[4] Typically reduction of the raw data is necessary before any interpretation can be attempted. The result of this process of reduction is a Bouguer anomaly map, which is conceptually equivalent to what we would measure if we stripped away everything above sea level, therefore observing the distribution of rock densities below a regular surface. It is standard practice to also detrend the Bouguer anomaly to separate the influence of basin or crustal scale effects, from local effects, as either one or the other is often the target of the survey. The result of this procedure is typically called Residuals anomaly and often shows subtler details that were not apparent due to the regional gradients. Reduction to rsiduals makes it easier to qualitatively separate mass excesses from mass deficits. For a more detailed review of gravity exploration method check agai nthe notes in [1] and refer to this article on the CSEG Recorder and reference therein.

[5] Speaking in general, 3D maps without lighting often have a flat appearance, which is why light sources are added. The traditional choice is to use single or multiple directional light sources, but the result is that only linear features orthogonal to those orientations will be highlighted. This is useful when interpreting for lineaments or faults (when present), but not in all circumstances, and requires a lot of experimenting. in other cases, like this one , directional lighting introduces a bright highlight, which obscures some detail. A more generalist, and in my view more effective alternative, is to use information derived from the data itself for the shading. One way to do that is to use a high pass filtered version of the data. i will show you how to do that in matlab in the next tutorial. Another solution, which I favored in this example, is to use a first derivative of the data.

Filed under: Cartography and mapping, Color, Geophysics, Tutorial, Visualization Tagged: 3D, Bouguer, color, colormap, data visualization, display, first derivative, Golden Software, gravity, residual anomaly, shadowgram, Surfer, terrain slope, visualization

Compare lists from text files using Matlab – an application for resource exploration

Matteo — Fri, 13 Jan 2012 01:58:41 +0000

INTRODUCTION

With today’s post I would like to share a Matlab script I used often to compare lists of ID numbers stored in separate text files. The ID numbers can be of anything: oil and gas wells, mining diamond drill hole locations, gravity or resistivity measurement stations, outcrop locations, you name it. And the script will handle any combination of ASCII characters (numbers, letters, etcetera).

I included below here some test files for you to try with the code, which is in the next section. Please refer to the comment section in the code for usage and file description. Have fun, and if you try it on your lists, let me know how it works for you.

TEST FILES

These files are in doc format; they need to be downloaded and saved as plain txt files. Test1 and Test2 contain 2 short lists of (fake) Canadian Oil and Gas wells; the former is the result of a search using a criterion (wells inside a polygonal area), the latter is a list of wells that have digital wireline logs. Test3 and Test4 are short lists of (fake) diamond drill holes.

*** Notice that script is setup to compare list in Test2.txt against list in Test1.txt and not the other way around. If using this on your own lists, you will have to decide in advance which subset of wells you are interested in.

THE SCRIPT

Here is the code:

% Script to compare lists of wells from 2 text files
% (wells are identified by a well ID string)
%
%%%% INPUT
% must be text files with 1-line header
%
% Sample test files provided:
% File 1 -  test1.txt - list of wells satisfying a criteria
%           (for example wells with digital wireline logs)
%
% File 2 -  test2.txt - list of all wells within an area
%           (for example returned using polygon search)
%
% These FILES contain (dummy) oil and gas wells from Canada
% i.e. wells with a 16-digit identifier of format 1XX021611428W600
% Header line is simply  'UWI' (Unique Well Identifier)
% For example:
% UWI
% 1XX021611428W600
% 1XX023511428W600
% ...
% ...
% To use with other files either
% 1 - actual name of the files must be changed to test1.txt and test2.txt
% or
% 2 - names in line 47 and 51 must be changed to 'yourfile1.txt' and
% 'yourfile2.txt', respectively
%
%%%% OUTPUT
% script extracts wells from list 2 that satisfy criteria and saves to
% File 3 -  out.xls
%
%%%%
% To use different header line and well identifier formats n and h must be
% changed accordingly
% For example suppose we had a file of diamond drill holes from a mining
% exploration permit, containing Diamond Drill Hole IDs in this format:
% DDH-ID (header line)
% DDH-23
% DDH-57
% ...
% ...
% we would need to change both h and n to 6
% try it with test3.txt and test4.txt
%
%% import data
curves=textread('test1.txt','%c');
% open file - this is list of wells with digital wireline logs (curves)
% notice script reads each character in the file as a row
%
inside=textread('test2.txt','%c');
% open file - this is list of wells inside rectangle or polygon
% notice script reads each character in the file as a row
%
%% cleanup, reshape variables, convert to cell array
n=16;       % number of characters in well identifier (for example, 16)
h=3;        % number of characters in header line (for example, 3)
[r,c]=size(inside); l=(r-h)/n;          % remove characters in the header
inside1=reshape(inside((h+1):r),n,l)';  % line and reshape
cellinside=cellstr(inside1);            % create cell array from char array
%
[t,s]=size(curves); f=(t-h)/n;          % remove characters in the header
curves1=reshape(curves((h+1):t),n,f)';  % line and reshape
cellcurves=cellstr(curves1);            % create cell array from char array
%
%% compare lists
for k=1:l
    for j=1:f
TF(k,j) = strcmpi(cellinside(k),cellcurves(j));
    end
end
% this is the kernel of the script:
% compare list of wells with logs (curves) vs. list of wells inside area
%
% From Matlab help: the command TF = strcmpi(s1,s2)
% compares the strings s1 and s2 and returns logical 1 (true) if they are
% the same (ignoring case), and returns logical 0 (false) otherwise
%
%% extract matching UWI and save output to Excel file
TTFF=max(int8(TF)')';
vi=logical(TTFF);
exp=cellinside(vi);
xlswrite('out.xls',exp);
%
%% optional house cleaning
clear inside curves n h r c l t s f TF TTFF j k vi

You could achieve the same result in Excel using a formula like this (applied to a single spreadsheet with lists in 2 columns, A and B):

 =IF(ISERROR(VLOOKUP(B2,$A$2:$A$999,1,FALSE)),0,VLOOKUP(B2,$A$2:$A$999,1,FALSE))

However I like Excel a lot less because the automation (with Macros, visual basic) is more cumbersome, and because it is more prone to errors that are less easy to QC (contrary to Matlab, which will give you error messages and has a straightforward debugger). When Excel is really necessary is in those instances in which in addition to finding a subset of wells using the 2 lists you want to extract information from other columns, such as coordinates, ground elevation, date drilled, operator, common well name, etcetera; then VLOOKUP is the right tool. I will show you how to do just that in a future post.

AN AFTERTHOUGHT

Since a picture is worth a thousand words I thought I’d add a pictorial example. Below is a geological map from a mining exploration area in Tuscany, Italy, where high grade asbestos was mined in the past, and gold exploration is under way. I added to this map black square symbols to indicate locations where stream sediments were collected during a geochemical prospecting campaign in the ’80s (samples derived from the sediments are analyzed for concentrations in gold pathfinder elements). Suppose we had 2 lists: List1 contains all locations with anomalous concentrations in 1 or more of the pathfinder elements (I marked these locations with red diamond symbols). List2 contains all locations (I marked those with green cross symbols) with outcropping fault contact between the metamorphic basement and the more recent sedimentary rock formations (faults bounding horst structures formed in extensional settings are believed to act as conduits for gold bearing fluids). If we run the Matlab script using List1 and List2 it would return a list of locations with fault contact AND anomalous pathfinder concentrations. Those are indicated by arrows on the map - the location with just the green cross and those with just the red diamond would not be included.

Sources (through the Regione Toscana): for map, University of Siena, Department of Earth Science, Geological Map of the Regione Toscana scale 1:10,000, sheet 343070; for Stream Sediment data: ENI/RIMIN, unpublished Industry Report, 1989

Filed under: Cartography and mapping, Geophysics, Matlab, ImageJ, et cetera, Tutorial Tagged: cell array, cellinside, cellstr, character array, compare lists, compare text files, digital curves, Excel, gold pathfinders, IF, ISERROR, logical, macro, Matlab code, Matlab script, mining, oil and gas, reshape, search criteria, strcmpi, stream sediment, string, textread, UWI, VLOOKUP, well list, xlswrite

Digital cartography picks

Matteo — Mon, 09 Jan 2012 22:42:09 +0000

My top pick is NASA’s new integration of the Apollo Zone Digital data. It was done at Ames Research Centre thanks to a newly developed software system for orbital imagery. The software allows fully automated image mosaicking and terrain modeling of data taken from different positions, with different exposure and resolution, and even selects best image when multiple coverage exists. You can read about this exiting new development in the article Powerful Pixels: Mapping the “Apollo Zone” which has links to the open source software libraries Ames Stereo Pipeline, Neo-Geography Toolkit and NASA Vision Workbench. You can click here to download a kml file for viewing the image mosaic and digital elevation model in Google Earth. I tried it out and it looks great. Check these screen captures below:

A more advanced interface – the Lunar Mapper – is available online at the Lunar Mapping and Modeling Project site (in the tools menu). I tried it last night and I was very impressed. It has an intuitive interface, easy access to layer with data from the various Apollo missions. Here’s an example:

You can switch to polar view with a single click. Below here I am showing the average temperature. It is great to have the option to use opacity (in this example I applied an opacity of 0.4) but on the down side I could not find a colorbar to turn on.

A great feature is the crater search:

There’s a video demo available: Lunar Mapper LMMP Demo final.mov

==============================================================================================

My second pick is The Essential Geography of the United States of America from David Imus. Imus put together an innovative new map of the United States, which is really good news. Why is this a big deal? There’s an excellent article about it on Slate magazine so I point you to that for a full review .

What I think is worth mentioning – and I like a lot in this map is that Imus used (simplified) Relief Shading to render the terrain, without sacrificing any of the requirements of a commercial cartographical product (labels, boundaries, etcetera). Here’s a quick comparison I did of a Bathymetry Map of the Hawaiian archipelago from USGS (top, converted to grayscale) and the provided for Hawaii on Imus Geographics website (bottom, also converted to grayscale).

I would encourage you to take a look at Imus’ very instructive Booklet on advancing geographic literacy in classrooms, and to watch his YouTube interview.

==============================================================================================
Bronze medal goes to the paper Adaptive smoothing for noisy DEMs which was posted on GIS and Science blog in December. I think it would be very interesting to try the method on geophysical data (for example structure maps from reflection seismic), where often the noise level is spatially variable.

Filed under: Cartography and mapping, Geophysics, Image processing, Visualization Tagged: Apollo digital data, digital cartography, geography literacy, Google Earth, Hawaii, Imus Geographics, Lunar Mapper, Moon crater search, NASA, Relief shading, United States Geography, USGS

MyCarta – 2011 in review

Matteo — Sun, 01 Jan 2012 18:12:25 +0000

The WordPress.com stats helper monkeys prepared a 2011 annual report for this blog.

Here’s an excerpt:

A San Francisco cable car holds 60 people. This blog was viewed about 2,400 times in 2011 [NOTE: blog started on September 31st]. If it were a cable car, it would take about 40 trips to carry that many people.

Click here to see the complete report.

Filed under: Uncategorized

An example of Forensic Image Processing in ImageJ

Matteo — Thu, 15 Dec 2011 03:17:27 +0000

In a previous post I introduced ImageJ, a very powerful open source image processing software. ImageJ allows users to display, edit, analyze, process, and filter images, and its capabilities are greatly increased by hundreds of plugins.

In a future post I will be showing how to use the watershed transform in ImageJ for medical image analysis and advanced geoscience map interpretation.

Today I am posting a submission entry by guest Ron DeSpain, an image and signal analysis software developer, and a fellow member from the Image Processing Interest Group on Linkedin. Ron’s note is about Feature Detection for Fingerprint Matching in ImageJ. I was thrilled to receive this submission as I really have a soft spot for Forensic science. Additionally, it is a nice way to introduce skeletonization, which I will be using in a future series on automatic detection of lineaments in geophysical maps. So, thanks Ron!

Please check this page for reference on fingerprint terminology. And if you are interested in the topic and would like to start a discussion, or make a suggestion, please use the comment section below. You can also contact Ron directly at ron_despain@hotmail.com if you want to enquire about the code.

==========================================================================================

Initial Feature Detection Steps for Fingerprint Matching – by Ron DeSpain

A common fingerprint pre-processing method called the crossings algorithm is used to extract from a fingerprint features called minutiae. Minutiae are located at the end of fingerprint ridges and where the ridges split (bifurcations) as shown in Figure 1. Once detected, minutiae are correlated with a database of known fingerprint minutiae sets. This article discusses the very first step in detecting these minutiae in a fingerprint.

Figure 1 Types of Fingerprint Minutiae

This fingerprint is available in a free database of fingerprint images at http://bias.csr.unibo.it/fvc2000/download.asp

I got the idea for this convolution based minutiae extractor from http://www.szabist.edu.pk/NCET2004/Docs/Session%20VII%20Paper%20No%202%20(P%20141-146).pdf where a slightly different counting scheme is used to identify minutiae.

This algorithm depends on the fact that the end and bifurcation patterns have unique numbers of crossings in a 3×3 local region, as depicted in Figure 2. This means that by simply counting the crossings you could detect the minutiae.

Figure 2 Minutiae Patterns

The pseudocode for this algorithm is as follows:

Convert the image to binary, normalized to 0 to 1 range, floating point data
Skeletonize the image
Convolve the skeleton with the unit 3×3 matrix to count the crossings
Multiply the skeletonized image by the convolved image = Features Image
Threshold the Features image at 2 for ridge ends
Threshold the Features image at 4 for bifurcations

The following imageJ macro will identify minutiae using this simple pattern recognition technique. You can download and install ImageJ free from http://imagej.nih.gov/ij/download.html. Don’t forget to get the user’s manual and macro coding guide from this site if you want to modify my macro.

//Minutiae Detection Macro
open();
run("Duplicate...", "title=Skeleton");
starttime = getTime();
run("Make Binary");
run("Skeletonize");
run("32-bit");
run("Divide...", "value=255.000");
run("Enhance Contrast", "saturated=0 normalize");
run("Duplicate...", "title=Convolution");
run("Convolve...", "text1=[1 1 1\n1 1 1\n1 1 1\n] stack");
imageCalculator("Multiply create 32-bit", "Skeleton","Convolution");
endtime = getTime();
selectWindow("Result of Skeleton");
rename("Features");
run("Tile");
run("Threshold...");
print("Processing Time (ms) = "+(endtime - starttime));
setTool(11);
selectWindow("Features");
run("Sync Windows");

Copy this code to a text file (.txt), drop it into the ImageJ macros folder, install and run it in ImageJ using the image at the end of this article.

The output of the above macro is shown in Figure 3 below:

Figure 3 ImageJ Macro Output

Setting the threshold control to show pixels with a value of 2 in red highlights will show the ridge end detections as shown in Figure 4. Note that the noise in the image produces false detections, which have to be identified with further processing not addressed here.

Figure 4 Ridge End Detections

Bifurcations are similarly found by setting the threshold to 4 as shown in Figure 5:

Figure 5 Bifurcations Detected

There are two fingerprint processing macros on the Mathworks user community file exchange for Matlab users and free fingerprint verification SDK at http://www.neurotechnology.com/free-fingerprint-verification-sdk.html for those of you who would like to dig deeper into this subject.

You can copy and save the fingerprint image I used in this article directly from this document’s Figure 6 to get you started either via screen capture, or right-click the image download.

Figure 6 Original Image

Filed under: Image processing, Matlab, ImageJ, et cetera, Submission, Tutorial Tagged: bifurcation, convolution, feature detection, fingerprint, fingerprint matching, forensic science, forensics, image analysis, image processing, ImageJ, macro, minutiae, ridge, scheletonization

Image processing tips for geoscientists – 1

Matteo — Wed, 30 Nov 2011 04:14:47 +0000

Today I would like to show a way to quickly create a pseudo-3D display from this map:

Original image

The map is a screen capture of a meandering river near Galena, Alaska, taken in Google Earth. I love this image; it is one of my favorite maps for several reasons. First of all it is just plainly and simply a stunningly beautiful image. Secondly, and more practically, the meanders look not too dissimilar to what they would appear on a 3D seismic time slice displayed in grayscale density which is great because it is difficult to get good 3D seismic examples to work with. Finally, this is a good test image from the filtering standpoint as it has a number of linear and curved features of different sizes, scales, and orientation.

The method I will use to enhance the display is the shift and subtract operation illustrated in The Scientist and Engineer’s Guide to Digital Signal Processing along with other 3×3 edge modification methods. The idea is quite simple, and yet extremely effective – we convolve the input image with a filter like this one:

0  0  0
0  1  0
0  0 -1

which will create a shifted version of the image (at the -1 location) and subtract it from the original (at the 1). Quoting the author:” this processing produces the optical illusion that some objects are closer or farther away than the background, making a 3D or embossed effect. The brain interprets images as if the lighting is from above, the normal way the world presents itself. If the edges of an object are bright on the top and dark on the bottom, the object is perceived to be poking out from the background”.

This is the same result you would achieve using Emboss filter in Photoshop and other similar programs. The only time I saw embossing used on seismic data was a few years ago when I took a Seismic Geomorphology course taught by Henry Posamentier for the Canadian Society of Exploration Geophysicists (CSEG). So thank you for the idea, Henry.

Like with derivative maps and with illumination, the outcome of this operation is not unique, in the sense that with changing the direction (or azimuth) of the process, i.e. the position of the -1, different features will be enhanced, and some in fact will be obscured as a result of the subtraction. Let’s look at a few examples below.

Here is the shift and subtract to the South, using this kernel:

 0  0  0
 0  1  0
 0 -1  0

: shift and subtract South

It looks like a good result, with most linear and curved features are enhanced. It has however suppressed features oriented N-S. For example many of the channels in the NE corner, and the “l” in Google, which is all but gone. Notice also that the pseudo-3D effect is noticeable but not as pronounced as in the shift and subtract to the Southwest below. Look at the main river on the left for instance. The reason is that when applied along the diagonals, the copied image is shifted both horizontally and vertically, as opposed to just vertically or just horizontally:

0  0  0      0  0  0
0  1  0      0  1  0
0 -1  0     -1  0  0

: shift and subtract Southwest

The shift and subtract to the West delivers the least pseudo-3D effect. This is because not only are we applying the shift in one direction, but also that happens to be the direction a lot of the channels in the center are oriented. So directionality is a drawback of the method, but on the other hand I find very often that going through the exercise of comparing the alternatives I will learn a lot more about my data and the geological features in it than if I hadn’t, so I do it if I have the opportunity.

shift and subtract West

Try it yourself, and let me know if you agree with my conclusions. Did you find any of the remaining 5 direction better overall? The original image is here, and the code to create all filters and output displays is below:

%% Shift and subtract script
% A simple script to apply shift and subtract edge enhancement
% Additional functionality to display and QC results
%
% Reference: The Scientist and Engineer's Guide to Digital Signal Processing
% Chapter 24 - Linear Image Processing
% 3x3 Edge Modification
% http://www.dspguide.com/ch24/2.htm
%
% Acknowledgements: linkzoom function from the Matlab File Exchange
% www.mathworks.com/matlabcentral/fileexchange/21414-linkzoom-m-v1-3-aug-2009
%
%
%% 1 - dealing with input
% once imported jpeg file as galenaAlaska
%
in=zeros(1094,1664);
n=1:1:1094; k=1:1:1664;
in(n,k)=0.2989 * galenaAlaska(n,k,1) + 0.5870* galenaAlaska(n,k,2) ...
 + 0.1140 * galenaAlaska(n,k,3);
%
%
%% 2 - initializing
% this section not only create and applies the filters through convolution
% but also stores input, filters, and output in portable multi-dimentional
% arrays for ease of access
%
% input image array
IN=repmat(in,[1 1 8]);
% filtered output array
[mi,ni]=size(in);
[mf,nf]=size(flt);
OUT=zeros(mi+mf-1,ni+nf-1,8);
% filters array
flt=zeros(3,3); flt(2,2)=1;
FLT=zeros(3,3,8);
% this is the main part, populates filters, performs the filtering,
% and outputs filtered images
idx=[1,2,3,4,6,7,8,9];
for i=1:8
 s=idx(i);
 flt1=flt;flt1(s)=-1;
 FLT(:,:,i)=flt1;
 out=conv2(flt1,in);
 OUT(:,:,i)=out;
end
%
% to give you an idea of how it is done, just type at the prompt:
% flt
% which will print:
% flt =
%
%  0  0  0
%  0  1  0
%  0  0  0
% which is the initial kernel without shifted copies
%
% and then type:
% FLT
% which will print:
% FLT(:,:,1) =
%
% -1  0  0
%  0  1  0
%  0  0  0
%
%
%FLT(:,:,2) =
%
%  0  0  0
% -1  1  0
%  0  0  0
%
%
%FLT(:,:,3) =
%
%  0  0  0
%  0  1  0
% -1  0  0
%
% ...
% ...
% FLT(:,:,8) =
%
%  0  0  0
%  0  1  0
%  0  0 -1
% which are all the filters to apply shift and subtract
% in the 8 neighbors of the central pixel

For plotting you try use something like these few lines, which is what I used for the three examples above:

imagesc(OUT(:,:,1),[-25 25]);colormap(gray);
axis tight
axis equal
axis off

and for the original:

imagesc(in,[0 255]);colormap(bone);
axis tight
axis equal
axis off

However, it is far better for comparison and QC to display all the directions at once together with the original, like this:

I did that with this additional code below. Notice that to run it you will need to download from the Matlab File Exchange the function linkzoom by Carlos Vargas Aguilera. Linkzoom allows synchronizing of all subplots so that when zooming or panning one of the images, all other images will zoom or pan accordingly.

%% 3 - displaying
% displays the input image and all results as a matrix
% uses function linkzoom.m from Matlab File Exchange to allow
% simultaneous zooming and panning of all subplots at once
%
figure;
hold;
ax= zeros(3,3);
ax(1)=subplot(3,3,1);imagesc(OUT(:,:,1),[-25 25]);colormap(gray);
set(ax(1),'Position',[0.13,0.7093,0.2774,0.2298]);
axis tight
axis equal
axis off
title('NW','fontsize',14);
ax(2)=subplot(3,3,2);imagesc(OUT(:,:,4),[-25 25]);colormap(gray);
set(ax(2),'Position',[0.4108,0.7093,0.2774,0.2298]);
axis tight
axis equal
axis off
title('N','fontsize',14);
ax(3)=subplot(3,3,3);imagesc(OUT(:,:,6),[-25 25]);colormap(gray);
set(ax(3),'Position',[0.6916,0.7093,0.2774,0.2298]);
axis tight
axis equal
axis off
title('NE','fontsize',14);
ax(4)=subplot(3,3,4);imagesc(OUT(:,:,2),[-25 25]);colormap(gray);
set(ax(4),'Position',[0.13,0.4096,0.2774,0.2298]);
axis tight
axis equal
axis off
title('W','fontsize',14);
ax(5)=subplot(3,3,5);imagesc((in),[0 255]);colormap(bone);
set(ax(5),'Position',[0.4108,0.4096,0.2774,0.2298]);
axis tight
axis equal
axis off
title('ORIGINAL','fontsize',14);
ax(6)=subplot(3,3,6);imagesc(OUT(:,:,7),[-25 25]);colormap(gray);
set(ax(6),'Position',[0.6916,0.4096,0.2774,0.2298]);
axis tight
axis equal
axis off
title('E','fontsize',14);
ax(7)=subplot(3,3,7);imagesc(OUT(:,:,3),[-25 25]);colormap(gray);
set(ax(7),'Position',[0.13,0.11,0.2774,0.2298]);
axis tight
axis equal
axis off
title('SW','fontsize',14);
ax(8)=subplot(3,3,8);imagesc(OUT(:,:,5),[-25 25]);colormap(gray);
set(ax(8),'Position',[0.4108,0.11,0.2774,0.2298]);
axis tight
axis equal
axis off
title('S','fontsize',14);
ax(9)=subplot(3,3,9);imagesc(OUT(:,:,8),[-25 25]);colormap(gray);
set(ax(9),'Position',[0.6916,0.11,0.2774,0.2298]);
axis tight
axis equal
axis off
title('SE','fontsize',14);
linkzoom('xy');
zoom on;

In future posts I will show how to get non-directional dependent results with several methods:

- edge enhancement (sharpening)

- local binary patterns

- using gradient to render shading (similar to bump mapping)

- using high pass filtered version of the data to render shading

Filed under: Geophysics, Image processing, Matlab, ImageJ, et cetera, Tutorial, Visualization Tagged: convolution, emboss, etcetera, filter, filtering, geology, geomorphology, geophysics, Google Earth, ImageJ, kernel, Linkzoom, Matlab, meander, seismic, shift and subtract

Is Indigo really a colour of the rainbow?

Matteo — Sun, 20 Nov 2011 19:24:53 +0000

Very good question asked on Colour Chat blog, and a partial answer, at the link below:

Post by @stephenwestland about the place of indigo in the rainbow. Should it be there? #visualization #color #science colourware.wordpress.com/2009/07/20/ind…—
Matteo Niccoli (@My_Carta) November 14, 2011

Also:

George Biernson, in 1972, wrote in the American Journal of Physics Why Did Newton See Indigo in the Spectrum? and “hypothesizes that Newton saw seven reasonably distinct colors in the artist’s paint mixture color circle (red, orange, yellow, green, blue, violet, and purple) and therefore assumed he could also see seven distinct colors in his crude spectral projections”.

Others have argued Newton was trying to add a seventh color to match the seven notes of the western world’s musical scale.

I will tackle the many problems of rainbow in my forthcoming series The rainbow is dead…long live the rainbow!!!

Stay tuned…

Filed under: Color, Visualization Tagged: color, colour, hue, indigo, rainbow, spectral

A rainbow for everyone

Traffic lights for everyone

A rainbow for everyone

RELATED POSTS (MyCarta)

RELATED CONTENT (External)

What Words Can’t Convey: Illustrations Promote Scientific Understanding

Visualization tips for geoscientists – Matlab

Introduction

Method 1 – with a slope mask and transparency

Method 2 – using Shaded Pseudo Color

Method 3: 3D

Contours

Visualization tips for geoscientists: Surfer

GOODIES

RELATED POSTS (MyCarta)

RELATED CONTENT (External)

ACKNOWLEDGEMENTS

NOTES

Compare lists from text files using Matlab – an application for resource exploration

INTRODUCTION

TEST FILES

THE SCRIPT

AN AFTERTHOUGHT

Digital cartography picks

MyCarta – 2011 in review

An example of Forensic Image Processing in ImageJ

Initial Feature Detection Steps for Fingerprint Matching – by Ron DeSpain

Image processing tips for geoscientists – 1

Is Indigo really a colour of the rainbow?

Very good question asked on Colour Chat blog, and a partial answer, at the link below:

Also:

Others have argued Newton was trying to add a seventh color to match the seven notes of the western world’s musical scale.

I will tackle the many problems of rainbow in my forthcoming series The rainbow is dead…long live the rainbow!!!

Stay tuned…