Discussion: contours

Would you like contours from the lidar data for the area?

He already has it :)

Ed, here's the base I made if you are interested. This was made from one of the .tif bare earth tiles that Brendan sent me.

prospect.zip - ocad6 basemap, including vector data but no by-hand additions.
prospect_unsharp_11.zip - bare earth lidar surface, unsharp-masked .bmp image, 1m/pix.
prospect_ortho_11.zip - color ortho corresponding section, binned to 0.5m/pix.

I'll take a look at those. I've written a bunch of processing scripts in the GRASS framework to generate the contours. The unsharp and veg images that you produce are much nicer than the ones I have been able to come up with so far though, so they will be much more useful to him.

Are you applying any gaussian smoothing to the data when doing the unsharp?
My unsharp filter for the same area produces: http://www.despard-design.com/bare221902_unsharp.t...

I use a boxcar. The unsharp mask is very basic: smooth the gridded surface with a 5x5m (pix) boxcar and subtract that from the non-smoothed version. I typically use 5x5, but it depends a little on the lidar post spacing.

Then I smooth the unsharp-masked image slightly to make it more "readable" in ocad. I forget what size for Boston - its either 5x5m or 7x7. Normally I would only use a 3x3 on the final unsharp-masked product just to cut the distraction of the duff+noise, but the Boston area bare earth .tifs are, unfortunately, full of pock-mark holes...some kind of error in the data. This larger-than-usual final smooth is why it looks a little soft. Would love to get ahold of the point cloud, as I'm sure that these 1m gridded tiles are worse than they need to be. That's no guarantee that these pock-marks aren't in the point cloud as well though.

Same thing with the PA statewide data - the 1m .tif grids they produced are dumbed down from the very same 1m grid I get when I start from the point cloud. My guess is they cut the res to save process time or something. Or just made a mistake.

Brendan was able to get some first-returns .tif tiles from the Harvard GIS library to go along with the bare-earth ones. I used those for a new base of the arboretum for Brendan and was able to generate an understory veg product. Still would prefer the point cloud for this kind of thing, but that's all we had.

I greyscale stretch (to 8 bits) the unsharp from -0.2 to 0.2m or whatever gives the best contrast, and then break them into subtiles for use in ocad. Only needed 1 tile for this small area, thus the _11 in the file name (x,y in the tile grid, starting at lower left).

I had a look at the image you linked. It looks like a gradient image rather than an unsharp mask. The shadowing is in one direction (from left to right). The unsharp mask won't have a direction - one of its best qualities. Also the relief in your image looks inverted, and I think its color-limited - like 4-6 bits rather than 8. Maybe the color table is truncated to a limited number by grass or something?

Yeah, looks like there are only 20 different values in your .tif image, with the majority between 8-12 - so basically only 5 shades of grey in your image. Histogram:

I will have to give your process a try tonight. That image was generated by taking the bare tif, and applying a 7x7 Laplacian of Gaussian filter to it. I then take the original image and subtract the result from the filter.
To color it, I assigned 0,0,0 to -10, and 255,255,255 to +10 with data clipped to -10, +10.

It should be very similar to your process, but my 7x7 filter matrix is smoothing things a lot.

The first returns data that we have only covers a smaller area centered around downtown boston. I'm not sure if there is more that we don't have, or if that is all they generated. I really need to email someone at MassGIS to see if the point cloud is available, as that would make it possible to do some better veg filters.

For the area that both the bare tiles and first tiles exist, I just took first-bare, which seems to be decent for identifying clearings and the centers of individual trees in a field.

Maybe grass is using integer arith in all your processing. You'll need float, otherwise the result of the smoothing will be truncated, and then the difference will be an int difference of that.

Yeah, I usually make an unsharp-mask of an all-returns surface, which effectively makes a veg-height image in one pass. Then I make two versions of this - one hard stretch for clearings, and a softer stretch to the full dynamic range to see differences in canopy height which can sometimes be useful for things like veg boundaries in the forest or building footprints.

The other product I make from the all-returns or first-returns is an understory vegetation density map. Do this by selecting only a range of veg heights (rather than all heights above ground, which is the product described above), say between 1 ft and 8 ft above bare earth. Then with a moving box, count the number of these "hits" per unit area (density). Make an image of this density, stretch between white and green and you have a nice understory veg map appropriately colored. Use that as a template in the field under the contours. Example of that in this .ppt presentation.

Nice photos!
Boston skyline looks like a bar graph.
And the green is a riot seen.

I just uploaded a new unsharp image. I think it works much better now.
GRASS was using integer math for some of it, and my method was just smoothing away any of the sharp detail.

Looks good! Are you using gaussian or square kernel or something else?

I went back and looked at my script and realized I was pre-processing the .tif before making the unsharp mask. This was to deal with the pock-marks. I pass a ring-median filter over the bare earth image *before* making the unsharp mask. This isn't something I normally do - just specific to the Boston dataset. Its the convolution with the ring-shaped kernel that causes the apparent "micro terracing" in my version. The ring kernel I used was inner radius = 1 and ring width = 1, so it looks kindof like a laplacian. A "plus" shape with a hole in the center.

This let me use a smaller smoothing (square) kernel for the unsharp mask so the unsharp image could retain as much sharpness as possible while still reducing the pock marks, but unfortunately it leaves the "terracing" imprint. A trade off. From my script it looks like I also tried a ring with a 2pix inner radius as well. That was overkill. The pock marks tend to be single pixel events in the .tifs. I suspect they were caused by the veg filter that was used. If we could get the point cloud we could confirm this and re-filter ourselves. Also could try to filter the first-returns .tif. Dunno how well that would work. Alot of the useful ground returns will be missing from a first returns only set.

I was discussing this with Ross this afternoon. I appreciate all the work people have done on this LIDAR, but I must say I was disappointed with the LIDAR I am using for Prospect Hill. I deleted the lesser intervals until I had a 2.5 interval basemap. Then I brought up the original 0.5 m basemap, and put in contours which diverged from the 2.5 m ones as form lines. So I figure I had everything there. But in the field, there are obvious features, such as small re entrants and large boulders and knolls which are larger than 0.5 meter, yet they are no where to be seen. It is as if either there is too much smoothing, or the LiDar was only getting returns at large, say 5-10 meter, intervals.
So although I am assuming the basic interval is OK, small ups and downs do not show.

Hey Jeff,

Although the contour interval in the ocad file I sent was 0.5m, the average lidar point spacing was definitely *not* 0.5m. We don't actually know what it is, but based on the fact that I can't see trails I'd estimate this is 1.5m avg post spacing. So the smallest point features that even have a chance of appearing are 3m. In order to see 0.5m sized features in the terrain you'd need a sample every 0.25m (Nyquist sampling), a sample density that I've never seen or heard of.

In addition - as Ed and I have been discussing - the veg filtering wasn't especially great, leaving some artifacts in the bare earth which have indeed required more smoothing than we'd normally use on a lidar dataset. The gist is this is not optimal lidar data for an O-basemap, its average-minus.

By the way, you don't have to delete lines from the ocad file I sent to get a 2.5m interval. The contours are differentiated with different symbols in the file. You can simply hide the 0.5m and 1m symbols in the symbol table and the symbols that are left will form a 2.5m interval map, with indexes. You could then edit the 0.5 and 1m symbols to be dashed lines and un-hide them. Or make then a different color, like purple.

"estimate this is 1.5m avg post spacing. So the smallest point features that even have a chance of appearing are 3m".
Well that explains a lot!