itkCenteredRigid2DTransform registration optimization problems

Discussion:

Felix Bollenbeck

2008-01-29 13:08:40 UTC

Hi ITKusers

Im using the itkCenteredRigid2DTransform to register a stack of
light-microscopy slice images containing sectionend objects in masked
background within the image-pyramid framework of ITK.

I have a question regarding the transformtype for registration of image
pairs:

objects in the images I use have a kind of symmetry axes, they are
'similar' on both sides but not the 'same'. The problem is, that
sometimes the optimizer gets stuck in a local optimum representing the
alignment of two images where the moving images is flipped along this
axis-can I make myself clear?

The problem is- that on broader scales, this 'similarity' turns out to
be more less 'same', which causes the parameters to remain in this
'valley' in the subsequent fine optimization on smaller scales.

starting with a finer resolution ends up in bad convergence, or bad
overall results.

I tried to tune stepwidths, optimizer scales etc. but i couldnt find any
overall tendency, due to my little detailed knowledge on the optmizer
(regularstepdescent).

Can anybody suggest a method how to optimize the rotation angle more
carefully?

Regards&TIA,

Felix.

--
Felix Bollenbeck Email: bollen-lOlAGW7ZbtGO8N+***@public.gmane.org
Pattern Recognition Group Phone: +49 (0)39482 5344
Leibniz Institute of Plant Fax : +49 (0)39482 5137
Genetics and Crop Plant Research
Correnstr.3 http://www.ipk-gatersleben.de
06466 Gatersleben,Germany

Felix Bollenbeck

2008-01-29 14:38:39 UTC

Permalink

Hi Gabriele,

ok, that is some kind of random search, right? - I have to read the
paper for understanding.

Was it difficult to tune the parameters? I am not using mutual
information metric, but I will try it out.

I guess I will have to anneal 'epsilon' when going to finer scales. can
you comment on your choice of parameters?

Thanks for your time!

FElix.

Hi Felix,
I've had a lot of problems with the regularstepgrdient descent , finally
I switch all my code to use the OnePlusOneEvolutionaryOptimizer that is
relly well suited for the MattesMutulaInformationMetric .
Using that classes , after finding the correct parameters tuning for the
optimizer,all problems due to miallignement disappers in my case (MRI
AffineRegistration) .
Best Reguards
Gabriele

Post by Felix Bollenbeck
Hi ITKusers
Im using the itkCenteredRigid2DTransform to register a stack of
light-microscopy slice images containing sectionend objects in masked
background within the image-pyramid framework of ITK.
I have a question regarding the transformtype for registration of
objects in the images I use have a kind of symmetry axes, they are
'similar' on both sides but not the 'same'. The problem is, that
sometimes the optimizer gets stuck in a local optimum representing the
alignment of two images where the moving images is flipped along this
axis-can I make myself clear?
The problem is- that on broader scales, this 'similarity' turns out to
be more less 'same', which causes the parameters to remain in this
'valley' in the subsequent fine optimization on smaller scales.
starting with a finer resolution ends up in bad convergence, or bad
overall results.
I tried to tune stepwidths, optimizer scales etc. but i couldnt find
any overall tendency, due to my little detailed knowledge on the
optmizer (regularstepdescent).
Can anybody suggest a method how to optimize the rotation angle more
carefully?
Regards&TIA,
Felix.

Robert Jason Harris

2008-01-29 14:45:46 UTC

Permalink

Hello all,

I used ITK for a project this past summer, and I have since been reviewing
what I have done. It occurred to me that it might be useful to enter
NaN values into some pixels of an image after it had been processed by a
filtering algorithm that I wrote. Is this defined in ITK, or will ITK
break? I have been looking for the use of NaN in ITK, and I've found
hints but nothing definite. I'm thinking of something like the following:

...
// process image and come up with criterion based
// on pixel values to determine which to set to NaN and
// which to keep at their current value

if ( pixel value < some value)
image_iterator.SetPixel(NaN)
else
//do nothing.

Thanks for any help you can provide!

Robert Harris

Daniel Mace

2008-01-29 16:57:47 UTC

Permalink

Robert,

Even if ITK would allow it (it may internally, but I think you would
have issues with writing and reading it), it probably would not be a
good idea to do it. Providing memory isn't an issue, the more practical
solution would be to create an identically sized mask image with 0's and
1's denoting your criteria. It's probably more computationally
efficient to check the state of the mask image and maintain two
iterators than it is to check if your image value is NaN (even faster if
you cast your original image and masked image into a VectorImage and use
a single VectorImageIterator to walk through the values (this is if you
need to do further processing on the image)). You can then write the
masked image out as a separate file, or use ScalarToArrayCastImageFilter
to construct it as a vector image and write it out.

Cheers,
Dan

Post by Robert Jason Harris
Hello all,
I used ITK for a project this past summer, and I have since been
reviewing what I have done. It occurred to me that it might be useful
to enter NaN values into some pixels of an image after it had been
processed by a filtering algorithm that I wrote. Is this defined in
ITK, or will ITK break? I have been looking for the use of NaN in
ITK, and I've found hints but nothing definite. I'm thinking of
...
// process image and come up with criterion based
// on pixel values to determine which to set to NaN and
// which to keep at their current value
if ( pixel value < some value)
image_iterator.SetPixel(NaN)
else
//do nothing.
Thanks for any help you can provide!
Robert Harris
_______________________________________________
Insight-users mailing list
http://www.itk.org/mailman/listinfo/insight-users

Michael Halle

2008-01-29 17:36:26 UTC

Permalink

Post by Daniel Mace
Robert,
Even if ITK would allow it (it may internally, but I think you would
have issues with writing and reading it), it probably would not be a
good idea to do it. Providing memory isn't an issue, the more practical
solution would be to create an identically sized mask image with 0's and
1's denoting your criteria.
It's probably more computationally
efficient to check the state of the mask image and maintain two
iterators than it is to check if your image value is NaN (even faster if
you cast your original image and masked image into a VectorImage and use
a single VectorImageIterator to walk through the values (this is if you
need to do further processing on the image)). You can then write the
masked image out as a separate file, or use ScalarToArrayCastImageFilter
to construct it as a vector image and write it out.

If there isn't ubiquitous support for NaNs in a future ITK, I think
there are at least a few compromise steps that would be extremely handy.
These aren't exactly what Robert wants (actually setting values to be
NaN) but they are related.

*) A standard way to produce the mask+image you just described from an
existing image that has NaNs (and vice versa). That allows a couple of
things:
-) support for existing data sets that use NaNs
-) Robert's application: he can use (0.0/0.0) to create a NaN and
use it in an image because it's convenient for him, then turn that
information into a mask because it's convenient for ITK.

*) A standard way to tell ITK that an image may have NaNs, and a way to
know ask if an image does actually have NaNs. Using NaNs can introduce
some extra tests or computational expense to filters, costs that can be
avoided in the more common case where NaNs don't exist. If there was a
way to warn ITK that an image might have NaNs, then those extra tests
could be turned off most of the time.

To avoid further computation costs, histograms could use this
"MayHaveNaNs" flag to enable an extra method, "DoesHaveNaNs".
DoesHaveNaNs says that a particular image does, in fact, have NaNs.
NaN-aware processing elements can use that information to choose between
fast paths or careful paths. If an image that "MayHaveNaNs" in fact
(not DoesHaveNaNs), then the fast path can be used. A filter that
doesn't deal with NaNs can also check this flag or method and bail out
accordingly.

I think these additions strike a balance between making all of ITK
NaN-aware (probably impossible at this point) and allowing specific
parts to choose to be NaN-aware in a standard way.

Following TEEM/AIR's lead, it might make sense for the flags to consider
Inf as well.

--Mike

Post by Daniel Mace
Cheers,
Dan

_______________________________________________
Insight-users mailing list
http://www.itk.org/mailman/listinfo/insight-users

Rupert Brooks

2008-01-29 21:51:38 UTC

Permalink

Hi Felix,

I think the problem you are having could be caused by two possibilities.

Message: 6
Date: Tue, 29 Jan 2008 15:38:39 +0100
Subject: Re: [Insight-users] itkCenteredRigid2DTransform registration
optimization problems
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hi Gabriele,
ok, that is some kind of random search, right? - I have to read the
paper for understanding.
Was it difficult to tune the parameters? I am not using mutual
information metric, but I will try it out.
I guess I will have to anneal 'epsilon' when going to finer scales. can
you comment on your choice of parameters?
Thanks for your time!
FElix.

--
Pattern Recognition Group Phone: +49 (0)39482 5344
Leibniz Institute of Plant Fax : +49 (0)39482 5137
Genetics and Crop Plant Research
Correnstr.3 http://www.ipk-gatersleben.de
06466 Gatersleben,Germany

--
--------------------------------------------------------------
Rupert Brooks
McGill Centre for Intelligent Machines (www.cim.mcgill.ca)
Ph.D Student, Electrical and Computer Engineering
http://www.cyberus.ca/~rbrooks

Rupert Brooks

2008-01-29 22:13:39 UTC

Permalink

Sorry about that last email - i accidentally clicked send long before
i should have

Felix-

I think the problem you are having could be caused by two possibilities.

1. Your image is starting out closer to the flipped solution, meaning
the optimizer will slide downhill to the flipped solution because that
is the bottom of the "valley" in which it starts. If this is the
case, the optimizer is doing what it should, in some sense, and it
will be hard to tune your way out of this problem with any optimizer.
I suspect that this is what is happening. However, it may not be that
difficult to solve.

If the correct solution has better cost function values than the wrong
one, then, after the first level of your pyramid, flip the image,
reoptimize and take the best of the two. (If you have an N-fold
symmetry, you may have to explore each of the N-possibilities.) In
practice, i have done something like this using the
ExhaustiveOptimizer as a first pass. However, there was a bug that
got in the way - i believe fixed now - check mantis before you try
using that class, if you do.

If the correct solution actually looks worse than the wrong one, in
terms of cost function values, then you have a very tricky problem.
You'll need to find some sort of bound or regularizer to limit the
search space.

2. The optimizer parameters are poorly tuned or the cost function is
very noisy - I don't think this is the case because if it was, the
problem would probably not be so specific (going to the flipped
position), but would rather show as generally lousy performance
(aborts, trapped in various minima, poor accuracy, etc). However, if
you think its the scale factors, I'll refer you to MICCAI short paper
of mine about how to set that. Alternatively, you could try using the
Powell optimizer - i've played with all of the ITK optimizers and i
find that its remarkably robust. Its slow though, so you might want
to use it only on the top level of your scale pyramid.

Scaling Angles and distances....
http://www.ia.unc.edu/MICCAI2005/ShortPapers/pdfs/paper947.pdf

Cheers,
Rupert B

Felix Bollenbeck

2008-01-30 09:39:07 UTC

Permalink

Hello Rupert,

Thanks for your reply and the paper. I am just running a modified scale
scheme, as you suggested. The ITK docs suggest just a static scaling,
accounting for the different radiant/pixels displacement
dimensions-which I used up till now.

Yes I agree - when reading the docs and examples, the reader might get
the opinion, that optimizers are some kind of 'magic box' finding the
best transform, once parameters are set properly. But as you said,
especially gradient descent optimizers just descents into the NEXT
minimum.

My registration results are either perfect or misaligned, the correct
solution definitely has the best metric value.

Yes I also thought that a grid-search on the lowest scale might be an
option, rising the question, how dense this grid should be.

Trying flippings on pyramid top might do, on the other hand, I would
like to keep cost down as much as possible, because I have to register
several tens-thousands of images in total (serial sectioning stacks).

In my MATLAB prototype I use an initial principal-axis-transform for
bulk initialization, but I found the itkImageMomentsCalculator not so
well documented. Have you used it?

Best regards,

Felix.
--
Felix Bollenbeck Email: bollen-lOlAGW7ZbtGO8N+***@public.gmane.org
Pattern Recognition Group Phone: +49 (0)39482 5344
Leibniz Institute of Plant Fax : +49 (0)39482 5137
Genetics and Crop Plant Research
Correnstr.3 http://www.ipk-gatersleben.de
06466 Gatersleben,Germany

Post by Rupert Brooks
Sorry about that last email - i accidentally clicked send long before
i should have
Felix-
I think the problem you are having could be caused by two possibilities.
1. Your image is starting out closer to the flipped solution, meaning
the optimizer will slide downhill to the flipped solution because that
is the bottom of the "valley" in which it starts. If this is the
case, the optimizer is doing what it should, in some sense, and it
will be hard to tune your way out of this problem with any optimizer.
I suspect that this is what is happening. However, it may not be that
difficult to solve.
If the correct solution has better cost function values than the wrong
one, then, after the first level of your pyramid, flip the image,
reoptimize and take the best of the two. (If you have an N-fold
symmetry, you may have to explore each of the N-possibilities.) In
practice, i have done something like this using the
ExhaustiveOptimizer as a first pass. However, there was a bug that
got in the way - i believe fixed now - check mantis before you try
using that class, if you do.
If the correct solution actually looks worse than the wrong one, in
terms of cost function values, then you have a very tricky problem.
You'll need to find some sort of bound or regularizer to limit the
search space.
2. The optimizer parameters are poorly tuned or the cost function is
very noisy - I don't think this is the case because if it was, the
problem would probably not be so specific (going to the flipped
position), but would rather show as generally lousy performance
(aborts, trapped in various minima, poor accuracy, etc). However, if
you think its the scale factors, I'll refer you to MICCAI short paper
of mine about how to set that. Alternatively, you could try using the
Powell optimizer - i've played with all of the ITK optimizers and i
find that its remarkably robust. Its slow though, so you might want
to use it only on the top level of your scale pyramid.
Scaling Angles and distances....
http://www.ia.unc.edu/MICCAI2005/ShortPapers/pdfs/paper947.pdf
Cheers,
Rupert B

Rupert Brooks

2008-01-30 12:22:26 UTC

Permalink

Post by Felix Bollenbeck
My registration results are either perfect or misaligned, the correct
solution definitely has the best metric value.
Yes I also thought that a grid-search on the lowest scale might be an
option, rising the question, how dense this grid should be.

A difficult question - one way to explore it is to probe the value of
your cost function around the minimum and look for how wide the dip
is, then divide up your space so that you are sure to fall into the
right dip. However, since you seem to have two fairly good dips in
your registration space, This might be difficult - it might be
difficult to tell one good dip from another. I lean towards the next
option.

Post by Felix Bollenbeck
Trying flippings on pyramid top might do, on the other hand, I would
like to keep cost down as much as possible, because I have to register
several tens-thousands of images in total (serial sectioning stacks).

This is the way i would probably go - and it might be the cheapest and
easiest to implement. You could put it in an observer that is called
when the registration changes levels. On the first change of levels,
the observer could rotate the object 180, evaluate the metric and, if
that is better set the next level registration parameters to the
rotated transform and carry on.

Whatever you do, I'd be very curious to know how it works out.

Post by Felix Bollenbeck
In my MATLAB prototype I use an initial principal-axis-transform for
bulk initialization, but I found the itkImageMomentsCalculator not so
well documented. Have you used it?

Unfortunately no, not beyond running the example in the ITK book. I'm
sure that someone out there has though :-)

Best,
Rupert