{ Computer Vision Specialist }
{ Research} Advised by Sanjiv Singh at the Robotics Institute, Carnegie Mellon University
Camera
Calibration This
project entailed creating a simple and fast way of performing camera
calibration for Athena Controls. An My biggest contributions were a very intuitive interface for selection of images and a fast method to go about the iterative steps to reducing error. My code is available here. The simplest way to learn more about it is to read the following “how-to” tutorial: Contents 1)
Taking an Image set This section will focus on how to take a reasonable image set to perform calibration. First, a target should be made, consisting of a black & white checkerboard on a flat background. The target should not be flexible, because any warping in the target will appear to come from the camera. Making the target bigger will reduce the weight of any small apparent warping. Here is an example of a target:
Note
that the aspect ratio should be similar to the images you wish
to take, e.g. 4x3 for 640x480. This is merely a practical matter
of being able to fit the entire target in the image. The number
of square does not matter. What is essential in creating the target
is that the squares are actually squares, a not stretched into
rectangles because of a faulty printing process. There
are two specific images you should include. The first is where
the side of the target is along the edge of the image,
so that you
can measure radial distortion. In another, there should be a noticeable
projective depth to the target. Example of a image set
2) Installing the software
3) Starting a calibration procedure To start a new calibration procedure, click “New”. To load a previously saved procedure, click “load”.
Upon selection of a directory, the path will be displayed in Choose_Directory text section. Note that if there are no images in this directory, you cannot continue to the next step of choosing images. Also, do not choose a directory which has many images, e.g. more than 30. Rather, create an image directory that just contains the images you will likely include. This will save you time later. Loading
a previously saved procedure
4)
Choosing images Note again that if the directory chosen in the previous step does not contain any images, you will not be allowed to move to this step. Also note that the directory you choose shouldn’t contain too many images, because choosing them will be difficult. This is because all the images in the directory will be loaded and displayed. If you have insufficient memory to load all the images, problems might arise, or if you have too many images, choosing the correct ones will be difficult because they are resized down so all images can be displayed, as can be seen below.
If you click on a black or blank area, nothing will happen. But, if you click on the wrong side, then you might get undesirable results. Clicking “Click to Exclude”, then clicking on an image that is already excluded, will cause the image which is at that same relative location in the include image group to be excluded. This can be reversed easily by clicking on the correct side. Following the written instructions should make this interface intuitive and straight forward. Below is an example with 4 images excluded, while in the process of excluding another image.
5) Marking Corners
Tracking the images which have been marked is done by the text areas with the headings “Images Marked:”, “Current Image:”, and “Images NOT Marked:” Their meaning is self-explanatory, but keep in mind that the numbers are only place holders, and do not correspond to the image names. They are simply the order in which the images were loaded into memory from the image directory. To navigate through this index to mark new images, so see past markings, the button on the bottom provide, again, self explanatory functionality, with “First”, “Previous”, “Next”, and “Last”. To stop marking corners at any time, press “Done”. To mark the external corners of an image, press “Mark Corners”. An “external” corner is one which is on the barrier between the last complete square and the partial squares. Internal corners are the corners that are inside the rectangle made by the external corners. Below is an image of the target, with the external and internal corners identified.
When “Mark
Corners” is
clicked, the message “Click
the 4 external corners clockwise from top left” will appear.
It is important to do this every time. Try to start from the same
point
in the
target, and move clockwise to the next external corner. It is also
important to make this accurate by trying to click what appears to
be the exact location of the corner. After the forth corner is marked,
an image will appear on the right with the internal and internal
corners marked. Below is what it should look like:
The red crosses in the left image are the initial guesses for the
image corners. For each corner, within windows marked in blue in
the right image, the exact location of the corner is found using
a method which finds the center of a saddle-point. The final locations
of the corners are marked in the red crosses on the right image.
Below is an example where the final location for the image corners is clearly wrong:
Note how the highlighted red crosses on the right do not correspond to the target corners. This is because of the radial distortion. To properly adjust for this issue, press the ‘+’ or ‘-’ buttons next to the “Estimate distortion” field until the red crosses in the right image appear over the target corners. This will adjust the guess of radial distortion, which is initially zero. In the case of these example images, the solution comes after pressing the ‘-’ twice. Below is the result:
This process might seem difficult because the error is actually only a few pixels. This is balanced by the fact that adjusting for one image will make the same adjustment for all the images. Also, this one image can be found easily by finding the image where the edge of the target is closest to the edge of the image. When you have marked the corners for all the images, and are satisfied with the estimates for the internal corner location, it is recommended that you cycle through the images, because internal corners are recalculated with new parameters each time the image is viewed. So, for example, if you change the radial distortion estimate mid-way through the procedure, you should view the previous images to force a recalculation. When you are done marking corners, click the “Done” button.
The “Default” button will return all the parameters to their default values. The “Set for All” button sets these parameters for all images, because each image’s advanced settings are unique, with the exception of radial distortion, which is a property of the camera and not the image. Note that you should not have different targets using in a single calibration run, so the square count should not change between images. Press “Done” to close the Advanced menu. 6) Performing Calibration The following GUI window will appear, which is very similar to the “Choose Images” interface seen before:
The image group on the left is the group of images which will be included for the calibration procedure, i.e. those that have been chosen and marked. The group on the right, empty by default, is the group of images which will be excluded from the calibration. To calibrate, press the “Calibrate” button. This procedure takes some time, as there is a good deal of computation which is hidden from the user. When the calibration has completed, the results will be printed to the light-grey area on the bottom of the window. Error Reduction
To reduce the error, which is the last item listed on the light
gray area, there are a number of methods.
The skew is a measure of how far the horizontal and vertical axes are from being orthogonal. The radial distortion should always be estimated, but the only question is to what order. A higher order estimation will probably yield better results. As a general caveat, if the error associated with a term is on the same order of magnitude as the term being calculated, then including that term in the calculation will probably increase the error. This information is contained in the results printed in the light grey area. A second way to reduce error is to change the window size used to find internal corners for a specific image. This is done by pressing the “+” or “-” button below the “Exclude” button, then clicking on an image in the left group. This will recalibrate entirely, so this will be a time consuming effort. Generally, you should stop when a minimum has been reached for each image. You might then notice that the error for that image is not the maximum image error, and this process will be iterative for each image which has high error. The third method to reduce error is to include or exclude images from the calibration. The images with the greatest error are prime candidates. This is done in exactly the same manner as choosing which images in the directory to include. To exclude an image, press “Exclude”, then an image on the left to exclude. To include an image, press “Include”, then an image on the right to include. Saving
Results
The default names are those used by the other vision software in the package, and the format is that which that code can read. You can change the name here as desired, and change the format manually as needed after saving. The results are saved to calibration directory. 7) Saving a Calibration procedure The following window will appear:
Edit the name or directory as needed. To change the directory, click the “Choose directory” button, where the standard folder window will appear:
After creating a name, and choosing a directory, click the “Save” button to save the state. This will take some time because the internal state of the program is quite large.
|
Professional
{
Research }
{ I-Kbot }