Are you sure you want to delete this access key?
This experimental module performs camera calibration (distortion, intrinsic, and extrinsic camera parameter extraction). It computes and saves the intrinsics parameters of the input images. It is built on top of OpenCV, but aimed to simplify the process for people with no calibration or computer vision background at all (or for lazy people like myself).
Note: We are not aiming to have the best calibration toolbox, but the simplest one. If very high quality calibration is required, I am sure there must exist many other toolboxs with better extrinsic parameter estimation tools.
VERY IMPORTANT: OpenPose requires an upper triangular matrix for the intrinsic camera matrix. If you plan to use your own camera intrinsics, be aware of this.
Check doc/installation.md#calibration-module for installation steps.
Note: In order to maximize calibration quality, do not reuse the same video sequence for both intrinsic and extrinsic parameter estimation. The intrinsic parameter calibration should be run camera by camera, where each recorded video sequence should be focused in covering all regions of the camera view and repeated from several distances. In the extrinsic sequence, this video sequence should be focused in making sure that the checkboard is visible from at least 2 cameras at the time. So for 3-camera calibration, you would need 1 video sequence per camera as well as a final sequence for the extrinsic parameter calibration.
w
x h
number of squares to a h
x w
one). Our algorithm assumes that the origin is the corner at the top left, so rotating the chessboard circularly will change this origin across frames, resulting in many frames being rejected for the final calibration, i.e., lower calibration accuracy../build/examples/openpose/openpose.bin --num_gpu 0 --write_images {intrinsic_images_folder_path}
.--flir_camera --flir_camera_index 0
(or the desired flir camera index) to the webcam command.--video {video_path}
to the webcam command.{intrinsic_images_folder_path}
, file names are not relevant.grid_square_size_mm
, grid_number_inner_corners
, etc.) by running the --help
flag:./build/examples/calibration/calibration.bin --help
./build/examples/calibration/calibration.bin --mode 1 --grid_square_size_mm 40.0 --grid_number_inner_corners "9x5" --camera_serial_number 18079958 --calibration_image_dir {intrinsic_images_folder_path}
{intrinsic_images_folder_path}/18079958.xml
.# With distortion (straight lines might not look as straight lines but rather with a more circular shape)
./build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --flir_camera_index 0
# Without distortion (straight lines should look as straight lines)
./build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --flir_camera_index 0 --frame_undistort
Examples:
# Ubuntu and Mac
# Get images for calibration (only if target is not `--image_dir`)
# If video
./build/examples/openpose/openpose.bin --num_gpu 0 --video examples/media/video_chessboard.avi --write_images ~/Desktop/Calib_intrinsics
# If webcam
./build/examples/openpose/openpose.bin --num_gpu 0 --webcam --write_images ~/Desktop/Calib_intrinsics
# Run calibration
./build/examples/calibration/calibration.bin --mode 1 --grid_square_size_mm 30.0 --grid_number_inner_corners "8x6" --calibration_image_dir ~/Desktop/Calib_intrinsics/ --camera_parameter_folder models/cameraParameters/ --camera_serial_number frame_intrinsics
# Output: {OpenPose path}/models/cameraParameters/frame_intrinsics.xml
# Visualize undistorted images
./build/examples/openpose/openpose.bin --num_gpu 0 --image_dir ~/Desktop/Calib_intrinsics/ --frame_undistort --camera_parameter_path "models/cameraParameters/frame_intrinsics.xml"
# If video
./build/examples/openpose/openpose.bin --num_gpu 0 --video examples/media/video_chessboard.avi --frame_undistort --camera_parameter_path "models/cameraParameters/frame_intrinsics.xml"
# If webcam
./build/examples/openpose/openpose.bin --num_gpu 0 --webcam --frame_undistort --camera_parameter_path "models/cameraParameters/frame_intrinsics.xml"
# Ubuntu and Mac
# Get images for calibration
./build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --flir_camera_index 0 --write_images ~/Desktop/intrinsics_0
./build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --flir_camera_index 1 --write_images ~/Desktop/intrinsics_1
./build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --flir_camera_index 2 --write_images ~/Desktop/intrinsics_2
./build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --flir_camera_index 3 --write_images ~/Desktop/intrinsics_3
# Run calibration
# - Note: If your computer has enough RAM memory, you can run all of them at the same time in order to speed up the time (they are not internally multi-threaded).
./build/examples/calibration/calibration.bin --mode 1 --grid_square_size_mm 127.0 --grid_number_inner_corners "9x6" --camera_serial_number 17012332 --calibration_image_dir ~/Desktop/intrinsics_0
./build/examples/calibration/calibration.bin --mode 1 --grid_square_size_mm 127.0 --grid_number_inner_corners "9x6" --camera_serial_number 17092861 --calibration_image_dir ~/Desktop/intrinsics_1
./build/examples/calibration/calibration.bin --mode 1 --grid_square_size_mm 127.0 --grid_number_inner_corners "9x6" --camera_serial_number 17092865 --calibration_image_dir ~/Desktop/intrinsics_2
./build/examples/calibration/calibration.bin --mode 1 --grid_square_size_mm 127.0 --grid_number_inner_corners "9x6" --camera_serial_number 18079957 --calibration_image_dir ~/Desktop/intrinsics_3
# Visualize undistorted images
# - Camera parameters will be saved on their respective serial number files, so OpenPose will automatically find them
./build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --frame_undistort
build\x64\Release\calibration.exe
(or the one from the binary portable demo) with the same flags as above.1 0 0 0 0 1 0 0 0 0 1 0
the camera matrix of each XML file that will be used for --combine_cam0_extrinsics
../build/examples/openpose/openpose.bin --num_gpu 0 --flir_camera --frame_undistort --write_images ~/Desktop/extrinsics
# Ubuntu and Mac
./build/examples/calibration/calibration.bin --mode 2 --grid_square_size_mm 127.0 --grid_number_inner_corners 9x6 --omit_distortion --calibration_image_dir ~/Desktop/extrinsics/ --cam0 1 --cam1 0
./build/examples/calibration/calibration.bin --mode 2 --grid_square_size_mm 127.0 --grid_number_inner_corners 9x6 --omit_distortion --calibration_image_dir ~/Desktop/extrinsics/ --cam0 1 --cam1 2
./build/examples/calibration/calibration.bin --mode 2 --grid_square_size_mm 127.0 --grid_number_inner_corners 9x6 --omit_distortion --calibration_image_dir ~/Desktop/extrinsics/ --cam0 1 --cam1 3
# Potentially more accurate equivalent for the calibration between cameras 1 and 3: If camera 3 and 1 are too far from each other and the calibration chessboard is not visible from both cameras at the same time enough times, the calibration can be run between camera 3 and camera 2, which is closer to 3. In that case, the `combine_cam0_extrinsics` flag is required, which tells the calibration toolbox that cam0 is not the global origin (in this case is camera 1).
# Note: Wait until calibration of camera index 2 with respect to 1 is completed, as information from camera 2 XML calibration file will be used:
./build/examples/calibration/calibration.bin --mode 2 --grid_square_size_mm 127.0 --grid_number_inner_corners 9x6 --omit_distortion --calibration_image_dir ~/Desktop/extrinsics/ --cam0 2 --cam1 3 --combine_cam0_extrinsics
:: Windows
:: build\x64\Release\calibration.exe with the same flags as above
WITH_CERES
flag in CMake), you can improve the calibration results by performing an additional Bundle Adjustment refinement step on top of the previous results. We use camera 0 as the baseline for the internal computation, so try to avoid weird camera configurations in which camera 0 is completely isolated from the other cameras. Ideally, camera 0 should physically be the closest to all other cameras (i.e., the one more centered). But in practice, the accuracy improvement is almost none (as long as it is not too far from the others). To perform this bundle adjustment refinement for the example above, simply run the following line:# Ubuntu and Mac
./build/examples/calibration/calibration.bin --mode 3 --grid_square_size_mm 127.0 --grid_number_inner_corners 9x6 --omit_distortion --calibration_image_dir ~/Desktop/extrinsics/ --number_cameras 4
:: Windows
:: Ceres-compatible version not implemented for Windows yet. Make a pull request if you have a working version in Windows.
--camera_parameter_path
(or the default one indicated by the --help
flag if the former was not used).CameraMatrix
is a 3 x 4 matrix (you can see that the subfield rows
in that file is 3 and cols
is 4).CameraMatrix
field define the global translation
(in meters) with respect to the global origin (in our case camera 1).translation
vector.translation
vector represents the x
, y
, and z
distances to the origin camera, respectively. The camera is looking along the positive z
axis, the y
axis is down, and the x
axis is right. This should match the real distance between both cameras.Your CameraMatrix will look something like:
<CameraMatrix type_id="opencv-matrix">
<rows>3</rows>
<cols>4</cols>
<dt>d</dt>
<data>
8.4965260991319647e-01 1.1164693980389649e-01
-5.1538859446064478e-01 2.1494190603291283e+00
-1.5848315388246692e-01 9.8621217567379460e-01
-4.7630184633558698e-02 -4.5237471366168569e-01
5.0296474270386005e-01 1.2214952060972525e-01 8.5563190813085876e-01
1.1418502919988400e+00</data></CameraMatrix>
This is a 3x4 matrix, which represents rotation (R as a 3x3 matrix) and translation (t as a 3x1 matrix) in the following format: [R | t]. They represent the rotation and translation with respect to the world origin. When calibrating with OpenPose, we set one of the cameras as the origin, but this can be easily modified with some manual post-processing.
If you plan to use the calibration tool without using OpenPose, you can manually save a video sequence of your desired camera into each of the camera image folders (i.e., in the above example, the ~/Desktop/intrinsics_0
, ~/Desktop/intrinsics_1
, etc. folders).
If you wanna eventually run that camera with OpenPose, check doc/modules/3d_reconstruction_module.md#using-a-different-camera-brand.
The naming convention for the saved images is the following: [%12d]_rendered[CAMERA_NUMBER_MINUS_1].png
, where [CAMERA_NUMBER_MINUS_1]
is nothing for camera 0, _1
for camera 1, _2
for camera 2, etc. E.g., for 4 cameras:
000000000000_rendered.png
000000000000_rendered_1.png
000000000000_rendered_2.png
000000000000_rendered_3.png
000000000001_rendered.png
000000000001_rendered_1.png
000000000001_rendered_2.png
000000000001_rendered_3.png
[...]
OpenPose generates them with the base name [%12d]_rendered
. Ideally, any other base number should work as long as the termination [CAMERA_NUMBER_MINUS_1]
is kept consistent for all the camera views. E.g., you could call them also as follows (assuming 4 cameras):
a.png, a_1.png, a_2.png, a_3.png,
b.png, b_1.png, b_2.png, b_3.png,
etc.
Again, the critical step is to keep the file termination fixed as _1
, _2
, etc.
Press p or to see the previous file or, n or to see the next file
Are you sure you want to delete this access key?
Are you sure you want to delete this access key?
Are you sure you want to delete this access key?
Are you sure you want to delete this access key?