Take home Midterm

Points: 100

Important

You can use libraries like NumPy, SciPy and Pytorch for matrix operations and optimization, but you should not use any high-level computer vision libraries that provide built-in camera calibration functions (like OpenCV’s calibrateCamera function).

Anyone that uses Pytorch and will be given 10 extra points as credit for this assignment transferrable to the total exam category. In addition, you can use the Kornia library and consult the source code that implements DLT and other camera calibration related functions to help you with the coding. You should not use any high-level Kornia APIs but you can use helper functions such as kornia.geometry.conversions.convert_points_to_homogeneous.

You can structure your code as a package using this template or you can author a single Jupyter notebook at the expense of usability.

Camera Calibration

In this assignment you will be implementing a camera calibration algorithm from scratch. The goal is to estimate the intrinsic and extrinsic parameters of a camera using images of a known calibration pattern (like a chessboard). You will then use these parameters to measure a room dimensions.

You will implement Zhang’s method from scratch including the DLT initialization step as well as the non-linear refinement step that can be implemented with stochastic gradient descent.

You can use these images as a test images - they are are available in industry standard dataset formats (Parquet). You can read the Parquet file into a pandas DataFrame df = pd.read_parquet(parquet_file_path). The solution of the camera calibration parameters using OpenCV is included in your course site. You can test the correctness of the implementation by comparing your results to the OpenCV results.

You can then repeat with taking photos of 9x7 checker board pattern using your smartphone / tablet set on a fixed zoom level.

  • Ensure good, diffuse lighting, keep the pattern flat, and make sure it’s in focus for every shot
  • Disable autofocus and set a fixed focus for the duration of the calibration process.
  • Capture the pattern from many different positions and angles, up to about 45 degrees.
  • Ensure the pattern fills a significant portion of the camera’s field of view and take images that cover the entire lens area.
  • Try to avoid images where the pattern is perfectly flat or facing straight on to the camera, as this provides less useful information.
  • Capture both wide shots and images where the pattern is closer and larger in the frame.
Source: Camera Calibration