How do calibrated stereo views determine a point’s depth?

Instruction: Two rectified pinhole cameras share focal length f=800 pixels and principal point (0,0). The right camera is B=0.20 m to the right of the left camera; their axes are parallel. A point has left coordinates (80,40) and right coordinates (48,40). Derive its coordinates in the left camera frame. Explain zero disparity, uncertain matches, and unknown baseline scale.

Context: Derive a multi-view reconstruction under explicit calibration, coordinate-frame and scale assumptions.

Updated

Worked answer

In the left frame let the point be (X,Y,Z), with positive Z in front of the cameras. Projection gives uL=fX/Z and vL=fY/Z. In the right camera its horizontal coordinate is X−B, so uR=f(X−B)/Z. Subtracting gives disparity d=uL−uR=fB/Z and therefore Z=fB/d.

Here d=80−48=32 pixels, so Z=800×0.20/32=5 m. Then X=uL Z/f=80×5/800=0.50 m and Y=vL Z/f=40×5/800=0.25 m. Reprojection into the right camera gives 800×(0.50−0.20)/5=48 pixels and vR=40, agreeing with the fixture.

For ideal rectified views, corresponding points share a scanline, a special case of the epipolar constraint xRᵀ F xL=0. Real images need calibration, distortion correction and robust correspondence checks; satisfying the constraint alone does not prove a match is correct. Near-zero disparity produces extremely uncertain large depths; zero disparity cannot yield a finite depth through this formula. With unknown baseline magnitude, reconstruction scale is ambiguous. A metric baseline and focal calibration are what justify meters here.

Reference: OpenCV calibration and 3D reconstruction.

Related Questions