You are confusing Follow with Tracking. Follow uses the GPS coördinates of the mobile device, it’s NOT based on optical Tracking. In Follow Mode you Frame the object (mobile device) by adjusting the Gimbal Pitch and drone Yaw.
Also have a look at this thread: