Utilizing local camera compute for camera calibration is provided. Perception outputs are generated, by respective camera processors of each of a plurality of cameras, the perception outputs indicating properties of calibration references located in raw images captured by the respective camera. In a calibration mode, a camera controller, in communication with the plurality of cameras over a data bus, receives the perception outputs from the plurality of cameras. In a runtime mode, the camera controller utilizes calibration data generated based on the perception outputs to fuse compressed images received from the plurality of cameras over the data bus into a combined surround view.