An electronic device may have a display with a brightness that is adjusted based on ambient light data from one or more ambient light sensors. An ambient light sensor may be formed from a semiconductor substrate such as a silicon substrate. Sensor structures may be formed in the silicon substrate. Conductive vias or other conductive paths may be used to interconnect sensor structures on a frontside surface of the ambient light sensor to contacts on a backside surface of the ambient light sensor. The ambient light sensor may be mounted on a substrate layer in the electronic device. The substrate layer may be a planar layer of glass or plastic such as a transparent display layer. The contacts of the ambient light sensor may be mounted to corresponding contacts on the surface of the substrate layer. The substrate layer may be a thin-film transistor layer in a liquid crystal display.