A brightness adjustment method and a brightness adjustment device of a display panel and a display device are provided. The brightness adjustment method of the display panel including a plurality of sub-pixels includes acquiring grayscale data received by the display panel; performing a data processing process on the grayscale data and at least inserting a plurality of transitional grayscales between a 0-order grayscale and a 1st-order grayscale to form target grayscale data; and adjusting the brightness of the plurality of sub-pixels based on the target grayscale data.