A method, a system and a computer device for initiating bi-directional compression of a video stream in a packet switched network, based on delay tolerance of a service or application. A video frame recompression (VFR) utility determines an end-to-end (E2E) delay tolerance retrieved from a Real-time Transport Control Protocol (RTCP) report. The VFR utility then determines the actual expected delay based on a deep inspection of packet headers. The VFR utility utilizes a processing opportunity delay (which determines whether the E2E delay tolerance is greater than the actual expected delay) to reprocess video content comprising Intra-coded (I) pictures/frames and Predicted (P) frames to improve compression efficiency. The VFR utility may also utilize a complexity ratio which is a ratio of the I-Frame rate and the P-Frame rate to select frames for compression. The VFR utility recompresses video content by replacing P-Frames with B-Frames.