H.264/AVC視訊編碼標準的編碼效能(coding performance)遠勝於之前的影像編碼標準。這是因為H.264/AVC使用了很多先進的技術,除了可變區塊大小、多幅參考畫面和四分之一精確度之移動估測技術外,其中更利用碼率失真最佳化技術(rate distortion optimization, RDO)檢查所有的畫面間模式與畫面內模式,以達到最佳的編碼效能。然而這些先進的技術造成H.264/AVC的編碼複雜度遠大於先前所有的視訊編碼標準。為了降低H.264/AVC計算複雜度並維持其編碼影像品質,本論文分別進行快速模式決策演算法及快速多幅參考畫面演算法之研究。在第二章中,進行畫面間模式決策之研究,我們利用8×8及4×4零區塊以判別該巨區塊之移動特性(stationary/non-stationary)並提出先進零區塊模式決策演算法(advanced zero-block inter mode decision)。在第三章中,我們提出可選擇性多幅參考畫面移動估測演算法(selective multiple reference frames motion estimation),其中將多幅參考畫面視為穩定的高斯隨機程序,並利用其穩定特性以判別該巨區塊是否需要繼續參考下一張參考畫面(reference frames)。另外,提出兩種提前中止參考畫面移動估測的演算法(early termination),進一步降低其運算複雜度。在第四章中,我們提出快速畫面內模式決策(intra mode decision)及畫面內模式預測(intra mode prediction)演算法,首先利用該巨區塊之離散餘弦係數中的低頻成份以進行畫面內模式預測(intra prediction);接著利用畫面內模式預測中之殘餘值資訊,包括絕對轉換誤差及變異數,以減少畫面內模式預測中方向性候選模式(candidates)。由各章節實驗結果可以發現,我們針對畫面間模式決策、多幅參考畫面之移動估測及畫面內模式決策,所提出的演算法皆可以有效地降低H.264/AVC編碼之運算複雜度,並且維持一定的影像品質。The H.264/AVC video coding standard achieves significantly better performance in both PSNR and visual quality at the same bit-rate compared to prior video coding standards. This is due to the fact that the H.264/AVC encoder features many advanced techniques such as variable block size motion estimation, multiple reference frames motion estimation (MRFME) and quarter-pixel motion estimation. One important technique is the uses of Lagrangian rate-distortion optimization (RDO) for inter mode decision as well as intra mode decision. The RDO technique is used to check all possible inter modes and intra modes to find the best coding result to obtain highest coding efficiency, but the computational load is far beyond prior video coding standards.To reduce the computational complexity and maintain good coding performance, in this dissertation we investigate and develop efficient algorithms for H.264/AVC video coding. In Chapter 2 many efficient methods for inter mode decision have been investigated. We make use of both 8×8 and 4×4 zero-blocks to describe the temporal stationary or non-stationary for video sequences and propose an advanced zero-block inter mode decision algorithm. In Chapter 3, we propose a selective multiple reference frames motion estimation (SMRFME), which characterizes MRFME as a stationary Gaussian random process, and uses the stationary property to check whether a mode or a block is necessary to perform motion estimation on next reference frames. In addition, two early stop criteria to further lessen computational cost. In Chapter 4, we suggest an efficient algorithm which is accomplished in two stage. In the first stage the low-frequency AC components of discrete cosine transform (DCT) block of an MB is used to select intra 4×4 prediction (I4MB) or intra 16×16 prediction (I16MB); while in the second stage sum of absolute transform difference (SATD) coefficients, including SATD value and its variance, is used to skip improper mode for RDO mode decision. The experimental results show that our proposed algorithms achieve significant computation improvement when compared to other distinct algorithms, while maintaining good coding performance.