視訊編碼廣泛應用於電視傳播、網路串流、監視系統、視訊會議、多媒體裝置…等等,是現今生活中不可或缺的一環。在新制定的H.264/AVC編碼標準中,增加了許多多樣化的編碼技術,相較於MPEG-2、H.263等過去的視訊編碼標準,H.264/AVC提供更佳的編碼效能,提高壓縮率並增進畫面品質;但所付出的代價是因應而來的巨大運算量,使得H.264/AVC無法在即時編碼的應用環境中輕易地實現。因此,如何有效降低編碼的運算量,並維持應有的壓縮品質,是重要且熱門的研究主題。本研究針對H.264/AVC之畫面內編碼與畫面間編碼,分別提出有效的快速模式選擇演算法,不但大幅度的降低了編碼所需的運算時間,更保持了相當優越的壓縮品質。經過實驗,本研究所提出的方法可在畫面內編碼與畫面間編碼中,分別帄均節省約60%與46%的編碼時間,並能維持非常接近H.264/AVC標準之壓縮品質。Video coding has been widely applied to TV transmission, network stream, monitoring system, video conferencing, and multimedia devices. In the video coding standard H.264/AVC, several coding techniques are added. Compared to old Standards such as MPEG-2 and H.263, H.264/AVC provides better coding efficiency. It raises the compression ability and enhances the quality of pictures. But it pays a huge computation time for it, making H.264/AVC cannot be realized easily in the real time coding environment. So, reducing the huge coding complexity with good compression quality has been a hot issue. This thesis focuses on speeding up both the inter coding and intra coding procedures. We propose two efficient mode decision algorithms for intra coding and inter coding. They greatly reduce the coding time but keep superior coding performance. After our experiments, the approaches we proposed speed up intra coding and inter coding by 60% and 46%, respectively. Besides, its coding performance is very close to original H.264/AVC.