Object Video Compression.
OBJECT VIDEO COMPRESSION
LAI FENG TIAN
(2)
Saya LAI FENG TIAN
mengaku membenarkan tesis (PSM/Sarjana/Doktor Falsafah) ini disimpan di Perpustakaan Fakulti Teknologi Maklumat dan Komunikasi dengan syarat-syarat kegunaan seperti berikut:
1. Tesis dan projek adalah hakmilik Universiti Teknikal Malaysia Melaka.
2. Perpustakaan Fakulti Teknologi Maklumat dan Komunikasi dibenarkan membuat salinan untuk tujuan pengajian sahaja.
3. Perpustakaan Fakulti Teknologi Maklumat dan Komunikasi dibenarkan membuat salinan tesis ini sebagai bahan pertukaran antara institusi pengajian tinggi.
4. ** Sila tandakan (/)
__________ SULIT (Mengandungi maklumat yang berdarjah keselamatan atau kepentingan Malaysia seperti yang termaktub di dalam AKTA RAHSIA RASMI 1972)
__________TERHAD (Mengandungi maklumat TERHAD yang telah ditentukan oleh organisasi/badan di mana penyelidikan dijalankan)
√ TIDAK TERHAD
___________________________ ___________________________ (TANDATANGAN PENULIS) (TANDATANGAN PENYELIA)
Alamat tetap: No.42, Jalan Daya 26, DR NUR AZMAN BIN ABU Taman Daya, kepong, 52100,
Kuala Lumpur. Nama Penyelia
Tarikh : ___________________ Tarikh: ____________________
CATATAN: * Tesis dimaksudkan sebagai Laporan Akhir Projek Sarjana Muda (PSM)
** Jika tesis ini SULIT atau TERHAD, sila lampirkan surat daripada pihak berkuasa
(3)
OBJECT VIDEO COMPRESSION
LAI FENG TIAN
This report is submitted in partial fulfilment of the requirements for the Bachelor of Computer Science (Computer Networking)
FACULTY OF INFORMATION AND COMMUNICATION TECHNOLOGY UNIVERSITI TEKNIKAL MALAYSIA MELAKA
(4)
DECLARATION
I hereby declare that this project entitled
OBJECT VIDEO COMPRESSION
is written by me and is my own effort and that no part has been plagiarized without citations
STUDENT: DATE: (LAI FENG TIAN)
SUPERVISOR: DATE:
(5)
DEDICATION
(6)
ACKNOWLEDGEMENT
First and foremost, I would like to take this opportunity to express my deepest appreciation to my dearest supervisor, Dr Nur Azman bin Abu. Sir, you had truly inspired me with your valuable guidance, knowledge and advices throughout the whole process of this final year project. Your willingness and commitment has been motivated me to work harder in order to finish this final year project on time. Moreover, your patience and understanding had already helped me to solve all the problems I had been faced from time to time. Sir, you did not just help me to solve but also gave me sufficient time and chances to modify and repair my weaknesses in this project. Therefore, I would like to say a million thank to you Sir.
Secondly, I would like to thank my evaluator for this project, Encik Othman bin Mohd. You had already evaluated my project and point out the weaknesses found in my final year project. Therefore, I appreciate your opinions and guidance for me to improvise my project.
Last but not least, I would like to say thank you to all my friends and family members who always give me support and understanding during the completion of this project. Without all of the helps mentioned above, I would definitely not able to complete my final year project.
(7)
ABSTRACT
Video Compression is one of the fields in Video Processing. The main purpose of Video Compression is to compress the file size in terms of reduce and remove reductant video data so that it can be sent over a network easily. Video Compression is divided into two types which are lossless and lossy. This project studies on the analysis of the lossy Video Compression by using moving frame coding. It is done by using Discrete Cosine Transform (DCT). The analysis of the proposed technique is to determine the compression of a video. The analysis is done by using the video frames qualities metrics such as Absolute Reconstruction Error (ARE), Mean Square Error (MSE), Peak Signal-to-Noise (PSNR) and structural similarity (SSIM).
(8)
ABSTRAK
Pemampatan Video merupakan salah satu bidang dalam Pemprosean Video. Tujuan utama Pemampatan Video adalah untuk memampat saiz file dari segi mengurang dan membuang data video supaya video dapat dihantar melalui internet dengan mudah. Pemampatan Video dibahagi kepada dua jenis iaitu lossless dan lossy. Kajian projek ini merupakai analisis Pemampatan Video secara lossy dengan menggunkan moving frame coding. Projek ini adalah dijalankan dengan menggunakan Discrete Cosine Transform (DCT). Analisa kajian projek ini adalah perbandingan antara dua teknik ini untuk menentukan teknik yang mana lebih sesuai untuk memampat satu video. Analisa ini adalah dijalankan dengan menggunakan quality rangka video seperti Absolute Reconstruction Error (ARE), Mean Square Error (MSE), Peak Signal-to-Noise (PSNR) dan Structural Similarity (SSIM).
(9)
TABLE OF CONTENTS
CHAPTER SUBJECT PAGE
DECLARATION i
DEDICATION ii
ACKNOWLEDGEMENT iii
ABSTRACT iv
ABSTRAK v
TABLE OF CONTENTS vi
LIST OF TABLES ix
LIST OF FIGURES x
LIST OF ABBREVIATIONS xiii
CHAPTER I INTRODUCTION
1.1 Project Background 2
1.2 Problem Statement 3
1.3 Research Question 4
1.4 Objective 5
1.5 Scope 6
1.6 Project Significant 6
1.7 Expected Results 7
1.8 Summary 7
CHAPTER II LITERATURE REVIEW
(10)
2.2.1 Video Compression
2.2.2 Frame Difference Approaches 2.2.3 Experimental Quality Metrics 2.2.3.1 Mean Square Error (MSE) 2.2.3.2 Peak Signal-to-Noise ratio (PSNR)
2.2.3.3 Similarity Index (SSIM) 2.2.3.4 Discrete Cosine Transform 2.3 Analysis of current problem, justification 2.4 Proposed Solution
2.5 Summary
10 11 13 14 15 16 16 17 17 18
CHAPTER III METHODOLOGY
3.1 Introduction
3.2 Project Methodology
3.3 Project Schedule and Milestones 3.4 Summary
19 19 24 27
CHAPTER IV IMPLEMENTATION
4.1 Introduction
4.2 Project Requirement
4.2.1 Software Requirement 4.2.2 Hardware Requirement 4.3 Training Video and Frames 4.3.1 Frames A
4.3.2 Frame Difference B 4.4 Coding Strategy
4.5 Sub-Block Coding Strategy 4.6 Summary
28 28 29 30 31 31 33 36 37 42
(11)
CHAPTER V TESTING 5.1 Introduction 5.2 Pre-processing 5.3 Processing
5.4 Quality Factor Determination
5.4.1 Mean Square Error (MSE) and Peak Signal-to-Noise (PSNR)
5.5 Accuracy Accessing
5.5.1 Overall Flowchart of Testing Process of Video Compression
5.5.2 Capture Front Moving Object, Frame C 5.5.3 8x8 Block Coding
5.6 Final Output
5.6.1 Movement from -1 to 1 5.6.2 Movement from -16 to 16 5.6.3 Discussion of the Result 5.7 Summary
43 43 45 46 49 50 58 58 59 61 62 62 68 74 74
CHAPTER VI PROJECT CONCLUSION
6.1 Introduction
6.2 Summarization of Objectives 6.3 Future Work
6.4 Contribution 6.5 Summary
75 75 75 76 77 77 REFERENCES APPENDIX A 78 81
(12)
LIST OF TABLES
TABLE TITLE PAGE
1.1 Summary of Problem Statement 3
1.2 Summary of Research Questions 4
1.3 Summary of Research Objectives 5
2.1 Quality Factor for 48 Real Images 13 2.2 Quality Factor for 48 Graphical Images 14
3.1 Activities of PSM 1 and PSM 2 25
4.1 Software Requirements 29
4.2 Hardware Requirement 30
4.3 Description of vector (i, j) 39
5.1 Average value of Mean Square Error (MSE) and Peak Signal-to-Noise (PSNR) for first 100 frames A
51 5.2 Average value of Mean Square Error (MSE) and Peak
Signal-to-Noise (PSNR) for first 100 frame difference, B
54 5.3 Percentage of value “0” in both movement of -1 to 1 and
-16 to 16
(13)
LIST OF FIGURES
FIGURE TITLE PAGE
2.1 Video Compression Summary Chart 9
2.2 MPEG-4 Frame Sequence Order 11
2.3 Structural Diagram of Frame Difference Approach 12 2.4 Two Consecutive Frames and Their True Motion 12 2.5 Equation of Mean Square Error (MSE) 14
2.6 Equation of PSNR 15
3.1 A Taxonomy of Image and Video Compression Methods 20 3.2 Five Main Stages of the Project 21 3.3 Algorithm to reconstruct frame A 22
3.4 Flow Chart of the Project 23
4.1 The first 20 frames A of the video P 31 4.2 The 256th to 275th frames A of the video P 32 4.3 Frames of video P (a) frame 54th; (b) frame 55th; (c) frame
229th; (d) frame 230th
33 4.4 The first 20 frame difference B of video P 34 4.5 The 256th to 275th frame difference B of video P 34 4.6 Frame Difference B of subtraction by frame A (a) 54th
and 55th (b) 229th and 230th
35 4.7 Number of Frames in One Second with Size 240 by 320 36
4.8 Structure of Frames A, B and C 36
4.9 Code Block of 240 x 320 37
4.10 (a) Sub-block of Frame 240x320 in 8x8, (b) Vector Motion in the Direction (i, j)
(14)
between (a) and (b)
5.1 Object Compression Testing Process 43 5.2 The first 20 frames A of the video P 45 5.3 The 256th to 275th frames A of the video P 46 5.4 The first 20 frame difference B of video P 47 5.5 The 256th to 275th frame difference B of video P 47 5.6 The first 20 frame BE of video P 48 5.7 The 256th to 275th frame BE of video P 49 5.8 Equation of Mean Square Error (MSE) 50 5.9 Process of Video Compression using Technique Discrete
Cosine Transform (DCT)
58 5.10 The first 20 frame C of video P 60 5.11 The first 20 frame C of video P 60 5.12 8x8 block of frame Ak and Ak + 1 61 5.13 Direction of highlighted yellow 61 5.14 Histogram of frame 1 and frame 2 in 3 blocks 62 5.15 Histogram of frame 2 and frame 3 in 3 blocks 63 5.16 Histogram of frame 3 and frame 4 in 3 blocks 63 5.17 Histogram of frame 4 and frame 5 in 3 blocks 64 5.18 Histogram of frame 5 and frame 6 in 3 blocks 64 5.19 Histogram of frame 273 and frame 274 in 3 blocks 65 5.20 Histogram of frame 274 and frame 275 in 3 blocks 65 5.21 Histogram of frame 275 and frame 276 in 3 blocks 66 5.22 Histogram of frame 276 and frame 277 in 3 blocks 66 5.23 Histogram of frame 276 and frame 277 in 3 blocks 67 5.24 Histogram of frame 1 and frame 2 in 33 blocks 68 5.25 Histogram of frame 2 and frame 3 in 33 blocks 69 5.26 Histogram of frame 3 and frame 4 in 33 blocks 69 5.27 Histogram of frame 4 and frame 5 in 33 blocks 70 5.28 Histogram of frame 5 and frame 6 in 33 blocks 70 5.29 Histogram of frame 273 and frame 274 in 33 blocks 71 5.30 Histogram of frame 274 and frame 275 in 33 blocks 71
(15)
5.31 Histogram of frame 275 and frame 276 in 33 blocks 72 5.32 Histogram of frame 276 and frame 277 in 33 blocks 72 5.33 Histogram of frame 277 and frame 278 in 33 blocks 73
(16)
LIST OF ABBREVIATIONS
TERM EXPLANATION
ARE Absolute Reconstruct Error
MSE Mean Square Error
PSNR Peak Signal-to-Noise SSIM Similarity Index
DCT Discrete Cosine Transform
RP Research Problem
RQ Research Question
(17)
CHAPTER I
INTRODUCTION
Nowadays, video compression plays an important role in today’s society especially in the field of multimedia. People like to download the video or movie online. If the file size is big or in other word, the resolution of the video is higher, the time to download particular video is getting longer. In order to overcome this problem, video compression technology is used in order to reduce and remove reductant video data so that a digital video file can be sent over a network easily. There are two types of compression such as lossy and lossless. The quality of the video will be affected if the compression level getting higher. The video compression techniques plays an important role since it can maintain the quality of the video and reduce the file size simultaneously. The expected outcome of this project is to find out the best technique, analyse the procedure of two techniques used so that we can identify which technique is the most suitable for the video compression with less quality affected. To investigate further, the analysis was conducted using Discrete Cosine Transform (DCT).
(18)
In this era, image processing and video processing has been developed and gradually improving. Video processing is related to images since video is the combination of images and audio. Besides that, quality of a video plays an important role to user as in the higher the quality of a video, the higher the users’ view and rate towards that particular video. As the quality of the video degraded or in other words, low resolution of particular video, the frames of the video seem to be blurred. Hence, the view and rate of this video will be decreased dramatically. There are several video compression technique such as Flow Control and Buffering, Temporal Compression, Spatial Compression, Discrete Cosine Transform (DCT), Vector Quantization (VQ), Fractal Compression, Discrete Wavelet Transform, Tchebichef Moment Transform (TMT) and so on.
When we want to compress a video or a sequence of images, we have an added dimension to exploit which is known as temporal dimension. Generally, there is just a slightly change in the spatial arrangement of objects between two or more consecutive frames in a video. Hence, it is advantageous to send or store the differences between consecutive frames rather than sending or storing each frame. The difference frame is called the residual or differential frame and it may consists far less details than the actual frame itself. Compression is achieved due to this reduction in the details in the differential frames.
In this project, an analysis of the video compression is conducted by comparing the movement of the frames in the video, differences of the frames in the video and the edge detection of the frames in each categories (differences of the frames accordingly). All the graphs are plotted by using Matlab.
(19)
1.2 Problem Statement
Video compression plays an important role in today’s society. It is a common video processing technology used to compress the video into a smaller file size without degrading the quality of the video. In order to justify and determine a specific technique to be applied on the video is not easy. Therefore, a few research problems are listed as shown in Table 1.1.
Table 1.1: Summary of Problem Statement RP Research Problem
RP1 Difficulty to separate front moving object and static background scene of video in each proposed techniques.
RP2 Difficulty to apply full independent video frames in the project.
RP3 Difficulty to compare the final output after compression has been done. As we can see from Table 1.1, there are three research problems were conducted to clarify for the research question. The elaboration for each of the Research Problem (RP) is explained as follow.
RP1: Difficulty to separate front moving object and static background scene of each proposed techniques.
The ideal and knowledge of the proposed techniques were yet to be discovered. Therefore, it is difficult to identify the front moving object and static background scene of video in each proposed techniques.
(20)
We need to find a video which is in the format of .avi. However, almost all of the video which can be downloaded is in the format of .mp4 which means it had already compressed. Therefore, there are some error in the processing of data. We need to determine the frame difference and construct the frame again then combine into a video.
RP3: Difficulty to compare the final output after compression has been done. The comparison of final output will not carry out without both RP1 and RP2.
1.3 Research Question
Based on the research problems which shown in Table 1.1, the research questions were identified as shown in Table 1.2 as shown below.
Table 1.2: Summary of Research Questions RP RQ Research Questions
RP1 RQ1 What are the uses of each proposed techniques?
RP2 RQ2 Which type of resolution, frame rate and frame size of video is the best to use in this analysis?
RP3 RQ3 How to compare the final output after the compression has been done?
The elaborations for each of the Research Questions (RQ) are discussed. RQ1: What are the uses of each proposed techniques in terms of separating front moving object and static background scene?
This research question is to analyse the use for each of the proposed techniques used in this project in order to separate front moving object and static background scene.
(21)
RQ2: How to apply full independent video frames for the techniques used? This research question is to identify and apply the independent video frames for each of the techniques used in this project.
RQ3: How to compare the final output after the compression has been done? This research question is to compare the final output after the compression has been done.
1.4 Objective
According to the research problems and research questions stated in section 1.2 and 1.3, there are three objectives are conducted as shown in Table 1.3.
Table 1.3: Summary of Research Objectives RP RQ RO Research Objectives
RP1 RQ1 RO1 To separate front moving object and static background scene of a video in proposed technique.
RP2 RQ2 RO2 To apply full independent video frames into a video by using the proposed technique.
RP3 RQ3 RO3 To compare the final output after compression has been done. Further explanation for each of the Research Objectives (RO) is explained as follow:
RO1: To separate front moving object and static background scene of a video in each proposed techniques.
This research objective is to separate the front moving object and static background scene of a video of proposed techniques.
(22)
techniques.
This research objective is to apply the full independent video frames into a video by using the proposed techniques.
RO3: To compare the final output after compression has been done.
This research objective is to make a comparison on the final output after the compression process has been done.
1.5 Scope
This project is conducted to study on the video compression techniques used which are Discrete Cosine Transform (DCT). A graphic video is chosen which is used in this project. This video is downloaded from www.youtube.com (Pigeons – cute animation cartoon). The format of this video will be converted into avi format. The length of this video is around 4 minutes and therefore it is shorten into 9 seconds in order to conduct the analysis. This video has
Frame rate: 30 fps (frames per second)
Frame height: 240 pixels
Frame width: 320 pixels
The software that was used to perform the following study was MATLAB2014b.
1.6 Project Significant
The main purpose of this project was to compress the video into a smaller file size but remain the quality of the video. It is mainly to find out the proposed techniques work on the video compression such as separating the front moving objects, static background scene and also the full independent frames. At the end of the project, the comparison between the video before and after is done.
(23)
1.7 Expected Result
By the end of this project, the expected results have to be achieved with the stated objectives of this project:
i. The functions of the proposed techniques are defined.
ii. The result to get the full independent video frames of a video.
iii. The comparison for proposed technique is clarified after the output is produced.
1.8 Summary
In conclusion, the problem statement, objective and scope are well stated in this chapter. Hence, literature reviews on the activities contributed to the related work were conducted in the next chapter as well as the analysis of current problem and justification along with their proposed solution.
(24)
CHAPTER II
LITERATURE REVIEW
2.1 Introduction
In this chapter, the previous related work on this project field will be discussed. There are some important term with its definition that were used throughout the whole project will be specified. Besides that, there are several topics that will be explained in this chapter in order to comprise the analysis of current problem and justification included the proposed solutions.
The related work will be discussed in more detail way in the first part of the topic. It consists of three sub topics to discuss about the definition of video compression; whether it is lossless or lossy, the front moving object and background static scene. The second topic discussed in this chapter is the analysis of problem and justification that were found in this project. Last but not least, the proposed solutions were justified too in order to solve those problems.
(1)
1.2 Problem Statement
Video compression plays an important role in today’s society. It is a common video processing technology used to compress the video into a smaller file size without degrading the quality of the video. In order to justify and determine a specific technique to be applied on the video is not easy. Therefore, a few research problems are listed as shown in Table 1.1.
Table 1.1: Summary of Problem Statement RP Research Problem
RP1 Difficulty to separate front moving object and static background scene of video in each proposed techniques.
RP2 Difficulty to apply full independent video frames in the project.
RP3 Difficulty to compare the final output after compression has been done. As we can see from Table 1.1, there are three research problems were conducted to clarify for the research question. The elaboration for each of the Research Problem (RP) is explained as follow.
RP1: Difficulty to separate front moving object and static background scene of each proposed techniques.
The ideal and knowledge of the proposed techniques were yet to be discovered. Therefore, it is difficult to identify the front moving object and static background scene of video in each proposed techniques.
(2)
RP2: Difficulty to apply full independent video frames in the project.
We need to find a video which is in the format of .avi. However, almost all of the video which can be downloaded is in the format of .mp4 which means it had already compressed. Therefore, there are some error in the processing of data. We need to determine the frame difference and construct the frame again then combine into a video.
RP3: Difficulty to compare the final output after compression has been done. The comparison of final output will not carry out without both RP1 and RP2.
1.3 Research Question
Based on the research problems which shown in Table 1.1, the research questions were identified as shown in Table 1.2 as shown below.
Table 1.2: Summary of Research Questions RP RQ Research Questions
RP1 RQ1 What are the uses of each proposed techniques?
RP2 RQ2 Which type of resolution, frame rate and frame size of video is the best to use in this analysis?
RP3 RQ3 How to compare the final output after the compression has been done?
The elaborations for each of the Research Questions (RQ) are discussed. RQ1: What are the uses of each proposed techniques in terms of separating front moving object and static background scene?
This research question is to analyse the use for each of the proposed techniques used in this project in order to separate front moving object and static background scene.
(3)
RQ2: How to apply full independent video frames for the techniques used? This research question is to identify and apply the independent video frames for each of the techniques used in this project.
RQ3: How to compare the final output after the compression has been done? This research question is to compare the final output after the compression has been done.
1.4 Objective
According to the research problems and research questions stated in section 1.2 and 1.3, there are three objectives are conducted as shown in Table 1.3.
Table 1.3: Summary of Research Objectives RP RQ RO Research Objectives
RP1 RQ1 RO1 To separate front moving object and static background scene of a video in proposed technique.
RP2 RQ2 RO2 To apply full independent video frames into a video by using the proposed technique.
RP3 RQ3 RO3 To compare the final output after compression has been done. Further explanation for each of the Research Objectives (RO) is explained as follow:
RO1: To separate front moving object and static background scene of a video in each proposed techniques.
This research objective is to separate the front moving object and static background scene of a video of proposed techniques.
(4)
RO2: To apply full independent video frames into a video by using the proposed techniques.
This research objective is to apply the full independent video frames into a video by using the proposed techniques.
RO3: To compare the final output after compression has been done.
This research objective is to make a comparison on the final output after the compression process has been done.
1.5 Scope
This project is conducted to study on the video compression techniques used which are Discrete Cosine Transform (DCT). A graphic video is chosen which is used in this project. This video is downloaded from www.youtube.com (Pigeons – cute animation cartoon). The format of this video will be converted into avi format. The length of this video is around 4 minutes and therefore it is shorten into 9 seconds in order to conduct the analysis. This video has
Frame rate: 30 fps (frames per second) Frame height: 240 pixels
Frame width: 320 pixels
The software that was used to perform the following study was MATLAB2014b.
1.6 Project Significant
The main purpose of this project was to compress the video into a smaller file size but remain the quality of the video. It is mainly to find out the proposed techniques work on the video compression such as separating the front moving objects, static background scene and also the full independent frames. At the end of the project, the comparison between the video before and after is done.
(5)
1.7 Expected Result
By the end of this project, the expected results have to be achieved with the stated objectives of this project:
i. The functions of the proposed techniques are defined.
ii. The result to get the full independent video frames of a video.
iii. The comparison for proposed technique is clarified after the output is produced.
1.8 Summary
In conclusion, the problem statement, objective and scope are well stated in this chapter. Hence, literature reviews on the activities contributed to the related work were conducted in the next chapter as well as the analysis of current problem and justification along with their proposed solution.
(6)
CHAPTER II
LITERATURE REVIEW
2.1 Introduction
In this chapter, the previous related work on this project field will be discussed. There are some important term with its definition that were used throughout the whole project will be specified. Besides that, there are several topics that will be explained in this chapter in order to comprise the analysis of current problem and justification included the proposed solutions.
The related work will be discussed in more detail way in the first part of the topic. It consists of three sub topics to discuss about the definition of video compression; whether it is lossless or lossy, the front moving object and background static scene. The second topic discussed in this chapter is the analysis of problem and justification that were found in this project. Last but not least, the proposed solutions were justified too in order to solve those problems.