## HomePage.EL-GY6123ImageAndVideoProcessing History

Hide minor edits - Show changes to markup

- Week 3 (2/9): Image sampling and resizing. Design of interpolation filters. Multiresolution representation: Pyramid and wavelets. Lecture note on ImageSampling, Lecture note on Wavelet. Δ

- Week 3 (2/9): Image sampling and resizing. Design of interpolation filters. Multiresolution representation: Pyramid and wavelets. Lecture note on ImageSampling (updated 2/8/18), Lecture note on Wavelet Δ (updated 2/8/2018).

Link(Updated 02/06/2018)

Link (Updated 02/06/2018)

Link (Updated 02/06/2018)

Link(Updated 02/06/2018)

Link (Updated 02/06/2018)

Link (Updated 02/06/2018)

Link (Updated 02/06/2018)

Link (Updated 02/06/2018)

Amirhossein Khalilian-Gourtani: TBA

Amirhossein Khalilian-Gourtani: Mon: 10:00 am - 11:30 am and Thur: 4:30 pm - 6:30 pm or by appointment

Chenge Li: Tue. 4:00-5:00 PM and Fri. 1:00-3:00 PM

Chenge Li: Tue. 4:00-5:00 PM and Fri. 3:00-4:00 PM or appointment by email

TAs: TBA

TAs:
Chenge Li: Tue. 4:00-5:00 PM and Fri. 1:00-3:00 PM

Amirhossein Khalilian-Gourtani: TBA

Lecture note. Part 2: Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization). Lecture note.

Lecture note (Updated 1/25/2018). Part 2: Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization). Lecture note (Updated 1/25/2018)

- Week 2 (2/2): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators. Lecture note.

- Week 2 (2/2): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators. Lecture note (Updated 2/1/2018).

- Computer assignment 1 (Learning Python and histogram equalization) (Due 2/5)

- Computer assignment 1 (Learning Python and histogram equalization) (Due 2/5)

- Computer assignment 1 (Learning Python and histogram equalization) (Due 2/5)

- Computer assignment 1 (Learning Python and histogram equalization) (Due 2/5)

Chenge Li (MTC2 Room 9.123, cl2840 at nyu dot edu) and Amirhossein Khalilian-Gourtani (MTC2 Room 9.130, akg404 at nyu dot edu)

Chenge Li (MTC2 Room 9.123, cl2840 at nyu dot edu) and Amirhossein Khalilian-Gourtani (MTC2 Room 9.130B, akg404 at nyu dot edu)

- Computer assignment 1 (Learning Python and histogram equalization) (Due 2/5) Lecture note.

- Computer assignment 1 (Learning Python and histogram equalization) (Due 2/5)

- Computer assignment 1 (Learning Python and histogram equalization) (Due 2/5)

- Computer assignment 1 (Learning Python and histogram equalization) (Due 2/5) Lecture note.

- Programming assignment 3 (Wavelet-based denoising) (Due 2/22): Programming _WaveletDenoising

- Programming assignment 3 (Wavelet-based denoising) (Due 2/22)

- Programming assignment 4 (Due 3/8): Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).
**Programming_features**

- Programming assignment 4 (Due 3/8): Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).

- Programming assignment 5 (Due 3/22):
**Image Segmentation.****Programming_Kmeans**

- Programming assignment 5 (Due 3/22):
**Image Segmentation.**

- Programming assignment 7 (Due 5/10):
**Video Coding Assignment**

- Programming assignment 7 (Due 5/10): Video Coding

- Computer assignment 1 (Due 2/2)

- Computer assignment 1 (Learning Python and histogram equalization) (Due 2/5)

- Computer assignment 2 (2D filtering) (Due 2/8)

- Programming assignment 2 (Due 3/2): Programming _WaveletDenoising

- Programming assignment 3 (Wavelet-based denoising) (Due 2/22): Programming _WaveletDenoising

- Programming assignment 3 (Due 3/9): Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).
**Programming_features**

- Programming assignment 4 (Due 3/8): Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).
**Programming_features**

- Programming assignment 4 (Due 3/23):
**Image Segmentation.****Programming_Kmeans**

- Programming assignment 5 (Due 3/22):
**Image Segmentation.****Programming_Kmeans**

- Programming assignment 5 (Due 4/6): Moving object detection.

- Programming assignment 6 (Due 4/5): Moving object detection.

- Programming assignment 6 (Due 5/11):
**Video Coding Assignment**

- Programming assignment 7 (Due 5/10):
**Video Coding Assignment**

- Computer assignment 1 (Due 2/2): CA1

- Computer assignment 1 (Due 2/2)

- Computer assignment 1 (Due 2/2): CA1

- Programming assignment 1 (Due 2/9): Programming_Filtering

- Tutorial on python (Time /location to be arranged) Python Tutorial files Δ

- Tutorial on python (Time /location to be arranged)

- Tutorial on python (Time /location to be arranged)

- Tutorial on python (Time /location to be arranged) Python Tutorial files Δ

- Intro to Python: Basics of Python and Its Application to Image Processing Through OpenCV:Link

- Basics of Python and Its Application to Image Processing Through OpenCV: Link

- Intro to Python: First three slides of the Python course at Columbia:Link

- Intro to Python: Basics of Python and Its Application to Image Processing Through OpenCV:Link

Yao Wang: Mon. 4-6, and Wed. 4-6PM or appointment by email.

TAs: Thur. 2-5 PM

Yao Wang: Wed. 4- 5:30PM and Fri. 10:30-12:00 or appointment by email.

TAs: TBA

Link To be updated.

Link (To be updated).

- Programming assignment 3 (Due 3/9): Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).
**Programming_features'''**

- Programming assignment 3 (Due 3/9): Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).
**Programming_features**

Late submission of written assignment and programming assignment will not be accepted unless under extraordinary circumstances and must be approved in advance by the instructor. Students can work in teams, but you must submit you own solutions. For programming assignment, please include your code (with documentation) and results (plots etc.) and discussions.

Written HW will be assigned after each lecture and due at the beginning of the following lecture time. Programming assignment will be due as posted. Late submission of written assignment and programming assignment will not be accepted unless under extraordinary circumstances and must be approved in advance by the instructor. Students can work in teams, but you must submit you own solutions. For programming assignment, please include your code (with documentation) and results (plots etc.) and discussions.

Link To be updated.

- Programming assignment 3 (Due 3/9):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).****Programming_features**

- Programming assignment 3 (Due 3/9): Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).
**Programming_features'''**

- Programming assignment 1 (Due 2/9): Linear filtering for noise removal and image sharpening and illustrating the effect in both spatial and frequency

domain. Programming_Filtering

- Programming assignment 1 (Due 2/9): Programming_Filtering

- Week 5 (2/23): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC)
**Lecture note.**

- Week 5 (2/23): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC).
**Lecture note.**

- Programming assignment 1 (Due 2/6): '''Linear filtering for noise removal and image sharpening and illustrating the effect in both spatial and frequency

domain.''' Programming_Filtering

- Programming assignment 1 (Due 2/9): Linear filtering for noise removal and image sharpening and illustrating the effect in both spatial and frequency

domain. Programming_Filtering

- Programming assignment 2 (Due 3/6): Programming _WaveletDenoising
- Week 5 (2/27): Project Proposal Due
- Week 5 (2/23): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC)
**Lecture note** - Programming assignment 3 (Due 3/20):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).****Programming_features** - Week 6 (3/2): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization
**Lecture note** - Week 7 (3/9): Image segmentation: region growing, split and merge, Otsu’s method, K-means, GMM clustering.
*'*'Lecture note. - Programming assignment 4 (Due 3/27):
**Image Segmentation.****Programming_Kmeans**

- Programming assignment 2 (Due 3/2): Programming _WaveletDenoising
- Week 5 (2/23): Project Proposal Due
- Week 5 (2/23): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC)
**Lecture note.** - Programming assignment 3 (Due 3/9):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).****Programming_features** - Week 6 (3/2): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization.
**Lecture note** - Week 7 (3/9): Image segmentation: region growing, split and merge, Otsu’s method, K-means, GMM clustering.
**Lecture note.** - Programming assignment 4 (Due 3/23):
**Image Segmentation.****Programming_Kmeans**

- Programming assignment 5 (Due 4/10):
**Moving object detection****Programming_Moving object detection**

- Programming assignment 5 (Due 4/6): Moving object detection.
**Programming_Moving object detection**

- Programming assignment 6 (Due 5/17):
**Video coding.****Video Coding Assignment**

- Programming assignment 6 (Due 5/11):
**Video Coding Assignment**

domain.'' 'Programming_Filtering

domain.''' Programming_Filtering

**Links to Resources (lecture notes and sample exams) in Previous Offerings: **

**Links to Resources (lecture notes and sample exams) in Previous Offerings: **

- The coursera image processing course by Prof. Katsaggelos:Link
- The image processing course at Stanford:Link
- The computer vision course at U. Washington:Link

- The coursera image processing course by Prof. Katsaggelos: Link
- The image processing course at Stanford: Link
- The computer vision course at U. Washington: Link

- Intro to Python: First three slides of the Python course at Columbia:Link

- Intro to Python: First three slides of the Python course at Columbia:Link

**Project Guideline: **

**Project Guideline: **

**Suggested Project List: **

**Suggested Project List: **

Mon. 4-6, and Wed. 4-6PM or appointment by email.

Thur. 2-5 PM (TA's office hours)

Yao Wang: Mon. 4-6, and Wed. 4-6PM or appointment by email.

TAs: Thur. 2-5 PM

Late submission of written assignment and programming assignment will not be accepted unless under special circumstances and must be approved in advance by the instructor. Students can work in teams, but you must submit you own solutions. For programming assignment, please include your code (with documentation) and results (plots etc.) and discussions.

Late submission of written assignment and programming assignment will not be accepted unless under extraordinary circumstances and must be approved in advance by the instructor. Students can work in teams, but you must submit you own solutions. For programming assignment, please include your code (with documentation) and results (plots etc.) and discussions.

Project Guideline

Project List

Sample Images

Middelbury Stereo Image Database

**Project Guideline: **

Link**Suggested Project List: **

Link**Sample Data: **

Sample Images

Middelbury Stereo Image Database

Professor Yao Wang, MTC2 Room 9.122, (646)-997-3469, Email: yaowang at nyu dot edu.

```
Homepage
```

Professor Yao Wang, MTC2 Room 9.122, (646)-997-3469, Email: yaowang at nyu dot edu. Homepage

Professor Yao Wang, MTC2 Room 9.122, (646)-997-3469, Email: yaowang at nyu dot edu.Homepage

Professor Yao Wang, MTC2 Room 9.122, (646)-997-3469, Email: yaowang at nyu dot edu.

```
Homepage
```

Yilin Song (MTC2 Room 9.123, ys1297 at nyu dot edu) and Anti-Chiang (MTC2 Room 9.123, atc327 at nyu dot edu)

Chenge Li (MTC2 Room 9.123, cl2840 at nyu dot edu) and Amirhossein Khalilian-Gourtani (MTC2 Room 9.130, akg404 at nyu dot edu)

Monday 12:25 AM-2:55PM, Room RH215.

Friday 12:25 AM-2:55PM, Room RH215.

Wed. 4-6PM and Thur 4-6 PM or appointment by email.

Mon. 4-6, and Wed. 4-6PM or appointment by email.

Exam: 40%, Final Project: 30%, Programming assignments: 30%, Written homework will be assigned but not graded. Solution to written HW will be provided.

Exam: 40%, Final Project: 30%, Programming assignments: 20%, Written assignments: 10%.

Late submissions of programming assignment will be accepted up to 7 days after the deadline, with penalty of 1 pt for each day (Each assignment has 10pt). Students can work in teams (at most 3 people) and each team should submit one completed assignment. You should include your code (with documentation) and results (plots etc.) and discussions. When working in teams, each person must understand what others did and what are included in the assignment!

Late submission of written assignment and programming assignment will not be accepted unless under special circumstances and must be approved in advance by the instructor. Students can work in teams, but you must submit you own solutions. For programming assignment, please include your code (with documentation) and results (plots etc.) and discussions.

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancement

- Week 1 (1/26): Part 1: Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Lecture note. Part 2: Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization). Lecture note.

- Week 2 (1/30): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators.Lecture note on Fourier Transform and linear filtering
- Programming assignment 1 (Due 2/6):
**Linear filtering for noise removal and image sharpening and illustrating effect in both spatial and frequency domain.**Programming_Filtering - Week 3 (2/6): Image sampling and resizing. Design of interpolation filters. Multiresolution representation: Pyramid and wavelets.Lecture note on ImageSampling , Lecture note on Wavelet Δ
- Week 4 (2/13): Sparsity and dictionary based image processing: Image representation using orthonormal transform/dictionary. Sparsity assumption. General formulation of image enhancement as an optimization problem, L0 vs. L1 vs. L2 prior, Basic optimization techniques. Applications in debluring, denoising, inpainting, compressive sensing, superresolution, dictionary learning (PCA and KSVD).

- Week 2 (2/2): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators. Lecture note.
- Programming assignment 1 (Due 2/6): '''Linear filtering for noise removal and image sharpening and illustrating the effect in both spatial and frequency

domain.'' 'Programming_Filtering

- Week 3 (2/9): Image sampling and resizing. Design of interpolation filters. Multiresolution representation: Pyramid and wavelets. Lecture note on ImageSampling, Lecture note on Wavelet. Δ
- Week 4 (TBA): Sparsity and dictionary based image processing: Image representation using orthonormal transform/dictionary. Sparsity assumption. General formulation of image enhancement as an optimization problem, L0 vs. L1 vs. L2 prior, Basic optimization techniques. Applications in debluring, denoising, inpainting, compressive sensing, superresolution, dictionary learning (PCA and KSVD).

- 2/20/17 Presidents’ Day. No class.

- Week 5 (2/27): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC)
**Lecture note on Feature**

- Week 5 (2/23): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC)
**Lecture note**

- Week 6 (3/6): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization
**Lecture note on geometric transformation** - 3/13/17-3/19/17: Spring Recess
- Week 7 (3/20): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).
**Lecture note on image segmentation** - Programming assignment 4 (Due 3/27):
**Implement a specified segmentation method yourself. Compare different methods using available implementations.****Programming_Kmeans** - Week 8 (3/27): Midterm Project Feedback (Individual meeting)
- Week 8 (3/27): Dense motion/displacement estimation: optical flow equation, optical flow estimation; block matching, multi-resolution estimation. Deformable registration (medical applications)
**Lecture note on MotionEstimation** - Week 9 (4/3): Moving object detection (background/foreground separation) (Gaussian mixture model, RPCA). Simultaneous estimation of camera motion and moving objects. Object tracking. Video shot segmentation.
**Lecture note on Moving object detection**

- Week 6 (3/2): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization
**Lecture note** - Week 7 (3/9): Image segmentation: region growing, split and merge, Otsu’s method, K-means, GMM clustering.
*'*'Lecture note. - Programming assignment 4 (Due 3/27):
**Image Segmentation.****Programming_Kmeans** - 3/12/17-3/16/17: Spring Recess
- Week 8 (3/23): Midterm Project Feedback (Individual meeting)
- Week 8 (3/23): Dense motion/displacement estimation: optical flow equation, optical flow estimation; block matching, multi-resolution estimation. Deformable registration (medical applications).
**Lecture note.** - Week 9 (3/30): Moving object detection (background/foreground separation) (Gaussian mixture model, RPCA). Simultaneous estimation of camera motion and moving objects. Object tracking. Video shot segmentation.
**Lecture note.**

- Week 10 (4/10): Stereo and multiview video: depth from disparity, disparity estimation, view synthesis. Depth camera (Kinect). 360 video camera and view stitching. Light field imaging.
**Lecture note on Stereo and Multiview Video Processing** - Week 11 (4/17): Exam
- Week 12 (4/24): Fundamentals of source coding: characterization of random sources by entropy, binary encoding, scalar quantization, vector quantization.
**Lecture note on source coding basics.** - Week 13 (5/1): Transform coding: Image representation using unitary transforms (orthogonal bases), Transform coding, JPEG image compression standard, Image representation using wavelet transform; concept of layered coding, JPEG2000 image compression standard.
**Lecture note on unitary transforms and Transform Coding.**

- Week 10 (4/6): Stereo and multiview video: depth from disparity, disparity estimation, view synthesis. Depth camera (Kinect). 360 video camera and view stitching. Light field imaging.
**Lecture note.** - Week 11 (4/13): Exam
- Week 12 (4/20): Fundamentals of source coding: characterization of random sources by entropy, binary encoding, scalar quantization, vector quantization.
**Lecture note.** - Week 13 (4/27): Transform coding: Image representation using unitary transforms (orthogonal bases), Transform coding, JPEG image compression standard, Image representation using wavelet transform; concept of layered coding, JPEG2000 image compression standard.
**Lecture note.**

- Week 14 (5/8): Part 1: Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity.
**Lecture note on Video Coding. Part 2: Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. Error resilience issues. Adaptive Video Streaming (DASH).****Lecture note on Video Coding Standards.** - Week 15 (5/15): Project Presentation.
- 5/17: Project Report and all other material must be uploaded.

- Week 14 (5/4): Part 1: Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity.
**Lecture note. Part 2: Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. Error resilience issues. Adaptive Video Streaming (DASH).****Lecture note.** - Week 15 (5/11): Project Presentation.
- 5/18: Project Report and all other material must be uploaded.

- Week 12 (4/24): Fundamentals of source coding: characterization of random sources by entropy. binary encoding, scalar quantization, vector quantization.
**Lecture note on source coding basics.** - Week 13 (5/1): Transform coding: Image representation using unitary transforms (orthogonal bases). Transform coding. JPEG image compression standard. Image representation using wavelet transform; concept of layered coding. JPEG2000 image compression standard.
**Lecture note on unitary transforms and Transform Coding** - Programming assignment 6 (Due 5/8):
**Video coding.****Video Coding** - Week 14 (5/8): Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity.
**Lecture note on Block-Based Hybrid Video Coding****Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH).**''', Lecture note on Video Coding Standards

- Week 12 (4/24): Fundamentals of source coding: characterization of random sources by entropy, binary encoding, scalar quantization, vector quantization.
**Lecture note on source coding basics.** - Week 13 (5/1): Transform coding: Image representation using unitary transforms (orthogonal bases), Transform coding, JPEG image compression standard, Image representation using wavelet transform; concept of layered coding, JPEG2000 image compression standard.
**Lecture note on unitary transforms and Transform Coding.** - Programming assignment 6 (Due 5/17):
**Video coding.****Video Coding Assignment** - Week 14 (5/8): Part 1: Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity.
**Lecture note on Video Coding. Part 2: Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. Error resilience issues. Adaptive Video Streaming (DASH).****Lecture note on Video Coding Standards.**

- Week 12 (4/24): Image representation using unitary transforms (orthogonal bases). Transform coding. JPEG image compression standard. Image representation using wavelet transform; concept of layered coding. JPEG2000 image compression standard.
**Lecture note on unitary transforms and Transform Coding** - Week 13 (5/1): Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity.
**Lecture note on Video coding**

- Week 12 (4/24): Fundamentals of source coding: characterization of random sources by entropy. binary encoding, scalar quantization, vector quantization.
**Lecture note on source coding basics.** - Week 13 (5/1): Transform coding: Image representation using unitary transforms (orthogonal bases). Transform coding. JPEG image compression standard. Image representation using wavelet transform; concept of layered coding. JPEG2000 image compression standard.
**Lecture note on unitary transforms and Transform Coding**

- Week 14 (5/8): Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH)
**Lecture note on Block-Based Hybrid Video Coding****, Lecture note on Video Coding Standards**

- Week 14 (5/8): Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity.
**Lecture note on Block-Based Hybrid Video Coding****Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH).**''', Lecture note on Video Coding Standards

- 5/20: Project Report and all other material must be uploaded.

- 5/17: Project Report and all other material must be uploaded.

- Programming assignment 6 (Due 5/8):
**Video coding.**

- Programming assignment 6 (Due 5/8):
**Video coding.****Video Coding**

- Week 12 (4/24): Image representation using unitary transforms (orthogonal bases). Transform coding. JPEG image compression standard. Image representation using wavelet transform; concept of layered coding. JPEG2000 image compression standard.
**, Lecture note on unitary transforms and Transform Coding**

- Week 12 (4/24): Image representation using unitary transforms (orthogonal bases). Transform coding. JPEG image compression standard. Image representation using wavelet transform; concept of layered coding. JPEG2000 image compression standard.
**Lecture note on unitary transforms and Transform Coding**

- Week 12 (4/24): Image representation using unitary transforms (orthogonal bases). Transform coding. JPEG image compression standard. Lecture note(updated 4/10/2016Week 11 ( Image representation using wavelet transform; concept of layered coding. JPEG2000 image compression standard. Lecture note(updated 4/25/2016)

- Week 12 (4/24): Image representation using unitary transforms (orthogonal bases). Transform coding. JPEG image compression standard. Image representation using wavelet transform; concept of layered coding. JPEG2000 image compression standard.
**, Lecture note on unitary transforms and Transform Coding**

- Week 14 (5/8): Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH)
**Lecture note on Block-Based Hybrid Video Coding****, Lecture note on Video Coding Standards****, Lecture note on Transform Coding**

- Week 14 (5/8): Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH)
**Lecture note on Block-Based Hybrid Video Coding****, Lecture note on Video Coding Standards**

- Week 14 (5/8): Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH)

**Lecture note on Block-Based Hybrid Video Coding
****Lecture note on Video Coding Standards
****Lecture note on Transform Coding
**

- Week 14 (5/8): Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH)
**Lecture note on Block-Based Hybrid Video Coding****, Lecture note on Video Coding Standards****, Lecture note on Transform Coding**

- Week 13 (5/1): Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity.Lecture note(updated 4/25/2016)

- Week 13 (5/1): Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity.
**Lecture note on Video coding**

- Week 14 (5/8): Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH) Lecture note(updated 5/7/2016)

- Week 14 (5/8): Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH)

**Lecture note on Block-Based Hybrid Video Coding
****Lecture note on Video Coding Standards
****Lecture note on Transform Coding
**

- Week 10 (4/10): Stereo and multiview video: depth from disparity, disparity estimation, view synthesis. Depth camera (Kinect). 360 video camera and view stitching. Light field imaging.

- Week 10 (4/10): Stereo and multiview video: depth from disparity, disparity estimation, view synthesis. Depth camera (Kinect). 360 video camera and view stitching. Light field imaging.
**Lecture note on Stereo and Multiview Video Processing**

- S15 Final Exam solution
- S16_midterm solution
- S16 final exam solution

- S15 Final Exam solution
- S16_midterm solution
- S16 final exam solution

- S15_midterm_w_solution

- S15_midterm_w_solution

- S15_midterm_w_solution

- S15_midterm_w_solution

- Week 6 (3/6): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization
**Lecture note on geometric transformation'''**

- Week 6 (3/6): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization
**Lecture note on geometric transformation**

- Week 7 (3/20): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).
**Lecture note on image segmentation'''** - Programming assignment 4 (Due 3/27):
**Implement a specified segmentation method yourself. Compare different methods using available implementations.****Programming_Kmeans'''**

- Week 7 (3/20): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).
**Lecture note on image segmentation** - Programming assignment 4 (Due 3/27):
**Implement a specified segmentation method yourself. Compare different methods using available implementations.****Programming_Kmeans**

- Week 8 (3/27): Dense motion/displacement estimation: optical flow equation, optical flow estimation; block matching, multi-resolution estimation. Deformable registration (medical applications)
**Lecture note on MotionEstimation'''** - Week 9 (4/3): Moving object detection (background/foreground separation) (Gaussian mixture model, RPCA). Simultaneous estimation of camera motion and moving objects. Object tracking. Video shot segmentation.
**Lecture note on Moving object detection'''** - Programming assignment 5 (Due 4/10):
**Moving object detection****Programming_Moving object detection'''**

- Week 8 (3/27): Dense motion/displacement estimation: optical flow equation, optical flow estimation; block matching, multi-resolution estimation. Deformable registration (medical applications)
**Lecture note on MotionEstimation** - Week 9 (4/3): Moving object detection (background/foreground separation) (Gaussian mixture model, RPCA). Simultaneous estimation of camera motion and moving objects. Object tracking. Video shot segmentation.
**Lecture note on Moving object detection** - Programming assignment 5 (Due 4/10):
**Moving object detection****Programming_Moving object detection**

- Programming assignment 4 (Due 3/27):
**Programming_Kmeans**

- Programming assignment 4 (Due 3/27):
**Programming_Kmeans'''**

- Week 6 (3/6): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization

- Week 6 (3/6): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization
**Lecture note on geometric transformation'''**

- Week 7 (3/20): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).
- Programming assignment 4 (Due 3/27):

- Week 7 (3/20): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).
**Lecture note on image segmentation'''** - Programming assignment 4 (Due 3/27):
**Programming_Kmeans**

- Week 8 (3/27): Dense motion/displacement estimation: optical flow equation, optical flow estimation; block matching, multi-resolution estimation. Deformable registration (medical applications)
- Week 9 (4/3): Moving object detection (background/foreground separation) (Gaussian mixture model, RPCA). Simultaneous estimation of camera motion and moving objects. Object tracking. Video shot segmentation.
- Programming assignment 5 (Due 4/10):
**Moving object detection**

- Week 8 (3/27): Dense motion/displacement estimation: optical flow equation, optical flow estimation; block matching, multi-resolution estimation. Deformable registration (medical applications)
**Lecture note on MotionEstimation'''** - Week 9 (4/3): Moving object detection (background/foreground separation) (Gaussian mixture model, RPCA). Simultaneous estimation of camera motion and moving objects. Object tracking. Video shot segmentation.
**Lecture note on Moving object detection'''** - Programming assignment 5 (Due 4/10):
**Moving object detection****Programming_Moving object detection'''**

- Programming assignment 3 (Due 3/20):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).**

- Programming assignment 3 (Due 3/20):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).****Programming_features**

- Week 7 (3/20): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).
**Programming_features**

- Week 7 (3/20): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).

- Week 7 (3/20): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).

- Week 7 (3/20): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).
**Programming_features**

- Programming assignment 2 (Due 2/27): Programming _WaveletDenoising

- Programming assignment 2 (Due 3/6): Programming _WaveletDenoising

- Programming assignment 3 (Due 3/6):

- Programming assignment 3 (Due 3/20):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).**

- Programming assignment 4 (Due 3/27):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).**

- Programming assignment 4 (Due 3/27):

- Week 4 (2/13): Sparsity and dictionary based image processing: Image representation using orthonormal transform/dictionary. Sparsity assumption. General formulation of image enhancement as an optimization problem, L0 vs. L1 vs. L2 prior, Basic optimization techniques. Applications in debluring, denoising, inpainting, compressive sensing, superresolution, dictionary learning (PCA and KSVD).Lecture note on Sparse Representation

- Week 4 (2/13): Sparsity and dictionary based image processing: Image representation using orthonormal transform/dictionary. Sparsity assumption. General formulation of image enhancement as an optimization problem, L0 vs. L1 vs. L2 prior, Basic optimization techniques. Applications in debluring, denoising, inpainting, compressive sensing, superresolution, dictionary learning (PCA and KSVD).

- Week 5 (2/27): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).

- Week 5 (2/27): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC)
**Lecture note on Feature**

- Week 6 (3/6): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC)
**Lecture note on Feature**

- Week 6 (3/6): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization

- Week 7 (3/20): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization

- Programming assignment 4 (Due 3/27):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).**sample image

- Programming assignment 4 (Due 3/27):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).**

**Sample Images: **

sample image

- Programming assignment 4 (Due 3/27):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).**sample image'''

- Programming assignment 4 (Due 3/27):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).**sample image

- Week 6 (3/6): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC)

- Week 6 (3/6): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC)
**Lecture note on Feature**

- Programming assignment 4 (Due 3/27):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).**

- Programming assignment 4 (Due 3/27):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).**sample image'''

- Programming assignment 1 (Due 2/6):
**Linear filtering for noise removal and image sharpening and illustrating effect in both spatial and frequency domain.**Programming_Filtering.pdf

- Programming assignment 1 (Due 2/6):
**Linear filtering for noise removal and image sharpening and illustrating effect in both spatial and frequency domain.**Programming_Filtering

- Week 4 (2/13): Sparsity and dictionary based image processing: Image representation using orthonormal transform/dictionary. Sparsity assumption. General formulation of image enhancement as an optimization problem, L0 vs. L1 vs. L2 prior, Basic optimization techniques. Applications in debluring, denoising, inpainting, compressive sensing, superresolution, dictionary learning (PCA and KSVD).
- Programming assignment 2 (Due 2/27): '''Programming _WaveletDenoising

- Week 4 (2/13): Sparsity and dictionary based image processing: Image representation using orthonormal transform/dictionary. Sparsity assumption. General formulation of image enhancement as an optimization problem, L0 vs. L1 vs. L2 prior, Basic optimization techniques. Applications in debluring, denoising, inpainting, compressive sensing, superresolution, dictionary learning (PCA and KSVD).Lecture note on Sparse Representation
- Programming assignment 2 (Due 2/27): Programming _WaveletDenoising

- Programming assignment 2 (Due 2/27):
**Wavelet based image denoising**Programming _WaveletDenoising

- Programming assignment 2 (Due 2/27): '''Programming _WaveletDenoising

- Programming assignment 2 (Due 2/27):
**Wavelet based image denoising**

- Programming assignment 2 (Due 2/27):
**Wavelet based image denoising**Programming _WaveletDenoising

Wed. 4-6PM and Thur 4-6 PM or appointment by email.

Wed. 4-6PM and Thur 4-6 PM or appointment by email.

Thur. 2-5 PM (TA's office hours)

- Week 3 (2/6): Image sampling and resizing. Design of interpolation filters. Multiresolution representation: Pyramid and wavelets.

- Week 3 (2/6): Image sampling and resizing. Design of interpolation filters. Multiresolution representation: Pyramid and wavelets.Lecture note on ImageSampling , Lecture note on Wavelet Δ

- Programming assignment 1 (Due 2/6):

- Programming assignment 1 (Due 2/6):

- Programming assignment 1 (Due 2/6):

- Programming assignment 1 (Due 2/6):

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancement

Solution of Written Homework 1

Solution of Computer programming Exercises

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancement

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancement

Solution of Written Homework 1

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancement

Solution of Written Homework 1

Solution of Computer programming Exercises

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancementSolution of Written Homework 1

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancement

Solution of Written Homework 1

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancement,

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancementSolution of Written Homework 1

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancement

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation , Lecture note on contrast enhancement,

- Programming assignment 1 (Due 2/6):

- Programming assignment 1 (Due 2/6):

- Programming assignment 1 (Due 2/6):

- Programming assignment 1 (Due 2/6):

- Week 2 (1/30): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators.Lecture note on Fourier Transform ans linear filtering

- Week 2 (1/30): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators.Lecture note on Fourier Transform and linear filtering

- Week 2 (1/30): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators.

- Week 2 (1/30): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators.Lecture note on Fourier Transform ans linear filtering

Sample Images

Sample Images

Project Guideline

Project Guideline

Project List

**Project Topics: **

Link

Link

Link

**Project Topic: **

**Project Topics: **

**Project Topic: **

Link

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation] , [[Attach:/home/visionpage/uploads/ContrastEnhancement.pdf| Lecture note on contrast enhancement

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation] , [[Attach:/home/visionpage/uploads/ContrastEnhancement.pdf| Lecture note on contrast enhancement

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization): Lecture note on image formation] , [[Attach:/home/visionpage/uploads/ContrastEnhancement.pdf| Lecture note on contrast enhancement

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization)ImageFormationContrastEnhancement

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization)

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization)ImageFormationContrastEnhancement

- Codeacdemy : python
- Anaconda

- Programming assignment 2 (Due 2/27): Wavelet based image denoising

- Programming assignment 2 (Due 2/27):
**Wavelet based image denoising**

- Programming assignment 3 (Due 3/6): Implement a specified segmentation method yourself. Compare different methods using available implementations.

- Programming assignment 3 (Due 3/6):

- Programming assignment 4 (Due 3/27): Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).

- Programming assignment 4 (Due 3/27):
**Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).**

- Programming assignment 5 (Due 4/10): Moving object detection

- Programming assignment 5 (Due 4/10):
**Moving object detection**

- Programming assignment 6 (Due 5/8): Video coding.

- Programming assignment 6 (Due 5/8):
**Video coding.**

- Programming assignment 1 (Due 2/6): Linear filtering for noise removal and image sharpening and illustrating effect in both spatial and frequency domain.

- Programming assignment 1 (Due 2/6):

**Links to resources (lecture notes and sample exams) in previous offerings: **

**Links to Resources (lecture notes and sample exams) in Previous Offerings: **

**Sample exams: **

**Sample Exams: **

Yilin Song (MTC2 Room 9.123, ys1297 at nyu dot edu) and Anti-Chiang (MTC2 Room 9.123, dawnandyknight at nyu dot edu)

Yilin Song (MTC2 Room 9.123, ys1297 at nyu dot edu) and Anti-Chiang (MTC2 Room 9.123, atc327 at nyu dot edu)

Professor Yao Wang, MTC2 Room 9.122, (646)-997-3469, Email: yaowang at nyu dot edu.Homepage

Professor Yao Wang, MTC2 Room 9.122, (646)-997-3469, Email: yaowang at nyu dot edu.Homepage

This course introduces fundamentals of image and video processing, including color image capture and representation; contrast enhancement; spatial domain filtering; two-dimensional (2D) Fourier transform and frequency domain interpretation of linear convolution; image sampling and resizing; multi-resolution image representation using pyramid and wavelet transforms; feature point detection and global alignment between images based on feature correspondence; geometric transformation, image registration; video motion characterization and estimation; video stabilization and panoramic view generation; image and video segmentation; selected advanced image processing techniques; basic compression techniques and standards (JPEG image compression standard; wavelet transform and JPEG2000 standard; video compression using adaptive spatial and temporal prediction; video coding standards (MPEGx/H26x); Stereo and multi- view image and video processing (depth from disparity, disparity estimation, video synthesis, compression). Students will learn to implement selected algorithms in Python. A term project will be required.

Graduate status. EL-GY 6113 and EL-GY 6303 preferred but not required. Undergraduate students must have completed EE-UY 3054 Signals and systems and EE-UY 2233 Probability.

Graduate status. EL-GY 6113 and EL-GY 6303 preferred but not required. Undergraduate students must have completed EE-UY 3054 Signals and systems and EE-UY 2233 Probability.

Professor Yao Wang, MTC2 Room 9.122, (646)-997-3469, Email: yaowang at nyu dot edu.Homepage

Professor Yao Wang, MTC2 Room 9.122, (646)-997-3469, Email: yaowang at nyu dot edu. Homepage: http://eeweb.poly.edu/~yao

Yilin Song (MTC2 Room 9.123, ys1297 at nyu dot edu) and Anti-Chiang (MTC2 Room 9.123, dawnandyknight at nyu dot edu)

Yilin Song (MTC2 Room 9.123, ys1297 at nyu dot edu) and Anti-Chiang (MTC2 Room 9.123, dawnandyknight at nyu dot edu)

Monday 12:25 AM-2:55PM, Room RH215.

Monday 12:25 AM-2:55PM, Room RH215.

Wed. 4-6PM and Thur 4-6 PM or appointment by email.

Wed. 4-6PM and Thur 4-6 PM or appointment by email.

Exam: 40%, Final Project: 30%, Programming assignments: 30%, Written homework will be assigned but not graded. Solution to written HW will be provided.

Exam: 40%, Final Project: 30%, Programming assignments: 30%, Written homework will be assigned but not graded. Solution to written HW will be provided.

Late submissions of programming assignment will be accepted up to 7 days after the deadline, with penalty of 1 pt for each day (Each assignment has 10pt). Students can work in teams (at most 3 people) and each team should submit one completed assignment. You should include your code (with documentation) and results (plots etc.) and discussions. When working in teams, each person must understand what others did and what are included in the assignment!

Late submissions of programming assignment will be accepted up to 7 days after the deadline, with penalty of 1 pt for each day (Each assignment has 10pt). Students can work in teams (at most 3 people) and each team should submit one completed assignment. You should include your code (with documentation) and results (plots etc.) and discussions. When working in teams, each person must understand what others did and what are included in the assignment!

:This course introduces fundamentals of image and video processing, including color image capture and representation; contrast enhancement; spatial domain filtering; two-dimensional (2D) Fourier transform and frequency domain interpretation of linear convolution; image sampling and resizing; multi-resolution image representation using pyramid and wavelet transforms; feature point detection and global alignment between images based on feature correspondence; geometric transformation, image registration; video motion characterization and estimation; video stabilization and panoramic view generation; image and video segmentation; selected advanced image processing techniques; basic compression techniques and standards (JPEG image compression standard; wavelet transform and JPEG2000 standard; video compression using adaptive spatial and temporal prediction; video coding standards (MPEGx/H26x); Stereo and multi- view image and video processing (depth from disparity, disparity estimation, video synthesis, compression). Students will learn to implement selected algorithms in Python. A term project will be required.

- This course introduces fundamentals of image and video processing, including color image capture and representation; contrast enhancement; spatial domain filtering; two-dimensional (2D) Fourier transform and frequency domain interpretation of linear convolution; image sampling and resizing; multi-resolution image representation using pyramid and wavelet transforms; feature point detection and global alignment between images based on feature correspondence; geometric transformation, image registration; video motion characterization and estimation; video stabilization and panoramic view generation; image and video segmentation; selected advanced image processing techniques; basic compression techniques and standards (JPEG image compression standard; wavelet transform and JPEG2000 standard; video compression using adaptive spatial and temporal prediction; video coding standards (MPEGx/H26x); Stereo and multi- view image and video processing (depth from disparity, disparity estimation, video synthesis, compression). Students will learn to implement selected algorithms in Python. A term project will be required.

:This course introduces fundamentals of image and video processing, including color image capture and representation; contrast enhancement; spatial domain filtering; two-dimensional (2D) Fourier transform and frequency domain interpretation of linear convolution; image sampling and resizing; multi-resolution image representation using pyramid and wavelet transforms; feature point detection and global alignment between images based on feature correspondence; geometric transformation, image registration; video motion characterization and estimation; video stabilization and panoramic view generation; image and video segmentation; selected advanced image processing techniques; basic compression techniques and standards (JPEG image compression standard; wavelet transform and JPEG2000 standard; video compression using adaptive spatial and temporal prediction; video coding standards (MPEGx/H26x); Stereo and multi- view image and video processing (depth from disparity, disparity estimation, video synthesis, compression). Students will learn to implement selected algorithms in Python. A term project will be required.

- This course introduces fundamentals of image and video processing, including color image capture and representation; contrast enhancement; spatial domain filtering; two-dimensional (2D) Fourier transform and frequency domain interpretation of linear convolution; image sampling and resizing; multi-resolution image representation using pyramid and wavelet transforms; feature point detection and global alignment between images based on feature correspondence; geometric transformation, image registration; video motion characterization and estimation; video stabilization and panoramic view generation; image and video segmentation; selected advanced image processing techniques; basic compression techniques and standards (JPEG image compression standard; wavelet transform and JPEG2000 standard; video compression using adaptive spatial and temporal prediction; video coding standards (MPEGx/H26x); Stereo and multi- view image and video processing (depth from disparity, disparity estimation, video synthesis, compression). Students will learn to implement selected algorithms in Python. A term project will be required.

_>This course introduces fundamentals of image and video processing, including color image capture and representation; contrast enhancement; spatial domain filtering; two-dimensional (2D) Fourier transform and frequency domain interpretation of linear convolution; image sampling and resizing; multi-resolution image representation using pyramid and wavelet transforms; feature point detection and global alignment between images based on feature correspondence; geometric transformation, image registration; video motion characterization and estimation; video stabilization and panoramic view generation; image and video segmentation; selected advanced image processing techniques; basic compression techniques and standards (JPEG image compression standard; wavelet transform and JPEG2000 standard; video compression using adaptive spatial and temporal prediction; video coding standards (MPEGx/H26x); Stereo and multi- view image and video processing (depth from disparity, disparity estimation, video synthesis, compression). Students will learn to implement selected algorithms in Python. A term project will be required.

_>This course introduces fundamentals of image and video processing, including color image capture and representation; contrast enhancement; spatial domain filtering; two-dimensional (2D) Fourier transform and frequency domain interpretation of linear convolution; image sampling and resizing; multi-resolution image representation using pyramid and wavelet transforms; feature point detection and global alignment between images based on feature correspondence; geometric transformation, image registration; video motion characterization and estimation; video stabilization and panoramic view generation; image and video segmentation; selected advanced image processing techniques; basic compression techniques and standards (JPEG image compression standard; wavelet transform and JPEG2000 standard; video compression using adaptive spatial and temporal prediction; video coding standards (MPEGx/H26x); Stereo and multi- view image and video processing (depth from disparity, disparity estimation, video synthesis, compression). Students will learn to implement selected algorithms in Python. A term project will be required.

- S15 Final Exam solution
- S16_midterm solution
- S16 final exam solution

- S15 Final Exam solution
- S16_midterm solution
- S16 final exam solution

- EL 5123 Image Processing
- EL 6123 Video Processing
- EL 6123 Image and Video Processing (S16)
- The coursera image processing course by Prof. Katsaggelos:Link
- The image processing course at Stanford:Link
- The computer vision course at U. Washington:Link

- EL 5123 Image Processing
- EL 6123 Video Processing
- EL 6123 Image and Video Processing (S16)
- The coursera image processing course by Prof. Katsaggelos:Link
- The image processing course at Stanford:Link
- The computer vision course at U. Washington:Link

- OpenCV: an open source package including many computer vision algorithms
- Numpy
- Scipy
- Matrix Reference Manual

- OpenCV: an open source package including many computer vision algorithms
- Numpy
- Scipy
- Matrix Reference Manual

- S15_midterm_w_solution
- S15 Final Exam solution
- S16_midterm solution
- S16 final exam solution

- S15_midterm_w_solution
- S15 Final Exam solution
- S16_midterm solution
- S16 final exam solution

**Project Guideline**Sample Images

'''Middelbury Stereo Image Database

Project Guideline

Sample Images

Middelbury Stereo Image Database

- Week 12 (4/24): Image representation using unitary transforms (orthogonal bases). Transform coding. JPEG image compression standard. Lecture note (updated 4/10/2016Week 11 ( Image representation using wavelet transform; concept of layered coding. JPEG2000 image compression standard. Lecture note

(updated 4/25/2016)

- Week 13 (5/1): Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity. Lecture note

(updated 4/25/2016)

- Week 12 (4/24): Image representation using unitary transforms (orthogonal bases). Transform coding. JPEG image compression standard. Lecture note(updated 4/10/2016Week 11 ( Image representation using wavelet transform; concept of layered coding. JPEG2000 image compression standard. Lecture note(updated 4/25/2016)
- Week 13 (5/1): Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity.Lecture note(updated 4/25/2016)

- Week 14 (5/8): Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH) Lecture note

(updated 5/7/2016)

- Week 14 (5/8): Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH) Lecture note(updated 5/7/2016)

- Week 3 (2/6): Image sampling and resizing. Design of interpolation filters. Multiresolution representation: Pyramid and wavelets.
- Programming assignment 2 (Due 2/27): Wavelet based image denoising
- 2/20/17 Presidents’ Day. No class.
- Week 5 (2/27): Project Proposal Due
- Week 5 (2/27): Image segmentation: region growing, split and merge, Otsu’s method, K-means, mean-shift, normalized cut, graph cut (optional).
- Programming assignment 3 (Due 3/6): Implement a specified segmentation method yourself. Compare different methods using available implementations.
- Week 6 (3/6): Feature detection (SIFT), feature descriptors and matching, and feature based global mapping estimation (Robust least squares and RANSAC)
- 3/13/17-3/19/17: Spring Recess
- Week 7 (3/20): Geometric transformation. Image warping. Image morphing. Panoramic view stitching. Video stabilization
- Programming assignment 4 (Due 3/27): Stitching a panoramic picture (Feature detection, finding global mapping, warping, combining).
- Week 8 (3/27): Midterm Project Feedback (Individual meeting)
- Week 8 (3/27): Dense motion/displacement estimation: optical flow equation, optical flow estimation; block matching, multi-resolution estimation. Deformable registration (medical applications)
- Week 9 (4/3): Moving object detection (background/foreground separation) (Gaussian mixture model, RPCA). Simultaneous estimation of camera motion and moving objects. Object tracking. Video shot segmentation.
- Programming assignment 5 (Due 4/10): Moving object detection
- Week 10 (4/10): Stereo and multiview video: depth from disparity, disparity estimation, view synthesis. Depth camera (Kinect). 360 video camera and view stitching. Light field imaging.
- Week 11 (4/17): Exam
- Week 12 (4/24): Image representation using unitary transforms (orthogonal bases). Transform coding. JPEG image compression standard. Lecture note (updated 4/10/2016Week 11 ( Image representation using wavelet transform; concept of layered coding. JPEG2000 image compression standard. Lecture note

(updated 4/25/2016)

- Week 13 (5/1): Video coding: block-based motion compensated prediction and interpolation, adaptive spatial prediction, block-based hybrid video coding, rate-distortion optimized mode selection, rate control, Group of pictures (GoP) structure, tradeoff between coding efficiency, delay, and complexity. Lecture note

(updated 4/25/2016)

- Programming assignment 6 (Due 5/8): Video coding.
- Week 14 (5/8): Overview of video coding standards (AVC/H.264, HEVC/H.265); Layered video coding: general concept and H.264/SVC. Multiview video compression. 360 Video compression. Error resilience issues. Adaptive Video Streaming (DASH) Lecture note

(updated 5/7/2016)

- Week 15 (5/15): Project Presentation.
- 5/20: Project Report and all other material must be uploaded.

@Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization) @Tutorial on python (Time /location to be arranged) @Week 2 (1/30): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators. @Programming assignment 1 (Due 2/6): Linear filtering for noise removal and image sharpening and illustrating effect in both spatial and frequency domain.

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization)
- Tutorial on python (Time /location to be arranged)
- Week 2 (1/30): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators.
- Programming assignment 1 (Due 2/6): Linear filtering for noise removal and image sharpening and illustrating effect in both spatial and frequency domain.

- Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization)
- Tutorial on python (Time /location to be arranged)
- Week 2 (1/30): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators.
- Programming assignment 1 (Due 2/6): Linear filtering for noise removal and image sharpening and illustrating effect in both spatial and frequency domain.

@Week 1 (1/23): Image Formation and Representation: 3D to 2D projection, photometric image formation, trichromatic color representation, video format (SD, HD, UHD, HDR). Contrast enhancement (concept of histogram, nonlinear mapping, histogram equalization) @Tutorial on python (Time /location to be arranged) @Week 2 (1/30): Review of 1D Fourier transform and convolution. Concept of spatial frequency. Continuous and Discrete Space 2D Fourier transform. 2D convolution and its interpretation in frequency domain. Implementation of 2D convolution. Separable filters. Frequency response. Linear filtering (2D convolution) for noise removal, image sharpening, and edge detection. Gaussian filters, DOG and LOG filters as image gradient operators. @Programming assignment 1 (Due 2/6): Linear filtering for noise removal and image sharpening and illustrating effect in both spatial and frequency domain.

- Tutorial on python (Time /location to be arranged)

- S16 final exam solution.

- S16 final exam solution

The School of Engineering encourages academic excellence in an environment that promotes honesty, integrity, and fairness. Please see the policy on academic dishonesty: Link

**Sample exams: **

**Sample exams: **

- S15_midterm_w_solution
- S15 Final Exam solution
- S16_midterm solution
- S16 final exam solution.

**Links to resources (lecture notes and sample exams) in previous offerings: **

**Links to resources (lecture notes and sample exams) in previous offerings: **

**Course Description::**

**Course Description:**

**Prerequisites:**

**Prerequisites:**

**Instructor:**

**Instructor:**

**Teaching Assistants:**

**Teaching Assistants:**

**Course Schedule:**

**Course Schedule:**

**Office Hour:**

**Office Hour:**

**Text Book/References:**

**Text Book/References:**

**Grading Policy: **

**Grading Policy: **

**Homework Policy: **

**Homework Policy: **

**Links to resources (lecture notes and sample exams) in previous offerings: **

**Links to resources (lecture notes and sample exams) in previous offerings: **

**Other Useful Links **

**Other Useful Links **

**Sample exams: **

**Sample exams: **

**Policy on Academic Dishonesty: **

**Policy on Academic Dishonesty: **

**Course Description::**

**Course Description::**

**Tentative Course Schedule **

**Tentative Course Schedule **

**Links to resources (lecture notes and sample exams) in previous offerings: **

**Links to resources (lecture notes and sample exams) in previous offerings: **

**Other Useful Links **

**Other Useful Links **

**Links to resources (lecture notes and sample exams) in previous offerings: **

**Links to resources (lecture notes and sample exams) in previous offerings: **

**Other Useful Links **

**Other Useful Links **

- Intro to Python: First three slides of the Python course at Columbia:"
**Link**" - "
**OpenCV**": an open source package including many computer vision algorithms - "
**Numpy**" - "
**Scipy**" - "
**Matrix Reference Manual**"

- Intro to Python: First three slides of the Python course at Columbia:Link
- OpenCV: an open source package including many computer vision algorithms
- Numpy
- Scipy
- Matrix Reference Manual

- [[http://eeweb.poly.edu/~yao/EL6123old/ | EL 6123 Video Processing]

- EL 6123 Video Processing

- "'EL 5123 Image Processing''
- "'[[http://eeweb.poly.edu/~yao/EL6123old/ | EL 6123 Video Processing]''
- "'EL 6123 Image and Video Processing (S16)''
- The coursera image processing course by Prof. Katsaggelos:
*Link* - The image processing course at Stanford:"
**Link**" - The computer vision course at U. Washington:"
**Link**"

- EL 5123 Image Processing
- [[http://eeweb.poly.edu/~yao/EL6123old/ | EL 6123 Video Processing]
- EL 6123 Image and Video Processing (S16)
- The coursera image processing course by Prof. Katsaggelos:Link
- The image processing course at Stanford:Link
- The computer vision course at U. Washington:Link

''Middelbury Stereo Image Database

'''Middelbury Stereo Image Database

'''Project Guideline'"

'''Project Guideline

'''Sample Images'"

'''Sample Images

''Middelbury Stereo Image Database'"

''Middelbury Stereo Image Database

"Project Guideline"

'''Project Guideline'"

"Sample Images"

'''Sample Images'"

"Middelbury Stereo Image Database"

''Middelbury Stereo Image Database'"

- "
**EL 5123 Image Processing**" - "
**[[http://eeweb.poly.edu/~yao/EL6123old/ | EL 6123 Video Processing]**" - "
**EL 6123 Image and Video Processing (S16)**" - The coursera image processing course by Prof. Katsaggelos:"
**Link**"

- "'EL 5123 Image Processing''
- "'[[http://eeweb.poly.edu/~yao/EL6123old/ | EL 6123 Video Processing]''
- "'EL 6123 Image and Video Processing (S16)''
- The coursera image processing course by Prof. Katsaggelos:
*Link*

"Project Guideline"

"Project Guideline"

"Sample Images"

"Sample Images"

"Project Guideline"

"Sample Images"

"Middelbury Stereo Image Database"Links to resources (lecture notes and sample exams) in previous offerings:

- "
**EL 5123 Image Processing**" - "
**[[http://eeweb.poly.edu/~yao/EL6123old/ | EL 6123 Video Processing]**" - "
**EL 6123 Image and Video Processing (S16)**" - The coursera image processing course by Prof. Katsaggelos:"
**Link**" - The image processing course at Stanford:"
**Link**" - The computer vision course at U. Washington:"
**Link**"

**Other Useful Links **

- Intro to Python: First three slides of the Python course at Columbia:"
**Link**" - "
**OpenCV**": an open source package including many computer vision algorithms - "
**Numpy**" - "
**Scipy**" - "
**Matrix Reference Manual**"

**Tentative Course Schedule ** **Sample exams: ** **Policy on Academic Dishonesty: **

//

//

//

//

//

//

//

//

//

//

//

//

//

//

//

//

//

//

//