BMI / CS 771 Learning Based Methods for Computer Vision
Instructor: Prof. Yin li
TA: Khoi Duc Nguyen
Term: Fall 2026
Location: 3345 Engineering Hall
Time: Mondays and Wednesdays, 2:30-3:45 PM
Course Overview
The course addresses the problems of representation and reasoning for large amounts of visual data, including images and videos, medical imaging data, and their associated tags or text descriptions. We will introduce deep learning in the context of computer vision and cover topics on visual recognition using deep models, such as image classification, object detection, human pose estimation, action recognition, 3D understanding, and medical image analysis. The course emphasizes the design of vision and learning algorithms and models, as well as their practical implementations.
Prerequisites
Students are strongly encouraged to have knowledge of computer vision (such as CS 566), or machine learning (such as CS 540), or medical image analysis (such as BMI/CS 567). In addition, the following skills are necessary for this course:
- Programming: Students should have basic proficiency in programming (Python). Projects are to be completed and graded in Python. Our teaching team will support questions about Python.
- Math fundamentals: Linear algebra, vector calculus, and probability theory.
Textbooks
- Optional: Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville (free electronic copy available on the website)
- Optional: Computer Vision: Algorithms and Applications by Rick Szeliski (free electronic copy available on the website)
Grading
- 48%: 4 homework assignments (mini-projects, mostly team-based) that involve programming
- 40%: 1 course project with several milestones
- 10%: Single-page course write-up
- 2%: Piazza participation
Late Submission Policy
We do not allow late homework assignments or projects. However, each student has three “late days” for the whole course. That is to say, the first 24 hours after the due date and time counts as 1 day, up to 48 hours is two and 72 for the third late day. These late days are intended to cover unexpected clustering of due dates, travel commitments, interviews, hackathons, etc. Don’t ask for extensions to due dates because we are already giving you a pool of late days to manage yourself.
Homework Assignments
The course will consist of 4 homework assignments. All assignments except the first one are team based. Teams of 2-3 students are preferred. Permission from the instructor is needed for a single-person team.
Please post all questions on Piazza so that others may learn from those questions as well. Do not email the teaching team directly with homework questions. All homeworks are to be submitted on Canvas by midnight on the due date. Late submissions should be emailed to the instructor.
Course Project
The final project is research-oriented. It can be a pure computer vision project or an application of existing vision methods in the student’s own research area. Students are expected to implement one (or more) related research papers, or think of some interesting new ideas and implement them using the techniques discussed in class. A team of 2-3 students are encouraged. Permission from the instructor is needed for a single-person team.
There will be four checkpoints for the final project: a project proposal, an intermediate milestone report, a final project report and a project presentation. The details are listed below.
- Project Proposal (5%): This will be a single-page document. You will explain what problem you are trying to solve, why you want to solve it, what are the possible steps to the solution, and how do you plan to evaluate your solution.
- Project Mid-Term Report (5%): This will be a single-page summary of current progress, including your current results, the difficulties that arose during the implementation, and how your proposal may have changed in light of current progress.
- Project Final Report (15%): The final report will be a four-page document. You will describe the motivation of the project, the previous literate, your method and the results. You can reuse the materials that are presented in your proposal / mid-term report. Please include your source code in the submission.
- Project Presentation (15%, in class): Each team will be allocated a 12-min slot in class. This slot includes a 10-min presentation and a 2-min QA session.
Course Write-up
The course write-up will be a document that captures your reflection on the course work, e.g., what you have learned, what are the most interesting findings in the course. The write-up must be completed individually.
Academic Integrity
This course follows the University of Wisconsin-Madison Code of Academic Integrity. Unless specifically authorized by the instructor, all coursework is to be done by the student working alone. Violations of the rules will not be tolerated.
Students are permitted and encouraged to discuss ideas with others. However, the core components of each assignment / project are expected to be implemented by individual student or team. Code, except for starter code / helper code that isn’t related to the core components, should not be posted publicly to Piazza.
Students are permitted and encouraged to use artificial intelligence (AI) tools and applications, including coding agent, as they support the learning objectives of this course. Please be aware students are responsible for the information submitted based on an AI query (i.e. ensure that the AI generated results do not contain misinformation or unethical content). Students must acknowledge the use of AI to conform to this course’s expectations.
Contact Info and Office Hours
If possible, please use Piazza to ask questions and seek clarifications before emailing the instructor.
Yin
- Email: yin(dot)li(at)wisc(dot)edu
- Office Hours: 11:00 am - 12:30 pm Monday and Tuesday, Morgridge Hall 6538 (walk-in) or via Zoom (by appointment only)
- Appointments can be also scheduled outside of normal office hours. Please send me an email if you plan to so.
Khoi
- Email: kdnguyen7(at)wisc(dot)edu
- Office Hours: TBD
Schedule
| Week | Date | Topic | Materials |
|---|---|---|---|
| 1 | Wed, Sep 2 | Course Introduction / Introduction to Visual Recognition |
|
| 2 | Mon, Sep 7 | No Class (Labor Day) | |
| 2 | Wed, Sep 9 | Data Driven Paradigm for Computer Vision | |
| 3 | Mon, Sep 14 | Introduction to Neural Networks (Part I) | |
| 3 | Wed, Sep 16 | Introduction to Neural Networks (Part II) | |
| 4 | Mon, Sep 21 | Convolutional Neural Networks (Part I) | |
| 4 | Wed, Sep 23 | Convolutional Neural Networks (Part II) | |
| 5 | Mon, Sep 28 | Tutorials on Deep Learning on the Cloud (Optional) |
|
| 5 | Wed, Sep 30 | Recurrent Neural Networks | |
| 6 | Mon, Oct 5 | Transformers (Part I) | |
| 6 | Wed, Oct 7 | Transformers (Part II) | |
| 7 | Mon, Oct 12 | Advanced Training (Part I) | |
| 7 | Wed, Oct 14 | Advanced Training (Part II) | |
| 8 | Mon, Oct 19 | Image Classification and Adversarial Samples | |
| 8 | Wed, Oct 21 | Object Detection and Instance Segmentation (Part I) |
|
| 9 | Mon, Oct 26 | Object Detection and Instance Segmentation (Part II) |
|
| 9 | Wed, Oct 28 | Semantic Segmentation and Dense Image Labeling |
|
| 10 | Mon, Nov 2 | Human Body Pose Estimation |
|
| 10 | Wed, Nov 4 | Action Recognition and Video Analysis |
|
| 11 | Mon, Nov 9 | 3D Deep Learning (Part I) |
|
| 11 | Wed, Nov 11 | 3D Deep Learning (Part II) |
|
| 12 | Mon, Nov 16 | Introduction to Medical Image Analysis |
|
| 12 | Wed, Nov 18 | Image and Text Generation (Part I) |
|
| 13 | Mon, Nov 23 | Image and Text Generation (Part II) |
|
| 13 | Wed, Nov 25 | Vision Language Models (Part I) |
|
| 14 | Mon, Nov 30 | Vision Language Models (Part II) |
|
| 14 | Wed, Dec 2 | Project Presentation | |
| 15 | Mon, Dec 7 | Project Presentation | |
| 15 | Mon, Dec 9 | Project Presentation |