Advanced topics in optimization: From simple to complex ML systems
| Email (instructor): anastasios@rice.edu | Web: https://akyrillidis.github.io/comp545/ |
| Email (course): RiceCOMP545@gmail.com | |
| Office hours: By appointment | Class hours: T\TH 15:10 - 16:30 |
| Office: DH 3119 | Classroom: Online |
| Course Syllabus | LaTEX template for scribing |
| Course description | Schedule | Grading policy | Literature |
This course is a continuation of COMP 414/514: Optimization - Algorithms, Complexity, and Approxima- tions. The course includes a list of more advanced topics in optimization, including variants of gradient descent, such as AdaGrad, Adam and RMSProp; duality theory, and Lagrange multiplier methods and variants; Interior point methods; Mirror descent and the Multiplicative Weight Updates algorithm; Adver- sarial robustness; Generative Adversarial Networks, min-max optimization; Discrete optimization, convex relaxations and Introduction to Quantum Algorithms; Gaussian Processes for inference. The main objective of the course is to highlight optimization as a vital part of contemporary research in ML/AI/SP, and draw the attention of students to open-questions in related advanced topics. In particular, the aim for students is to (i) learn how to distinguish differences in research papers of related fields, (ii) understand the connection between them and how researchers advance each area, and (iii) be able to consider possible extensions of these works, as part of the final (open-ended) project of the course.
Textbook
There is no textbook for the class. The class will be a collection of lectures, prepared by the instructor, as well as presentations of research papers. Links to resources will be provided during the course.
Prerequisites
Basics of calculus, linear algebra and basic knowledge of machine learning topics or COMP 414/514.
Course outcomes
The main objective of the course is to highlight optimization as a vital part of contemporary research in ML/AI/SP, and draw the attention of students to open-questions in related topics. In particular, the aim for students is to (i) learn how to distinguish differences in research papers of related fields, (ii) understand the connection between them and how researchers advance each area, and (iii) be able to consider possible extensions of these works, as part of the final (open-ended) project of the course.
After successful attendance, students are expected to:
- have a good understanding of more involved problem cases where optimization is used.
- have a good comprehension how optimization plays a key role in different areas of research.
- be able to read and review advanced papers on similar subjects, as well as present the papers in front of an audience.
Course Policies
- During Class
The electronic recording of notes will be important for class and so computers will be allowed in
class. Please refrain from using computers for anything but activities related to the class.
Drinking (coffee, tea, water) is allowed in class. Try not to eat your lunch in class as the
classes are typically active.
- Policies on Late Assignments
Assignments (scribing, reviews, project) should be turned on time. You receive a 10% penalty for
each day of delay, up to 2 days. No submissions after the 2 day grace period. Exceptions will be
given to very extreme circumstances, with proper documentations.
- Academic Integrity and Honesty
Students are required to comply with the university policy on academic integrity found in the
Honor System Handbook (http://honor.rice.edu/honor-system-handbook/).
- Accommodations for Disabilities
If you have a documented disability that may affect academic performance, you should: 1) make sure
this documentation is on file with Disability Resource Center (Allen Center, Room 111 /
adarice@rice.edu / x5841) to determine the accommodations you need; and 2) meet with me to discuss
your accommodation needs.