DESCRIPTION       (Syllabus)

The course’s primary focus will be smooth optimization techniques with machine learning and artificial intelligence applications. The course will introduce the basics of algorithms on continuous optimization, starting from the classical gradient descent algorithm in convex optimization towards more sophisticated approaches in non-convex scenarios. The course will explore the fundamental theory, algorithms, complexity, and approximations in nonlinear optimization.

PREREQUISITES

The prerequisites are linear algebra, multivariate calculus, probability, and statistics. AI/Machine learning courses are optional but highly recommended.

COURSE OUTCOMES

After successful attendance, students are expected to:

• Have a good understanding of the theory involved in optimization via machine learning/AI applications
• Understand the differences and the reasoning/logic behind optimization algorithms, such as SGD, adaptive methods (Adam, RMSProp, Adagrad, etc), and second-order methods.
• Have a good understanding of standard convex optimization techniques, both in theory and practice.
• Have a good understanding of the differences/difficulties of convex and non-convex optimization.
• Understand how optimization plays a crucial role in different ML/AI/SP areas.
• Be able to read and review advanced papers on similar subjects.

COURSE POLICIES

• During Class The electronic recording of notes will be necessary for class, so that computers will be allowed in class. Please refrain from using computers for anything but activities related to the class. Drinking (coffee, tea, water) is allowed in class. Avoid eating your lunch in class, as the classes are typically active.

• Academic Integrity and Honesty Students are required to comply with the university policy on academic integrity found in the Honor System Handbook (http://honor.rice.edu/honor-system-handbook/).

• Accommodations for Disabilities If you have a documented disability that may affect academic performance, you should: 1) make sure this documentation is on file with Disability Resource Center (Allen Center, Room 111 / adarice@rice.edu / x5841) to determine the accommodations you need; and 2) meet with me to discuss your accommodation needs.