ECE 5973-961/983: Artificial Neural Networks and Applications

A bit about this course

Artificial neural networks was introduced in the 50’s of the last century. However, in the last decade, there has been strong resurgence of neural networks as processing techniques where they have been applied to many real-world problems. This leads to numerous breakthroughs on image, video, and natural language processing applications.

This course is aimed to be quite hands-on and should provide students with sufficient details for them to quickly apply to their own research. In particular, applications relating to computer vision and natural language processing will be discussed. There may be some math but we will not spend too much time going into proofs. Instead, we may try to go through (not exhaustively) some of the free libraries (mostly PyTorch). And you are definitely encouraged to explore and leverage them for your course project.

Textbook

• Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, MIT Press.

It is not required but is a very good reference.

Disscusion forum

Please sign up Discord through the link on Canvas. Please raise any questions, comments, or concerns there. You may contact me privately through Discord as well.

Reference

• Bengio's Textbook

• Lecun's NYU course

• Stanford CS231n

• Stanford CS224d

• Stanford CS20si

• Toronto CSC321

• Hinton's Coursera Course

• Fast AI course

• Oxford Course

• Berkeley CS 294

• MIT 6.S191

• MIT 6.S094

• Pythonprogramming.net Course

• Neural Network Course by Larochelle

• Princeton COS 598

• Video Recordings of the ICML’15 Deep Learning Workshop

• A good collection of papers

• Representation learning: A review and new perspectives

• Deep learning: a critical appraisal

• AI Playbook

• Machine learning for humans

• An AI reading list

Some Deep Learning Toolboxes and Libraries

• Tensorflow: From Google, probably most popular package. Not quite optimized for single PC

• JAX: Another experimental framework from Google

• Caffe2: From Facebook

• Caffe: From Berkeley

• Torch 7: From NYU, and used by Facebook/Twitter

• PyTorch: The Python version of Torch

• Theano: From Bengio's group in Montreal

• Keras: High-level layer on top of Theano/Tensorflow

• Lasagne: High-level layer on top of Theano

• matconvnet: From Oxford, kind of restricted

• mxnet: From Amazon

• Neon: From Intel

• Deeplearning4j

Office Hours

There are no “regular” office hours. And you are welcome to come catch me anytime or contact me through Discord. I am quite responsive and usually reply you within a day.

Course Syllabus (Tentative)

• Overview of machine learning

• History of artificial neural networks

• Perceptrons

• Backpropagation algorithms

• Regularization and dropout

• Weight initialization

• Optimization methods

• Convolutional neural networks (CNN)

• R-CNN, faster R-CNN

• Weight visualization, Deep visualization, t-SNE, deepdream

• Recurrent neural networks

• LSTM networks

• Restricted boltzmann machines

• Autoencoders

• Deep belief networks

Projects

Written report is due on May 11. Please read this for guideline. Video presnetation is worth a maximum 10% extra credit.

Grading

Homework: 20%. Written and programming assignments

Presentation: 20%. Including presentation abstract and peer reviews.

Literature survey: 20%.

Final Project: 40%. Including project proposal and progress report.

In-class participation and quizzes (extra credits): 0-10%.

Final grade:

• A: above 90%

• B: above 80% but not more than 90%

• C: above 70% but not more than 80%

• D: above 60% but not more than 70%

• F: not more than 60%

Prerequiste

Calculus (MATH 1914 or equivalent), linear algebra (MATH 3333 or equivalent), basic probability (MATH 4733 or equivalent), and intermediate programming skill (experience on Python/Numpy is preferred)

Note that the ability to program in Python is expected. Python is not difficult if you are familiar with any other high level general programming languages such as C/C++/C#/Java/Javascript/Perl/Matlab etc. If you don't know anything about Python, I would recommend you to try out this app.

Late Policy

• There will be 5% deduction per each late day for all submissions

• The deduction will be saturated after 10 days. So you will get half of your marks even if you are super late

Calendar

Academic Integrity

Academic honesty is incredibly important within this course. Cheating is strictly prohibited at the University of Oklahoma, because it devalues the degree you are working hard to get. As a member of the OU community, it is your responsibility to protect your educational investment by knowing and following the rules. For specific definitions on what constitutes cheating, review the Student’s Guide to Academic Integrity.

Religious Observance

It is the policy of the University to excuse the absences of students that result from religious observances and to reschedule examinations and additional required classwork that may fall on religious holidays, without penalty.

Reasonable Accommodation Policy

The Accessibility and Disability Resource Center is committed to supporting students with disabilities to ensure that they are able to enjoy equal access to all components of their education.  This includes your academics, housing, and community events.  If you are experiencing a disability, a mental/medical health condition that has a significant impact on one or more life functions, you can receive accommodations to provide equal access.  Possible disabilities include, but are not limited to, learning disabilities, AD(H)D, mental health, and chronic health.  Additionally, we support students with temporary medical conditions (broken wrist, shoulder surgery, etc.) and pregnancy.  To discuss potential accommodations, please contact the ADRC at 730 College Avenue, (ph.) 405.325.3852, or adrc@ou.edu. 

Title IX Resources and Reporting Requirement

Anyone who has been impacted by gender-based violence, including dating violence, domestic violence, stalking, harassment, and sexual assault, deserves access to resources so that they are supported personally and academically. The University of Oklahoma is committed to offering resources to those impacted, including: speaking with someone confidentially about your options, medical attention, counseling, reporting, academic support, and safety plans. If you would like to speak with someone confidentially, please contact OU Advocates (available 247 at 405-615-0013) or another confidential resource (see “Can I make an anonymous report?”). You may also choose to report gender-based violence and discrimination through other means, including by contacting the Institutional Equity Office (ieo@ou.edu, 405-325-3546) or police (911). Because the University of Oklahoma is committed to the safety of you and other students, I, as well as other faculty, Graduate Assistants, and Teaching Assistants, are mandatory reporters. This means that we are obligated to report gender-based violence that has been disclosed to us to the Institutional Equity Office. This includes disclosures that occur in: class discussion, writing assignments, discussion boards, emails and during StudentOffice Hours. For more information, please visit the Institutional Equity Office.

Adjustments for Pregnancy/Childbirth Related Issues

Should you need modifications or adjustments to your course requirements because of documented pregnancy-related or childbirth-related issues, please contact me as soon as possible to discuss. Generally, modifications will be made where medically necessary and similar in scope to accommodations based on temporary disability. Please see www.ou.edu/content/eoo/faqs/pregnancy-faqs.html for commonly asked questions.

Final Exam Preparation Period

Pre-finals week will be defined as the seven calendar days before the first day of finals. Faculty may cover new course material throughout this week. For specific provisions of the policy please refer to OU’s Final Exam Preparation Period policy.

Emergency Protocol

During an emergency, there are official university procedures that will maximize your safety. Severe Weather: If you receive an OU Alert to seek refuge or hear a tornado siren that signals severe weather 1. LOOK for severe weather refuge location maps located inside most OU buildings near the entrances 2. SEEK refuge inside a building. Do not leave one building to seek shelter in another building that you deem safer. If outside, get into the nearest building. 3. GO to the building’s severe weather refuge location. If you do not know where that is, go to the lowest level possible and seek refuge in an innermost room. Avoid outside doors and windows. 4. GET IN, GET DOWN, COVER UP. 5. WAIT for official notice to resume normal activities. Link to Severe Weather Refuge Areas, Severe Weather Preparedness - Video

Armed Subject/Campus Intruder

If you receive an OU Alert to shelter-in-place due to an active shooter or armed intruder situation or you hear what you perceive to be gunshots: 1. Avoid: If you believe you can get out of the area WITHOUT encountering the armed individual, move quickly towards the nearest building exit, move away from the building, and call 911. 2. Deny: If you cannot flee, move to an area that can be locked or barricaded, turn off lights, silence devices, spread out, and formulate a plan of attack if the shooter enters the room. 3. Defend: As a last resort fight to defend yourself. For more information, visit OU’s Emergency Preparedness site. Shots Fired on Campus Procedure – Video

Fire Alarm/General Emergency

If you receive an OU Alert that there is danger inside or near the building, or the fire alarm inside the building activates: 1. LEAVE the building. Do not use the elevators. 2. KNOW at least two building exits 3. ASSIST those that may need help 4. PROCEED to the emergency assembly area 5 ONCE safely outside, NOTIFY first responders of anyone that may still be inside building due to mobility issues. 6. WAIT for official notice before attempting to re-enter the building. OU Fire Safety on Campus

Mental Health Support Services

If you are experiencing any mental health issues that are impacting your academic performance, counseling is available at the University Counseling Center (UCC). The Center is located on the second floor of the Goddard Health Center, at 620 Elm Rm. 201, Norman, OK 73019. To schedule an appointment call (405) 325-2911. For more information, please visit University Counseling Center.