Into the Unknown
AI First Think about the complex pipeline, to be successful at Machine Learning, besides creating models, you need to think about serving out machine learning predictions using notebooks, dashboards, applications, reports, etc. Operationalizing a machine model is hard. Batch data and stream data should be treated the same way. To be good at Machine Learning,…
Deep Q-Network (DQN) is the first successful application of learning, both directly from raw visual inputs as humans do and in a wide variety of environments. It contains deep convolutional network without hand-designed features. DQN is actually no more than standard Q-learning bundled with stability and epsilon-greedy exploration. In any application, the first thing you…
Deduction to supervised learning problem In tabular method, each Q(s, a) could be seen as a parameter. There are more parameters than states, because there are as many parameters for each state as the possible number of actions for each state. There are also situations where states include continuous components. It means we need a…
Value Iteration in real world n real world, we don’t have the state transition probability distribution or the reward function. You may try sampling them, but you will never know the exact probabilities of them. As the result, you can not compute the expectation of the action values. We want a new algorithm that would…
Spinning all rotors of a quad-rotor in the same direction will cause the robot to constantly rotate. The thrust F and drag moment M are quadratic to angular velocity 蠅. You have to size the motor to product the torque to over come this drag moment. If resultant forces and moments are non-zero, you get…
Reward That all of what we mean by goals and purposes can be well thought of as maximization of the expected value of the cumulative sum of a received scalar signal (reward). R. Sutton This signal is ‘reward’, and sum of the signals is ‘return’. Each immediate reward depends on the agent action and environment…
Introduction to Applied Machine LearningAlberta Machine Intelligence Institute This course looks at machine learning in a different light: business. Fanciful ideas and hypes are still almost everywhere nowadays, but clear-headed people remembers the motto “no silver bullet”.This course introduced many fundamental knowledge without too many technical terminology. To me, the most impressive section is the Machine Learning Process Lifecycle.This…
Introduction to Self-Driving CarsUniversity of Toronto Self-driving cars! I am thrilled to pass this tough course. You have to implement a few kinematic and dynamic models, and the BIG FINAL challenge is to use CARLA simulator to drive a ‘real’ car! Very exciting!Familiarize yourself with physics, mechanics, classic control techniques, and optimization, otherwise you will have difficulties when understanding equations.I am proud to…
Reinforcement Learning SpecializationUniversity of Alberta Anyone who loves reinforcement learning should not miss this specialization. Thank you University of Alberta! My Specialization Certificate Courses I am Kesler Zhu, thank you for visiting. Checkout all of my course reviews at http://KZHU.ai
A Complete Reinforcement Learning System (Capstone)University of Alberta For moon landing, let’s build a complete Deep RL system. You will get hands dirty in an expected SARSA agent using neural networks as function approximation and Adam as optimizer, also don’t forget softmax and replay buffer, and later parameter study! What an achievement! But meanwhile – as Prof. Martha White and Prof. Adam White pointed out – this is only the “first step towards learning…
Prediction and Control with Function ApproximationUniversity of Alberta This course reaches the apex of the entire reinforcement learning specialization! Don’t even bother ? if you are not ready for supervised learning and neural networks. The last topics policy parameterization and Actor-Critic algorithm are really the shiny pearls of the crown. Love it. A round of applause to Prof. Martha White and Prof. Adam White. Thank you very…
Sample-based Learning MethodsUniversity of Alberta I had a terrific time! This course covers quite a few fundamental & crucial reinforcement learning algorithms: Temporal Difference, SARSA, Dyna-Q, their variants, and their relationship with Dynamic Programming, Monte Carlo covered in previous course. The well-prepared programming assignments are both challenging and enlightening – helping me comprehend those complicated algorithms. My…
Fundamentals of Reinforcement LearningUniversity of Alberta Many thanks to Prof. Martha White and Adam White for this great reinforcement learning course – I wish I could have done this when I was in university. The course teaches fundamental concepts like Bellman equations, dynamic programming , etc. All of them become crystal clear. Finally, 2020 is over, let’s keep learning in 2021. Happy…
How to Win a Data Science Competition (with Honors)Higher School of Economics This course rocks! ? I found many tips, tricks and techniques very useful, some of them are a bit challenging to me. exploratory data analysis (EDA) is critical; ensembles are powerful; various kinds of hyper-parameters and metrics optimizations are the devil hides in the detail. A successful machine learning model is more…
Introduction to Deep LearningHigher School of Economics Don’t be misled by the name, it ain’t no way a course for beginners. On the contrary, many *advanced* deep learning topics are included鈥攃onsolidate your knowledge and get ready for the whole specialization. Programming assignments for Generative Adversarial Networks (GAN) and Image Captioning are very impressive. Give it…