Into the Unknown
Practical Reinforcement LearningHigher School of Economics I am very proud that I survived and completed this thorny but exciting journey with Honors. I spent nearly 1 month on this very challenging course, which covers enormous amount of knowledge. The lecture videos are all well-prepared, they not only help me consolidate my original understanding, but also…
Exploration is needed to find unknown actions which lead to very large rewards. Most of the reinforcement learning algorithms share one problem: they learn by trying different actions and seeing which works better. We can use a few made-up heuristics (e.g. epsilon-greedy exploration) to mitigate the problem and speed up the learning process. Multi-armed bandits…
What is fashion Fashion can be many things: creativity, business, trend that people share, or a way to express personality. There is no consensus about what fashion is. Fashion is about impulse and desire. We live not according to reason, but according to fashion. Seneca, mid 1st century. Fashion is a balancing act between the…
Third party beneficiaries Third party beneficiaries are people who benefit under a contract, but are not part to the contract. What rights do third party beneficiaries have? The answer depends on what type of beneficiary each of these parties is: ONLY intended beneficiaries can enforce agreements as a third party, who must have been vested….
The problems of value-based methods The idea behind value-based reinforcement learning (say, Q-learning) is to find an optimal action, in a state, based on how much discounted reward you will get, by following a policy. The first problem here is value-based methods do not explicit learn “what to do”, instead it learns “what kind of…
Introduction to TensorFlowGoogle Quite a lot stuff of TensorFlow was distilled into this course, from the basics to advanced techniques. I guess some topics probably will make beginners confused, but will strike a chord with experienced developers. The course first introduced the layers of API, the very basic tensor and Variable. Then put much effort…
TensorFlow is an open-source, high-performance library for any numerical computation (not just for machine learning). For example, you could use TensorFlow to solve partial differential equations. In order to make TensorFlow work, you need to create directed graph, which represent the computation you want to do. In a graph, nodes represent mathematical operations, edges represent…
Introduction to Google SEOUniversity of California Davis This is a wonderful course about marketing by means of search engine optimization. Over the years, SEO is no longer as simple as following a checklist, it needs a holistic approach since there is a higher barrier of entry. The course covers a lot of material: what SEO…
What is SEO? SEO stands for Search Engine Optimization, which is the practice of improving visibility of a website in search. SEO fits into a larger digital marketing strategy, it is more focused on free organic traffic. Other roles include: SEOs want their page to show on the first page of search result. The goal…
UCC stands for Uniform Commercial Code, UCC Article 2 pertains sales of goods, which provides additional rules when it comes to contracts for the sales of tangible, movable goods. Sometimes a contract calls for both goods and services, in which case use “Predominant factor test’ to distinguish between them. The question is “What is the…
Exploratory Data Analysis with MATLABMathWorks Even though Python, R and others are getting more and more popular, MATLAB is still irreplaceable and indispensable in some fields. If you are willing to refresh your MATLAB skill set, don’t miss this official course from MathWorks. It is quite easy to sign up an MathWorks account, get a…
How Google does Machine LearningGoogle Cloud There is no math, no Tensorflow, no algorithms in this short course – don’t be upset. It shared a lot valuable stuff : the enterprise know-how that Google have obtained over the years. I think In week 2 the lecture is rather wonderful and inspiring! The topics include the…
AI First Think about the complex pipeline, to be successful at Machine Learning, besides creating models, you need to think about serving out machine learning predictions using notebooks, dashboards, applications, reports, etc. Operationalizing a machine model is hard. Batch data and stream data should be treated the same way. To be good at Machine Learning,…
Multi-period binomial model Multi-period binomial model is really just a series of one-period model spliced together. When pricing an European option, you can calculate it backward 1 period at a time. But you may also do the same thing just as one calculation. It appears the true probabilities p (price going up) and 1-p (price…
Deep Q-Network (DQN) is the first successful application of learning, both directly from raw visual inputs as humans do and in a wide variety of environments. It contains deep convolutional network without hand-designed features. DQN is actually no more than standard Q-learning bundled with stability and epsilon-greedy exploration. In any application, the first thing you…