Machine learning in trading: theory, models, practice and algo-trading - page 1557

[Eugeni Neumoin]

Mihail Marchukajtes: This is the reason why it is better not to use ZZ as a target

I've been using ZZ for about 15 years now. I know all the disadvantages of this tool. And I know many other things, that's why I told above about my idea.

---

The system should find buy/sell points on its own. It competes not with the players on the market, but with the market itself (sorry for some tautology).

[Mihail Marchukajtes]

Eugeni Neumoin:

I've been working with ZZ for about 15 years now. I know all the disadvantages of this tool. And I know a lot of other things, that's why I told above about my idea.

---

The system must itself detect points for buying/selling. It competes not with the players on the market, but with the market itself (sorry for some tautology).

And here's where I want to be more specific. How did you implement it? When there is a struggle between buyers and sellers and the result is a quote I understand, but if the opponent in the game is the market itself I do not understand. If we're talking about Alpha, it has learned to play by itself according to certain rules. You cannot dictate the rules to the market, so it is not competing with the market, but trying to be like it. It is learning the market, but it is not competing with it. If you imagine that the market is a perfect player and we are trying to become better than it. Well, yes. I agree, but then the example of alpha is irrelevant. Because alpha was self-taught. She was self-training with herself. And yet describe in a nutshell how you made it exactly a competitive moment between the market and ns in which the market is the a priori winner. ZZ doesn't have finite values a priori, so it's useless

[Igor Makanu]

An article on Habra with the high-profile title "AI for people: simple words about technology"https://habr.com/ru/company/jetinfosystems/blog/471626/

I flipped through the article - it all boiled down to Machine Learning, ..... It reminded me of an old anecdote:

A student is taking a zoology exam. He only knows about fleas. On the exam he gets a question about dogs.

The student begins:

- Dogs are mammals, covered with fur. There are fleas in their fur...then it's all about fleas....

Preacher:

- Okay, young man, tell me about cats.

Student:

- Cats are mammals, covered with hair. There are fleas in their fur....then all about fleas....

Presenter:

- Let's talk about fish.

Student:

- Fish are not mammals. They don't have fur. They are covered with scales, but if they were covered with fur there would be fleas in them....

[Maxim Dmitrievsky]

[Maxim Dmitrievsky]

The finale of returnee training

[Vladimir Perervenko]

Eugeni Neumoin:

Do not use ZZ or any additional indicators. Only OHLC from several tf (tf should differ 4-6 times. For example, 1-5-30-H3... up to a monthly timeframe. Select it independently) and, perhaps, more ticks for early warning.

By the prices of maximums and minimums separately convolutional structures. By OHLC - the recurrence structure. And so on for all the used prices. All signals of that are directed, for example, to the full mesh network.

Also, the ticks passed through the recurrence network are fed to one of the inputs of the full mesh network.

Optimize by speed of deposit increase. As a result, the cumulative network should decide by itself, what is the lot volume and select the points of opening and closing. Approximately like this.

Why are you using words? Just draw the structure of your proposed network. And you, too, will have a lot of questions.

And in the meantime, it is just a flutter. The idea is right, but the direction is wrong.

[Vladimir Perervenko]

Maxim Dmitrievsky:

The finale of returnee training

Can you attach a homemade tester, I wonder if you avoided one very common mistake? Or in person.

Good luck

[Maxim Dmitrievsky]

Vladimir Perervenko:

And can you attach a self-made tester, I wonder if you avoided one very common mistake? Or in person.

Good luck

zhupiter notebook will do? skipped the whole, no secret, because the whole code is in the video

[Vladimir Perervenko]

Maxim Dmitrievsky:

jupiter notebook will do? discounted the whole, not secret, because the whole code is in the video

Sure. Thank you. I'll look at it and report back

[Vladimir Perervenko]

Maxim Dmitrievsky:

Jupiter notebook will do? I have downloaded the whole code, it is not secret, because the whole code is in the video

It's a lot of work to take apart someone else's code. Let's look at only the custom_tester() function and only the highlighted part.

def custom_tester(symbol, timeframe, frompos, pricesnumber, lag, num, markup = 0.0):   
    symbols_list = [symbol]
    pr_test = pd.DataFrame()
    for i in symbols_list:
        rates = MT5CopyRatesFromPos(i, timeframe, frompos, pricesnumber)
        pr_test[i] = [y.close for y in rates]

    #creating price returns with lag
    returns = np.array(np.log(1 + pr_test.pct_change(lag)))
    returns = returns[lag:]
    comp_returns = np.array(returns)
    #                   
    for i in range(num-1):
        comp_returns = np.c_[comp_returns, returns]
    #putting returns into dataframe    
    comp_returns = pd.DataFrame(comp_returns)
    #shifting columns to use it as delayed features
    for i in range(num):
        comp_returns[i] = comp_returns.shift(i)
    #deleting first 'num' rows with NaN's
    comp_returns_shifted = pd.DataFrame(comp_returns[num:]).reset_index(drop=True)
    
    testpr = np.array(pr_test[lag+num:])
    X = pd.DataFrame(comp_returns_shifted)
    p = model.predict_proba(X) # непрерывное предсказание классификатора (типа вероятность)
    
    result = np.array([0])
    
    lastdeal = int(2)
    lastpr = float(0.0)

    for i in range (X[0].size):
# При первом входе
        if lastdeal == 2:
# переводим непрерывное предсказание в номинальные значенияю Порог 0.5
            if p[i][0] > 0.5:
                lastdeal = 1
            else:
                lastdeal = 0
            lastpr = testpr[i] # сохраняем значение Close 
            continue
# при следующей итерации
        if lastdeal == 1 and p[i][0] > 0.5:
            lastdeal = 0 # это я не понял зачем
# При вычислении результата ошибка
            result = np.append(result, result[-1] - markup + lastpr - testpr[i])
            lastpr = testpr[i]
            continue
# при последующей итерации
        if lastdeal == 0 and p[i][0] < 0.5:
            lastdeal = 1 # здесь не должно быть -1?
# и здесь ошибка
            result = np.append(result, result[-1] - markup + testpr[i] - lastpr)
            lastpr = testpr[i]
            continue
            
    plt.figure(figsize=(20,5))
    plt.plot(result)
    
    from sklearn.linear_model import LinearRegression
    y = result.reshape(-1,1)
    X = np.arange(result.size).reshape(-1,1)
    lr = LinearRegression()
    lr.fit(X,y)
#     print(lr.score(X,y))
    return lr.score(X,y)

What is the error in calculating the result? You calculate the result for each iteration by adding result +=testpr[i] - lastpr to the previous value. This is the difference between the Close of the current bar and the previous one. Ideally, it would be better to use Close - Open, but it does not matter. The important thing is that having received a signal at the close of the current bar, you calculate the signal premium as diff(Close) of the same bar. This is incorrect. The signal premium of the current bar is diff(Close) of the next bar. To correctly calculate the result, you must shift p = model.predict_proba(X) to the right by one bar. I will give further calculations on R, it is easier for me.

ifelse(p >= 0.5, 1, -1)%>% lag()%>% na.omit() -> sig
cumsum(sig * {diff(testpr) %>% tail(lenght(sig))})-> result

In the first line, convert the predictions to nominal (1,-1), shift to the right by one bar, remove NA and get a vector of signals. In the second line cumulatively summarize the product of the signals vector and diff(Close) vector, having previously aligned it with the signals vector in length. Then we get the correct result.

Good luck