What to feed to the input of the neural network? Your ideas... - page 85

 
Сергей Криушин #:

Hi, smoker alive, yes, this is our, pension they there twist as they want, they do not accept any confirmation even with stamps, accidentally witnessed their conversations ... consider in their favour every nick or incorrect writing in the employment book or in the payroll, there is such a deaf bureaucracy that it is better to go straight to court, and if there are no deductions to the pension, they do not talk at all.

Hi, more alive than all the living )) Just a little labour experience, all the 90's were their own businesses, did not think about old age.
 
Idea: to input the results of trades without market context

Problem statement:
There is a basic trading strategy (TS). It gives signals, we trade, we get a yield curve.
.

For each signal we know:

  • Result (% or points).
  • Duration of the trade (in bars or minutes).
  • Maximum drawdown within a trade.
  • Maximum profit within the deal.
  • Direction of the deal (Long/Short).
  • Market volatility at the moment of the deal.

The point: we feed all this information to the neural network. This will give it more resulting context. For example, it may notice: "A series of deals where we made a small profit, but the volatility was falling - it means that the market is up, it is better not to trade".
No market data (prices, indicators, volumes) are fed to the neural network input. Can a neural network trained solely on the sequence of these results learn to filter signals (skip only those that are more likely to be profitable) or even predict the direction of the next trade? This is an open question.

What can be hypothetically extracted from a sequence of trades?
  1. Series Effect: Many strategies tend to "overheat" or "cool down" after a series of successes/failures. For example, after three losing trades in a row, the probability of a profitable one may increase (due to a return to the average) or, on the contrary, fall (if the strategy broke down in the current market).

  2. Time dependence: If trades are time-based, you can detect intraday or intra-week patterns: for example, a strategy often makes a loss on Mondays and a profit on Wednesdays. The network will be able to learn this.

  3. Volatility of results: The size of gains and losses can also be clustered: periods of high volatility (large movements) are followed by periods of calm. The network can learn to predict not only the sign but also the approximate magnitude of the outcome.

  4. Behavioural patterns: If a strategy contains elements that are sensitive to the state of the account (e.g. martingale), the sequence of outcomes will contain information about that state.



Data distribution trap:

We need to train the model on the unfiltered transaction history of the underlying strategy. If we train the network on data that has already been filtered once, the problem of distribution bias after filtering will appear. Let's assume that we have trained the neural network on all signals of the strategy (including loss signals). After training, we start using it in real trading: now some signals are rejected. Only those trades that the filter missed are included in the portfolio. Accordingly, the history of real results, which we see at the filter input at the next moment, does not consist of all signals, but only of the filtered ones. The distribution of this "live" history may differ from the training sample. For example, if the filter is good at filtering out losses, then there will be few losing trades in the real history, and the input window for the forecast will consist mostly of profits. A model trained on a mixture of profits and losses may misinterpret such a "skewed" window.

Therefore, for the scheme to be correct, we need to distinguish two concepts: "Real Account" and "Virtual Monitoring".

Brief scheme:

  1. Background Mode (Shadow Mode): even if the neural network is already filtering trades, a "shadow" copy of the basic strategy should always be running.

    • The basic strategy generates a signal.
    • The result of this signal is recorded in the "Virtual Log".
    • Even if the neural network says "Do not open a trade", it is still opened virtually, and it is logged until the end.

  2. Neural network input: The results from the Virtual Journal (raw data of the underlying TS), not from the real account, are fed to the neural network input.

  3. Neural network output: Looking at the state of the raw yield curve, the neural network decides: "Now the basic strategy has entered the band of failures (or successes), so we skip (or flip) the trade to the real account".


 
Ivan Butko # :

For each signal we know:

  • Result (% or points).
  • Duration of the trade (in bars or minutes).
  • Maximum drawdown within a transaction.
  • Maximum profit within a transaction.
  • Direction of the transaction (Long/Short).
  • Market volatility at the time of the transaction.

The gist: we feed all this information to the neural network. This will give it more context for the resulting trade. For example, it might notice, "A series of trades where we made a small profit, but volatility fell—means the market has stalled, better not to trade."
No market data (prices, indicators, volumes) is fed to the neural network. Can a neural network trained solely on these results learn to filter signals (allowing only those most likely to be profitable) or even predict the direction of the next trade?

Sure - but this is all from the future. It will be pure peeking. It is necessary to learn only on what is known before the moment of opening a deal. In real trading - where to get these parameters from?

I agree with everything written below.

 
Aleksei Kuznetsov #:

Sure - but it's all from the future. It's pure peeking. It is necessary to learn only on what is known before the moment of opening a trade. In real trading - where do these parameters come from?

I agree with everything written below.

I meant the last known (last N closed trades on the basic strategy).

Let's say we trade online with a trained model. The basic TS gives a signal to enter - we collect the last statistics on this basic TS (the last trade or several trades - this will be the information for the neural network).

Further - our neural network, let's say, refuses to open a trade - we skip it. But for the basic TS we save the result of the current trade (when it closes). And in the next signal we use this result as the only or composite (from several deals) inputs for our neural network.
 
Ivan Butko #:
It meant the last known (last N closed trades on the basic strategy).


There can be few of them. For example, one yesterday, one a week ago, one two weeks ago. And these parameters indirectly describe the price chart. But long past.
I am afraid that without fresh data from the chart - it will be more difficult to predict something.

I have been working with the basic strategy and its filtering as you described for a long time. The first one was like this https://www.mql5.com/en/code/903 - a signal on each bar, which we filter.
Then I added something of my own.

Sampler
Sampler
  • 2012.06.01
  • www.mql5.com
Индикатор i_Sampler рассчитывает идеальные входы, предназначен для обучения нейросети.
 
Aleksei Kuznetsov #:

There may not be many of them. For example, one yesterday, one a week ago, one two weeks ago. And these parameters indirectly describe the price chart. But long past.
I'm afraid that without fresh data from the chart - it will be more difficult to predict something.

Partly yes, given the technical regularity of dependence of the next price value on the previous one.

But, based on the practice of manual traders who look far back in history, there are working areas in it, which can either somehow influence the "next price value" or give some additional information about the future movement.

In such a case (paradigm, rather) any area of the chart has, if not direct predictive power, then at least a composite one, being a part of some general pattern, stretched in time, li
 
I like the idea of inputting different characteristics of a strategy (expectation matrix, Sharpe, drawdown, etc.). The idea is not mine, it is from the smartlab user bascomo (I think he used to live here too, but I don't know his nickname here). As soon as I get other details sorted out, I will try this approach.
 
Alexander Generalov #:
I like the idea of inputting different characteristics of a strategy (expectation matrix, Sharpe, drawdown, etc.). The idea is not mine, it is from the smartlab user bascomo (I think he used to live here too, but I don't know his nickname here). As soon as with other details will be sorted out, I will try this approach.
This idea is in the air. We have already tried everything else (price chart and what can be made of it).
 
Alexander Generalov #:
I like the idea of inputting different characteristics of a strategy (expectation matrix, Sharpe, drawdown, etc.). The idea is not mine, it is from the smartlab user bascomo (I think he used to live here too, but I don't know his nickname here). As soon as I get the other parts sorted out, I will try this approach.
Do I understand correctly that you are going to set strategy characteristics as a goal for the neural network?
 
Aleksei Kuznetsov #:
This idea is in the air. Everything else (price chart and what can be invented from it) has already been tried.

"AI, write me a profitable EA..."

You have not tried everything, not everything! Ask about scalping on the glass, if you want clarity on what goes where and how in pure form.