Machine learning in trading: theory, models, practice and algo-trading - page 545

 
Maxim Dmitrievsky:

especially if we use multidimensional clustering, we can try to feed vectors with attributes and vectors with lag, say, increments... so that we could divide them into groups - which attribute properties would correspond to which increments in the future

and then use this set for training of NS, for example... i.e. like datamining

yes, exactly before training... or as a separate thing for TS

Yes.

By the way, in many python packages (there should also be some in R) there is such a thing besides batch_size - as time_steps, it is just used in time series analysis. That is, a matrix of time_steps length is fed to the input at once, which is taken into account inside the tensors inside the model.


If you do not know how to use data mining, then you may use it as an example and not as a marketing tool:
Markets are interconnected, so capital movements occur within a single whole system. So with a good deep model and computing resources of course, these movements of money masses can be caught.

 
Maxim Dmitrievsky:

especially if we use multidimensional clustering, we can try to feed vectors with attributes and vectors with lag, say, increments... so that we could divide them into groups - which attribute properties would correspond to which increments in the future

and then use this set for training of NS, for example... i.e. like datamining

yes, exactly before training... or as a separate thing for TS

I like @Mihail Marchukajtes idea about options and volatility smile. Only there is a hitch with parser.
 
Aleksey Terentev:
And for data mining I like the idea of @Mihail Marchukajtes about options and volatility smile. Only there was a hitch with the parser.

So far I have some unknown mixture of autoregressive model over-optimized + adaptive elements... unknown stuff, but cool, I shove everything I learn there :)

it kind of works but not always

If I run out of ideas I will try maybe an intermarket analysis or external information like options. I'll try to analyze something else, maybe from the outside, like options.

 

I found the problem, if you want to take the spread into account. And the spread is necessary if you do something pipsing.

So, when training, the data is unloaded from the history through CopySpread, and there is the minimum spread on the bar. And because the MOE carries out a bar analysis, then to properly account the spread, it is needed at the moment of calculation, that is the Open, Close, High or low price (depending on what you want to add this spread). For example, I work at Open prices. And of course the minimum spread in the bar is almost never equal to the spread at the moment of bar opening. As a result, the TP or SL may be 10-20 points different from the one that will be set in reality. I.e. the system profitability will be quite different from what the MO model assumes.
As a result, training with minimal spreads on a bar cannot be reproduced in reality.

I.e. we have a problem in 2 places:

1 - in the history, on which we teach (the spreads are different).

2 - at the moment of carrying out a deal (if the decision is made at the moment of bar opening). Different spreads will give different TP and SL.


I noticed this peculiarity when comparing the results of the trained model using opening prices and real ticks. The difference is very large. I.e. the difference will be very big in real trading.

It is either not to use spreads (thus rejecting the possibility of using scalping from the capabilities of a teachable system), or to collect matrices for training at real ticks and then test them at real ticks - it takes much longer than testing at opening prices. As an alternative, to speed up the process, you can previously create a custom symbol passing through real ticks and collecting the necessary spreads.

 
elibrarius:

I look at the MO models so far: the model gives the decision to enter or not, and the rest is a matter of money management and trading strategy.

Moreover, neural networks learn well from candlestick history.

If we transfer the selection of entry/exit points to the MO, or any other details, I would do it with the second model. Or complicate the model, in which case only deep learning with parallel tensors will work effectively.

In short: entry points, profit ratios, pips counting are not all for a simple model.
 
Aleksey Terentev:

I look at MO models so far: the model gives the decision to enter or not, and the rest is a matter of money management and trading strategy.

Moreover, neuronets learn well from candlestick history.

If we transfer the selection of entry/exit points to the MO, or any other details, I would do it with the second model. Or to complicate the model, in which case only deep learning with parallel tensors will be effective.

Briefly: entry points, profit ratios, pips counting - everything is not for a simple model.

have you tried prophet from facebook yet?

For some reason I have a stubbornly refusing to install, put anaconda, python... hangs at the installation stage and that's it. my friend has it working

well, it's just an interesting one

http://pythondata.com/stock-market-forecasting-with-prophet/

Stock market forecasting with prophet
Stock market forecasting with prophet
  • 2017.09.12
  • Eric Brown
  • pythondata.com
This article highlights using prophet for forecasting the markets. You can find a jupyter notebook with the full code used in this post here. For this article, we’ll be using S&P 500 data from FRED. You can download this data into CSV format yourself or just grab a copy from the my github ‘examples’ directory here. let’s load our data and plot...
 
Maxim Dmitrievsky:

have you tried prophet from facebook yet?

For some reason I have a stubbornly refusing to install, put anaconda, python... hangs at the installation stage and that's it. my friend has it working

well, it's just an interesting one

http://pythondata.com/stock-market-forecasting-with-prophet/

I have read these articles and, frankly, I'm not impressed by Dr. Brown's approach.

But the library is interesting. I'm sure it will be interesting to look into trend models.

I still stick with Keras. All the more so, training with reinforcement there is easily implemented together with Keras-RL.


Note about installation, when using Anaconda, packages should be installed through "conda" instead of "pip".

Keras Documentation
Keras Documentation
  • keras.io
Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research. Use Keras if you need a deep learning library that: Allows...
 
Maxim Dmitrievsky:

For some reason, my liba stubbornly refuses to install...

Don't cry)))) on the "advanced" game for vr.

https://cran.r-project.org/web/packages/PSF/vignettes/PSF_vignette.html

Introduction to Pattern Sequence based Forecasting (PSF) algorithm
  • Neeraj Bokde, Gualberto Asencio-Cortes and Francisco Martinez-Alvarez
  • cran.r-project.org
This section discusses about the examples to introduce the use of the PSF package and to compare it with auto.arima() and ets() functions, which are well accepted functions in the R community working over time series forecasting techniques. The data used in this example are ’nottem’ and ’sunspots’ which are the standard time series dataset...
 
Vizard_:
Don't cry)))) for an "advanced" game for vr.

https://cran.r-project.org/web/packages/PSF/vignettes/PSF_vignette.html


something interesting... for python I can't find

I have already accumulated a lot of stuff, it's time to build a new bot :)

 
Aleksey Terentev:

Note on the installation, when using Anaconda, packages should be installed through "conda" instead of "pip".


Yes, I did it via conda and I did it via pip in naked python. I have some bug in the system itself, it cannot build and compile what I downloaded... maybe I need to re-download it by hand

well that's nothing to worry about.

Reason: