Machine learning in trading: theory, models, practice and algo-trading - page 1284

[Forester]

Maxim Dmitrievsky:

In my 1st article "fast" saving and loading via binary. In the 2nd article slow loading via deserialization, the saving speed does not seem to suffer

Maybe the files are small there. But if it's 300 mb you were talking about, there will probably be a noticeable difference

[Maxim Dmitrievsky]

elibrarius:

Maybe there are small files. But if it's 300 mb which you were talking about, there will probably be a noticeable difference

If you save in ANSI it is twice less than in unicode. Well, there is a direct loading in the forest structure is, in my opinion, I do not know how else faster

at least now everything is fast, in short, as in the 1st article I'm satisfied now

int setRDF=FileOpen("RDFBufsize"+_Symbol+(string)_Period+".txt",FILE_READ|FILE_WRITE|FILE_CSV|FILE_ANSI|FILE_COMMON);
      RDF.m_bufsize=(int)FileReadNumber(setRDF);
      FileClose(setRDF);

      setRDF=FileOpen("RDFNclasses"+_Symbol+(string)_Period+".txt",FILE_READ|FILE_WRITE|FILE_CSV|FILE_ANSI|FILE_COMMON);
      RDF.m_nclasses=(int)FileReadNumber(setRDF);
      FileClose(setRDF);

      setRDF=FileOpen("RDFNtrees"+_Symbol+(string)_Period+".txt",FILE_READ|FILE_WRITE|FILE_CSV|FILE_ANSI|FILE_COMMON);
      RDF.m_ntrees=(int)FileReadNumber(setRDF);
      FileClose(setRDF);

      setRDF=FileOpen("RDFNvars"+_Symbol+(string)_Period+".txt",FILE_READ|FILE_WRITE|FILE_CSV|FILE_ANSI|FILE_COMMON);
      RDF.m_nvars=(int)FileReadNumber(setRDF);
      FileClose(setRDF);

      setRDF=FileOpen("RDFMtrees"+_Symbol+(string)_Period+".txt",FILE_READ|FILE_WRITE|FILE_BIN|FILE_ANSI|FILE_COMMON);
      FileReadArray(setRDF,RDF.m_trees);
      FileClose(setRDF);

The last - the heaviest, which is necessary to write/read the binary

[Forester]

Maxim Dmitrievsky:

If you save in ANSI it is twice less than in unicode. Well, there is a direct loading in the forest structure is, in my opinion, I do not know how else faster

at least now everything is fast, in short, as in the 1st article I'm satisfied now

The last one is the heaviest, and it is necessary to write/read it into a binary.

That's more or less how I imagined it)
But it should be even faster in binary and the file size is much smaller. And if you convert the data array to float before saving, the size will be reduced by another factor of 2.
But I don't understand why it works.

setRDF=FileOpen("RDFMtrees"+_Symbol+(string)_Period+".txt",FILE_READ|FILE_WRITE|FILE_BIN|FILE_ANSI|FILE_COMMON);

You read and write binary data into the text file. And before that in text form. MQL seems to be able to combine these functions in some way. It would be more logical to do it like this

FILE_READ|FILE_WRITE|FILE_BIN|FILE_COMMON

save and read.

[Maxim Dmitrievsky]

elibrarius:

That's more or less how I imagined it.)
But it should be even faster in binary and the file size is several times smaller. And if you convert the data array to float before saving, the size will become two times smaller.
But I don't understand why it works.

You read binary data into the text file. The data in text form before that. MQL seems to be able to combine these functions in some way. It would be more logical to do it like this

save and read.

Here the extension does not play any role, you can put any. The main thing is the|FILE_BIN flag

[Forester]

Maxim Dmitrievsky:

Here the extension does not play any role, you can put any. The main thing is the flag|FILE_BIN

I didn't notice that the file names are different. It is possible to save everything in one.

[Maxim Dmitrievsky]

elibrarius:
a - didn't notice that the file names are different. You can save everything in one.

can be in 1, the rest have almost no effect on performance, only the last with the structure of the forest, the heaviest. The rest are there by 1 term

[Forester]

Maxim Dmitrievsky:

can be in 1, the rest have almost no effect on performance, only the last with the structure of the forest, the heaviest. The rest are there by 1 term

then you can get confused in this heap of files yourself)
And 1 file for 1 model is more convenient

[Maxim Dmitrievsky]

elibrarius:
then you can get confused in this pile of files)
And 1 file for 1 model is more convenient

Yes, the other files are simple: number of trees, number of features, number of classes and number of samples. If they are known in advance, you can not even save. I just put what were on the default, and save only the structure with splits, well, more the number of samples varies.

[Aleksey Vyazmikin]

elibrarius:

No time - that's right. Even reading the forum distracts from more useful things. And you about the preparation of the release... It's a few hours to make it all correctly and provide and explain, and then ask for support)).

You are a programmer - as a programmer I am much more comfortable when it is clear what and why. Go into the code. You'll understand the tree build function in a couple of hours and plug in the limitation on the number of examples and whatever you want.

Gone to do the prefetching of predictors.

I'm not a programmer, so working with code requires a lot of effort, but in general, I understand the position.

[Forester]

Aleksey Vyazmikin:

I'm not a programmer, so working with code takes a lot of effort, but in general the position is understood.

Don't feel bad about it). This is a pretty good idea: you have 5 products (or maybe you didn't write them?) and a lot of signals (probably from your own Expert Advisors), which, judging by your own EAs, allow you to make a living out of the Forex market.

I'm still in search and live off of something else.