Machine learning in trading: theory, models, practice and algo-trading - page 1762

 
Aleksey Vyazmikin:

I now have 16 4x4 cells, there is a dynamic window, and I count how many bars closed in each of the cells - if primitive.

Well, yes, that's roughly how I understood it, it's +- like in "text mining" when counting frequencies of words or letters in the text

Aleksey Vyazmikin:

The target must be generated by the Expert Advisor, otherwise it's not clear how to get new predictors - based on what data.

We don't need any new data yet, let's take 60 000 observations divided by test and trace and go to battle.

Same with the target, create for example ZZ at once, binarize it into 1001100 and go ahead too.

Aleksey Vyazmikin:

Then it is possible to set a lot of targets, which will allow to reveal predictive power of predictors, because good predictor will be successful on different targets.

If we forecast the increase of one minute then it's one indication, if we forecast which level in weeks will be broken through and which not, then completely different indications

But I do support the collaboration in the targets of course! But it can be done in the form of a sign, for example a forecast by the target Y...n to use as a sign for the main target

Aleksey Vyazmikin:

The instrument should be the same for all - Moex futures or glue is better. I like Si.

I agree, that's why it is better to take something universal from forex without gluing...

I don't think that one futures has enough data, while glues are distortions in prices + a lack of liquidity at the beginning and at the end of glues.

 
mytarmailS:

Why? We don't need any new data yet, take at least 60k observations, divide them into test and track and go to battle.

It is the same with the target, create at once for example ZZ, binarize it in 1001100 and also go ahead.

I agree, that is why it is better to use something universal from Forex without gluing.

There is not much data in one futures, and glues are distortions in prices + liquidity failures at beginnings and ends of glues.

If we work with a ready sample, it's an attempt to extract something useful from the collected data, but who guarantees that the data are useful? I'm interested in working with my predictors first, otherwise there's no point in wasting time. That's why the partitioning has to be reproducible by everyone, so that they can add their own predictors for training.

For ZZ, I don't know what your target is - how do you get money out of it? After all, in addition to markup you need the financial result of the actions of the classification.

Well, I am not interested in Forex, and on Si there are no liquidity gaps on the gluing. If it is glued incorrectly - I exclude the day from the sample. Every brokerage company has its own Forex.

 

There's something you don't understand....

Aleksey Vyazmikin:

If we're working with a ready sample, it's an attempt to pull something useful out of the collected data, but who guarantees that the data is collected in a useful way?

What do you mean useful data collected? , the initial data is ohlcv + time + Y(this data is the same for everyone)they cannot be useful or not, they just exist, this is the initial features, then from this data we already generate the features ourselves according to the criterion of improving the prediction. Did you improve it? added ohlcv+my.feature+.... and gave it back to people, another one opened it, invented a new feature, checked whether it improves the error in the test, added ohlcv+my.feature+his.featue... gave it to people and so on...

Aleksey Vyazmikin:

For ZZ, I do not know what is your target - how to get money out of it? After all, in addition to markup you need the financial result of the actions of the classification.

I was against ZZ before, but it was because of misunderstanding ... Look, you're talking about a financial result, the ideal, which should strive for is the perfect trade, those are maximum profits per unit time, if you take ZZ, then this is precisely the very same ideal trade, training AMO on ZZ is just the approach to the most efficient trading the better we get close to ZZ, the better the financial result

 
Aleksey Vyazmikin:

I am primarily interested in working with my predictors, otherwise there is no point in wasting time.

That's why the markup should be reproducible by everyone, so that he can add his own predictors for training.

Work with your chips, but on a common dataset, that's the whole point, and there will be not only your chips, but the best chips that the other best forum members came up with, can you imagine what synergy that is? How many new ideas? How much will it improve your features in total?

And the improvement will not be in words, as is customary here, one empty blah blah, but in deed ... If you know what you say, prove it, prove it on the dataset, here's your target ohlc, make a better approximation than others ... can't you do it? well then pssh and p.d.)) you know what i mean ?

ZZ is reproducible by everyone.

 
mytarmailS:

There's something you don't understand....

What do you mean useful data collected? , the initial data is ohlcv + time + Y(this data is the same for all)they can not be useful or not, they just exist, this is the initial features, then from this data we ourselves generate features according to the criterion of improving the prediction. Did you improve it? added ohlcv+my.feature+.... and gave it back to people, another one opened it, invented a new feature, checked whether it improves the error in the test, added ohlcv+my.feature+his.featue... ...gave it to people, etc...

So why take data from dataset if you can take it from the chart? Plus not everyone's target is generated on every bar.

mytarmailS:

I used to be against ZZ, but it's all because of misunderstanding ... Look, you're talking about a financial result, the ideal, which should strive to be an ideal trade, those are maximum profits per unit time, if you take ZZ, it's just that very same ideal trade, training AMO on ZZ is and is approaching the most efficient trade the better we get close to ZZ, the better the financial result

The financial result is not an ideal, but the consequences of the decision. And financial results may be different for the same target, if the system is not reversal, and the exit occurs on TP\SL.

And then, I still don't understand what your target is - write more details.

 
mytarmailS:

Work with your chips, but on a common dataset,

It's inconvenient.

 
Aleksey Vyazmikin:

So why take the data from the dataset when you can take it from the chart? In addition, not everyone's target is generated on every bar.

Fin result is not an ideal, but a consequence of a decision. And finnish results may be different at the same target if the system is not reversal, and the exit occurs on TP\SL.

And then, I still don't understand what your target is - write more details.

Aleksey Vyazmikin:

This is inconvenient.

Sorry, but judging by your questions you do not understand what I'm talking about, you know what kaggle? That's what I want to do similar to, or rather in this format. Please get acquainted and you will get rid of a lot of ridiculous questions like ...

Why do we need one dataset for all?

Why can't we take data from the chart?)

Why can't we generate a target ? each for himself ))

ect....

 
mytarmailS:

Sorry, but judging by your questions you do not understand what I'm talking about here, you know what kaggle is? here's something similar or rather in this format I want to do. Please get acquainted and you will get rid of a lot of ridiculous questions like ...

Why do we need one dataset for all?

Why can't we take data from the chart?)

Why can't we generate a target ? each for himself ))

etc....

Don't you understand that contests are meaningless (the contest winners and runners-up themselves say so)?

I'm interested in working to improve the system, not to tinker with a particular dataset.

 
Aleksey Vyazmikin:

Don't you understand that contests are meaningless (the winners and runners-up of contests themselves say so)?

What kind of nonsense is that? Where do they say that? These competitions are so meaningless that the customers of these competitions pay for the best solution to the winners by 5-50 thousand dollars. That's the kind of money they pay for pointless solutions.

Aleksey Vyazmikin:

I'm interested in working on improving the system, not in fitting it to a particular dataset.

And your improvement is not a fitting? Explain to me why your solution is not a fitting, but if you take your input data, write it into csv and synthesize your solution from this data, it is already a fitting? Have not you heard about crossvalidation?

Awesome....

Okay, forget it, you do not even have to answer, this conversation is pointless.

 
mytarmailS:

What is this nonsense? Where do they say that? These contests are so meaningless that the customers of these contests pay $5,000 to $50,000 for the best solution. Yes, they pay that kind of money for pointless solutions.

They pay for ideas in the source code, which are then tested and discarded. Not the fact that they take the winner's results. The alternative to the contest outcome is to offer the winners real work on the problem, with complete data and its interpretation.

mytarmailS:

And your improvement is not a fitting? Explain to me dumb why your solution you came up with is not a fitting, and if you take your input data write it into csv and synthesize from this data your solution is suddenly a fitting?? Have not you heard of crossvalidation?

Awesome....

Okay, forget it, you can not even answer anything, the conversation makes no sense

Can you read carefully - earlier I wrote that the target may not be taken at every bar, which prevents getting intermediate data from the CSV file.

Why would you assume that a public target would be your target?

The sample will always have the data fitted to the sample - you need validation on the data outside the sample, and for that you should be able to create a new training sample.

Look more practically at the situation without being overly selfish. And in negotiations you must learn to concede, looking for a way to the main goal, rather than grasping at irrelevant points for concession.


By the way, here's a video and channel about contests - see what the participants say.


Reason: