Machine learning in trading: theory, models, practice and algo-trading - page 3508
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
The point is not to compare my method in the end (it is not perfect), but to show in experiments that with each iteration the probability of choosing an efficient split (which will be with the same bias vector on new data) decreases and this is, in fact, the reason for obtaining unstable models.
In doing so, I detailed the situation on an iteration-by-iteration basis for each predictor, and showed that there are ranges in which the predictor gives quantum splits with a high probability of stability on new data. It is different for all predictors - hence the conclusion about the importance of the sequence of predictor selection for the split.
The question is how one can influence the probability of selecting an efficient split (in my case double split - quantum cutoff). If one can increase the probability, then there will be fewer erroneous splits.
Terrible) In response to an attempt to normalise the use of one term, you just spill a bunch of new terms) Obviously, this algorithm will never converge to anything reasonable and publicly available).
Terrible) In response to an attempt to normalise the use of one term, you just spew out a handful of new terms) Obviously, this algorithm will never converge to anything sensible and universally accessible)
Who wants to understand, asks what he doesn't understand.
Who wants to understand - asks what he doesn't understand.
You seem to be alone in not understanding the algorithm.....
Who wants to understand - asks what he doesn't understand.
Well, I asked you why you think your algorithm is not greedy (in the conventional sense) - I didn't see any answer.
Continuing my experiments, last past post was here.
Let me remind you briefly, the essence is to take a sample, split it into quantum segments, evaluate these segments for stability of the probability shift relative to the binary probability of all rows of the subsample. We select from the selected set of quantum segments the quantum segment that we consider to be the best candidate for selecting the region with negative class. We remove from the sample the range that fell into the quantum segment and repeat the iteration. In essence, this is very similar to building tree models, but here stump selection takes place. Gradually the sample decreases, the probability of a positive class grows, which should eventually lead to a positive balance on the sample outside of training (exam - I call it).
Earlier I have already published a graph with falling probability of choosing an effective quantum segment at a large number of iterations. Let me remind you how it looks on a concrete example for two classes.
The graph shows that with increasing iterations the probability of choosing the correct (with positive probability shift on new data) quantum segment (split by range) decreases.
I decided to look at how this probability manifests itself on a particular predictor - is there stability from iteration to iteration. At each iteration I counted the number of selected quantum splits and the number with positive probability bias on new data, and then determined the percentage of the latter. It turned out that there are cases when quantum segments from the predictor were not selected at the iteration - then the value of the index is zero, but if they were selected, but there are no correct (necessary/effective) ones among them, then the value gets -100.
The zoo turned out to be quite diverse, here are some samples.
It can be seen that the probability decreases in different ways, both smoothly and by jumps within individual predictors, why it is so different is not clear yet.
It also seems interesting that there are predictors that supply quantum segments that pass selection on the training sample, but on new data they are not effective at all - they will interfere well with training - we need additional methods to reduce the probability of hitting them.
It also seems interesting that predictors become a supplier of quantum segments (selected) not necessarily from the first iterations, which leads to the idea that it is important to follow the sequence of predictor use at different depths of the tree.
If you have any thoughts, please write. I am thinking about some aggregate metric for evaluation of the process, that it would be possible to twist the settings of selection of predictors and see their influence on the change of the result.
## Analysis of your experiment on quantised segments and their effectiveness
Your experiment on quantized segments and their effectiveness in building a model is fascinating. Here's my analysis of your findings:
**Key Observations:**
* The number of selected quantised segments for each target does not show a linear relationship with the iteration number.
* There seems to be a correlation between the number of targets and the number of selected segments in each iteration.
* The percentage of stable quantised segments for each target exhibits an inverse correlation.
* There is a significant drop in the chance of selecting a good quantised segment for target "0" around the 40th iteration, followed by a rebound to the initial probability.
* The probability of selecting an effective quantised segment decreases with the increasing number of iterations.
* There is a variation in the effectiveness of quantised segments across different predictors.
**Possible Explanations:**
* The inverse correlation between the number of targets and the percentage of stable quantised segments could be due to the fact that with more targets, the model has more opportunities to find good segments, leading to a higher chance of selecting less effective ones.
* The drop and rebound in the chance of selecting a good segment for target "0" could be related to the specific characteristics of the data or the model itself. Further investigation is needed to understand the exact reasons.
* The decrease in the effectiveness of quantised segments with increasing iterations might be due to the fact that the model becomes more selective over time, leading to the selection of less effective segments.
**Further Research Directions:**
* Investigate the reasons for the inverse correlation between the number of targets and the percentage of stable quantised segments.
* Analyse the specific characteristics of the data and the model that lead to the drop and rebound in the chance of selecting a good segment for target "0".
* Explore methods to mitigate the decrease in the effectiveness of quantised segments with increasing iterations.
* Develop a comprehensive metric for evaluating the process of selecting quantised segments, taking into account factors like stability, effectiveness, and diversity.
**Additional Considerations:**
* It would be interesting to compare the performance of your approach with other methods for selecting features or building models.
* Exploring different strategies for selecting and using quantised segments might lead to further improvements in the model's effectiveness.
Overall, your experiment provides valuable insights into the dynamics of selecting and using quantised segments for building a model. Further investigation and optimisation could lead to significant improvements in the model's performance.
Scored your last few posts on chatgpt4:
Continuation:
If you remove the target variables from your method, it can become similar to hierarchical clustering in the sense that both approaches can be used to cluster data without regard to pre-known class labels. Hierarchical clustering is a data analysis method that seeks to build a hierarchy of clusters. The main difference between hierarchical clustering and other types of clustering is that it does not require the number of clusters to be specified in advance. Instead, the result is a dendrogram that allows us to understand the relationships between individual objects and groups.