Market etiquette or good manners in a minefield - page 79

 
paralocus писал(а) >>

By the way, see what a characteristic picture I get of the distribution of the first difference series over a series of transactions, if you do a vertical breakdown of the minutes with one spread (3 points):

You build the distribution function, it represents the features more clearly. What you've brought up is a view of the FR from above and a timescale resolution. Interesting, of course, but not as illustrative.

Now it's just over 2(2,153) then it gets closer to 2, then further away, depending on H, but always just over 2

The nature of the market must have been trending. In large stretches, the valuation tends to be <2.

 
Neutron >> :

You build a distribution function, it represents the features more clearly. What you've brought up is a view of the FR from above and a time scale resolution. Interesting, of course, but not as illustrative.

Now it's just over 2(2,153) then it gets closer to 2, then further away, depending on H, but always just over 2

The nature of the market must have been trending. On the larger plots, the estimate is usually <2.

I'll build it today. I have a really small period of just over a month (20 000 min) and my brokerage company does not allow to download a longer history of mines.



I had an idea last night and I'd like to voice it here. Applies to ALL, gentlemen.

Let's call it SOI (so that nobody guessed it -:) )


So, for a proper data analysis we need a good (statistically sufficient and without gaps) tick history, which we do not have for obvious reasons.

The essence of the idea is how to create it with the least effort and the greatest benefit to any trader. The solution is simple - rent dedicated servers (2, or better 3) and install tick collectors on those servers... on any tool and brokerage company. And not only that (more on this later). One person this venture will be very expensive, but the idea is just how to do it is not free, but cheap, reliable and practical.


Let's do the math:

****************************** *************************

1. renting one dedicated virtual server with enough space today costs around 100 quid a month (you need 2 independent servers) ... 200 bucks/month.

2. It would take a good coder to keep an eye on the whole thing and patch the story properly.

The coder should of course receive some money for this work. The daily work there will not be much, so 200 quid / month (however, this is not the limit of what the coder can earn in the project)

That is, in fact, all the basic costs! 400 quid a month.


You will also need a unified online resource, which must be placed:

a) A marketplace for selling teak history.

b) A marketplace for selling EAs, where you can watch the current work of any of the proposed EAs on the demo for FREE.

c) A forum where constructive criticism of DC is acceptable.



***************************** REVENUE **************************

1. Payment for downloading tick history - 1 month ticks for any instrument of any brokerage company - 1WMZ (payment is convenient via WebMoney)

2. Fee for hosting advisors on a remote server (work for real) - 20WMZ per week (if advisor does not earn so much, just throw it away)

3. Demo advisor hosting fee - 5WMZ per week

4. Interest from advisors, that were sold through the platform (I can't tell the exact figure)

******************************************************************************************


In general, the project looks quite profitable and useful. MetaQuotes will probably also be interested, because SOI will make a good PR for their product - MT5.

In a word - everybody wins, except DCs. Although, honest brokerage companies have nothing to fear.

For the start of any concession we need shareholders. I personally am ready to share money in amount of 100WMZ. Who else ...?



 

1. I collect ticks for my own needs. Enough is enough.

2. As a commercial project, this project is inferior, in my opinion, to another project under the brand "Forex".

3 If you need ticks only for analysis, you need to show the feasibility of the project first. Show the real income on ticks and prove its superiority to other ways of representing the price series. It is clear that references to authorities are not valid.

And the last one. Everyone trades in the history of his brokerage company with its own unique quotes filter. For this particular brokerage company NS is trained. Consequently, we are not talking about universality of the proposed project! Who will be interested in it, but you? 1 or 2 more people...

 
Yes, about the filters, I hadn't thought of that. Although I wrote about it as recently as yesterday... Thank you.
 
Have you managed to normalise the inputs to a constant distribution? I've been struggling with a similar problem recently. Describe the essence of rationing with an example, I want to make sure that I understand your difficulties correctly. If I understand it correctly, I will be able to suggest a solution.
 
I have a different kind of difficulty.
 

Question to Neutron and paralocus. I understand that Neutron trains its grid by a modified RPROP method, while paralocus trains its grid by a modified BPROP method. In the classical RPROP method, the weight steps are calculated by multiplying the gradients of the training error squared (dE^2/dw = -delta*Neuron_Input) by the learning rate. This method can be applied locally, when moving from one training set to another, or epochally, by accumulating error gradients for all training sets and then calculating the weighting steps. Your modification is that the weight steps are additionally multiplied by (1-L/N)*dw/sqrt(dwNorm). Neutron only takes incremental signs of dw, as in RPROP. I've tried almost all kinds of RPROP methods (BPROP, RPROP, IRPROP+/-, SARPROP, GRPROP) and got pretty dismal results. When weights are initialized with some random numbers (say, in the range of -0.5...0.5), initial learning error is very high and the learning methods successfully reduce it. There is no problem here. But if weights are initialized with zero values or very small random values (e.g. -0.05...0.05), then the initial learning error is small and equal to the ultimate learning error when weights are initialized with "large" random numbers. Then what is the point of training the network if we know that zero weights give us an error close to the final learning error when the initial weights are random? Here is a graph of training error with modified ORO method like paralocus (judging from earlier posts, Neutron used the same method in the beginning)


All training methods I have mentioned show approximately the same behaviour: a nice and inspiring reduction of the learning error from epoch to epoch at random initial weights (one can write a paper or a dissertation). At zero initial weights the error is already small enough that the network does not improve it at all. Here is the example of IRPROP+ training with the same input data



May be, such behaviour of training error is specific to my network (two layers, 12 inputs, 5 hidden neurons, 1 output, 120 training sets) or to input data (relative price increments with different delays). I would like to see how your networks learn with random and zero initial weights. If you don't mind, show me graphs of NON-NORMED learning error as a function of epoch for these two cases.

Here I was thinking a little bit about my result and came to a conclusion that if weights of the network tend to zero in the learning process or in other words zero initial weights are close to trained state of the network, then the network tries to describe very weakly correlated inputs and outputs, i.e. random market in which prediction is impossible. I have "invented" such a theorem


 
gpwr писал(а) >>

Maybe this behaviour of learning error is specific to my network (two layers, 12 inputs, 5 hidden neurons, 1 output, 120 training sets), or to the input data (relative price increments with different delays). I'd like to see how your networks learn with random and zero initial weights.

This is what the learning error reduction process of a bilayer NS looks like, depending on epoch number:

The network has 5 inputs, 4 neurons in the hidden layer, and one neuron in the output. EURUSD1h open prices normalized to its standard deviation were used as a training vector. In Fig. the circles show the training error found as a standard deviation averaged over 50 numerical experiments without normalization of the error to the length of the input vector. The thin lines show the statistical scatter at the 1/e level. Blue shows the implementation of this process for the initial randomization of weights in the range 0. Red shows the +/-1 range with a band-shaped distribution.

It is seen that NS learns much better if the initial initialization of weights by random values is carried out, in comparison to starting from zero. Moreover, increasing the range of starting randomization of weights leads in general case to reducing the final learning error. However, this entails an increase in the number of epochs needed (the graph, as a whole, goes up).

 
Neutron >> :

This is how the learning error reduction process of bilayer NS looks like depending on epoch number:

The network has 5 inputs, 4 neurons in the hidden layer, and one neuron in the output. EURUSD1h open prices normalized to its standard deviation were used as a training vector. In Fig. the circles show the training error found as a standard deviation averaged over 50 numerical experiments without normalization of the error to the length of the input vector. The thin lines show the statistical scatter at the 1/e level. Blue shows the implementation of this process for the initial randomization of weights in the range 0. Red shows the +/-1 range with a band-shaped distribution.

It is seen that NS learns much better if the initial initialization of weights by random values is carried out, in comparison to starting from zero. Moreover, increasing the range of starting randomization of weights leads in general case to reducing the final learning error. However, this entails an increase in the number of epochs needed (the graph as a whole goes up).

Thanks. Why is the statistical spread drawn near the blue circles? If the weights start with zero values then there shouldn't be any statistical scatter.

One more question. If you used EURUSD1h opening prices normalised by their standard deviation, then their average is not zero. Or have you subtracted the average?

Reason: