a question to statisticians and mathematicians

Carl Schreiber 2013.12.13 12:32

Hi,

we all back-tested EAs and got the statistical report. Then some EAs went live.

The idea is now to enable an EA to change its own risk-percent (=opening lot size) due to a comparison of the OrderHistory to the statistical results of the back-tests.

I have prepared a function to read the order-histoty:


extern double RiskPerCent       = 2.0; // 2%
extern double RiskChgCoeff      = 0.9;
extern int    MyMagicNo         = 123789;
extern string FirstDateCalc     = "2013.01.01";// for OrderHistory (date of last change)

// from backt-tests (BT) kind of expectations
extern double BT_ProfitFactor   = 1;
extern double BT_ExptPayoff     = 2;
extern int    BT_ConsLooser     = 3;
extern double BT_MaxDD          = 4;
extern int    BT_TotalTRades    = 1000;


double adjRisk(){ // parts from here:  https://www.mql5.com/en/articles/1486
        int nBu, nSe, nPos, nNeg, ConsWin, ConsLos, maxConWin, maxConLos;
        double resBu, resSe, GrossProfit, GrossLoss, maxDD, maxProf, tmpRes;
        double ProfitFactor, ExptPayoff;
        bool significant_worse = false, significant_better = false;
        if ( RiskChgCoeff > 1.0 ) RiskChgCoeff = 1.0 - RiskChgCoeff;

        int oH =  OrdersHistoryTotal();
        while(oH>0) { oH--; // hopefully from oldest to newest trades => consWinner + ConsLooser
                if( !OrderSelect(oH,SELECT_BY_POS,MODE_HISTORY) ) continue;
                if ( OrderMagicNumber() != MyMagicNo ) continue;
                if ( OrderOpenTime() < StrToTime(FirstDateCalc) ) continue;
                
                tmpRes = OrderProfit()/OrderLots(); // 1.0 lot-mormalized 
                
                if ( tmpRes > 0.0 ) {
                        GrossProfit += tmpRes;
                        nPos++;
                        if ( tmpRes == OP_BUY ) {
                                nBu++;
                                resBu  += tmpRes;
                        }
                        if ( OrderType() == OP_SELL ) {
                                nSe++;
                                resSe  += tmpRes;
                        }
                        maxProf = MathMax(maxProf, GrossLoss+GrossProfit);
                        if ( ConsLos > 0 ) {
                                maxConLos = MathMax(maxConLos,ConsLos);
                                ConsLos = 0;
                                ConsWin = 1;
                        } else 
                                ConsWin++;
                        
                } else {
                        GrossLoss = tmpRes;
                        nNeg++;
                        if ( OrderType() == OP_BUY ) {
                                nBu++;
                                resBu  += tmpRes;
                        }
                        if ( OrderType() == OP_SELL ) {
                                nSe++;
                                resSe  += tmpRes;
                        }               
                        maxDD = MathMax(maxDD, maxProf - (GrossLoss+GrossProfit) );
                        if ( ConsWin > 0 ) {
                                maxConWin = MathMax(maxConWin,ConsWin);
                                ConsLos = 1;
                                ConsWin = 0;
                        } else 
                                ConsLos++;
                }
        } // while History
        ProfitFactor = GrossProfit/GrossLoss;
        ExptPayoff = (nPos /(nPos+nNeg) ) * (GrossProfit / nPos) - 
                  (nNeg /(nPos+nNeg) ) * (GrossLoss / nNeg);

        // what would be the statistically correct criteria to decide: 
        // a) significant_worse  = true => risk decreases
        // a) significant_better = true => risk increases
        
	if ( significant_worse ) RiskPerCent = RiskPerCent*RiskChgCoeff; // decrease
	else if ( significant_better ) RiskPerCent = RiskPerCent/RiskChgCoeff; // increase
        
}

To find a good solution to set significant-worse or -better true needs to deal with sample size and the amount of the change of the values of the backtest.

This requests statistical knowledge, which I don't have, but may be someone here - and he could help?

The code above is not tested yet. If s.o. has a suggestion or sees a mistake please don't hesitate to let me know.

Thanks in advance,

gooly

Ubzen 2013.12.13 12:51 #1

gooly:

Hi,

we all back-tested EAs and got the statistical report. Then some EAs went live.

The idea is now to enable an EA to change its own risk-percent (=opening lot size) due to a comparison of the OrderHistory to the statistical results of the back-tests.

I have prepared a function to read the order-histoty:

To find a good solution to set significant-worse or -better true needs to deal with sample size and the amount of the change of the values of the backtest.

This requests statistical knowledge, which I don't have, but may be someone here - and he could help?

The code above is not tested yet. If s.o. has a suggestion or sees a mistake please don't hesitate to let me know.

Thanks in advance,

gooly

Have you seen this article? https://championship.mql5.com/2012/en/2008. I like to look at 100 Independent Positions ( not necessarily 100 deals/orders ). Or 3_weeks of trading which ever is higher. However, this really depends upon the system/trader. Should someone have a system which only trades once per week, they might choose a different matrix depending on the share_power of the system.

Big changes for MT4, Critique of MetaTrader 5 The best trading system

Carl Schreiber 2013.12.13 13:33 #2

No I did not know it but I don't think that I can get, what I an looking for:

He wrote:

The Trading Game: "The standard for a very good system is to generally take a 10 percent lower winning percentage while maintaining a 1.0 better win/loss ratio over the breakeven point. If this combination exists, you are about as close to a Holy Grail strategy as you are going to get."

So his advice is directed at the trade-logic. But it would be another challenge for an EA to change its own trading logic!

I know the problem of the sample size which can be either the time between first and last trade of back-test versus order-history or just the number of trades of both samples. But I know there are statistical methods to deal with even small sample sizes and the amount of the difference of a signal-value to be able to decide whether to change the risk (lot-size) or not.

Errors, bugs, questions FX TopGun Trading Requests & Ideas, the

Ubzen 2013.12.13 13:58 #3

The first part talks about the zero_curve and the second part talks about comparing_systems. One way of applying this is, should your Trading System Safety Factor (TSSF) fall below the green line .... then reduce the lot_size. <--this I believe is what you're looking for. The other way is trading a different system which is performing better.

But I know there are statistical methods to deal with even small sample sizes and the amount of the difference of a signal-value to be able to decide whether to change the risk (lot-size) or not.

Which methods? What do you mean by "difference of a signal-value"?

arrows in object list Spectrum derivative (or spectrum Certainly MT5 was far

William Roeder 2013.12.13 14:04 #4

gooly: The idea is now to enable an EA to change its own risk-percentCould EA Really Live By Order_History Alone? - MQL4 forum

I've been using drawdown 1) Could EA Really Live By Order_History Alone? - MQL4 forum and 2)

   double   ddFrctn        = MM_DD_Max_Prct / 100.,   // MM_DD_0_TSS1_DDS2_3
            AB             = AccountBalance(),
            perPipPerLot   = PipValuePerLot(market_pair),
               riskPips    = risk / pips2dbl,
            maxLossPerLot  = riskPips * perPipPerLot,
            chartRisk      = GetMaxRisk();
   lotsNew  = chartRisk / maxLossPerLot;     string   why = "Max Chart Risk ";
   if( (MM_DD_0_TSS1_DDS2_3 & 1) != 0 ){
      //{Trading System Solutions - Publications - Money Management - The Basics
      // of Money Management III
      //http://www.tsresearch.com/public/money_management/money_management3/
      //
      // suggests chartRisk = % Risk * (Capital – (1 – Max_%_Drawdown) * hi bal.
      // E.g. hi=110K cur=100K available = 100 - 80%*110 = 12K * riskPct

      // trade = rate (AB - 80% hibal) hibal  capital   avail  5%cap 25% avail
      // when AB = hiBal:              10000    10000    2000  500   500
      // trade = rate DDfract AB       10000     9800    1800  490   450
      // trade = percent * AB          10000     9600    1600  480   400
      // prcnt = chartRisk / AB        10000     9400    1400  470   350
      // rate  = percent / DDfract     10000     9200    1200  460   300
      // 25%   =   5%    /   20%       10000     9000    1000  450   250
      //                               10000     8800     800  440   200
      //                               10000     8600     600  430   150
      //}                              10000     8400     400  420   100
      double   percent  = chartRisk / AB,    // MMMODE_GEOMETRICAL & MM_Risk_Max
               rate     = percent   / ddFrctn,     // or equivalent percent.
               avail    = AB - (1. - ddFrctn) * history_hiBal,
               TssRisk  = avail * rate;
      if(chartRisk > TssRisk){
                  lotsNew  = TssRisk / maxLossPerLot; why = "Max DD Risk ";   }
   }  // TSS

Carl Schreiber 2013.12.13 14:29 #5

ubzen:

Which methods? What do you mean by "difference of a signal-value"?

"difference of a signal-value": I have the values from the back-test let's say: max-4-cons.-losers and avg-2-cons-lLosers. Now from the OrderHistory I count 6 cons.-losers. Now I want to know if this is statistically significant due to the numbers of trades in the OrderHistory to reduce the risk or not.

The back-test produces values that I want to use as expectant values for a statistical test.

The zero_curve does not solve my problem.- Of course I can calc the zero_curve, I can even calc it on the base of the back-test, but still I don't know whether the sample size of the order history is big enough if I compare both.

how to set the Last Closed Order with Is there a way

Ubzen 2013.12.13 18:27 #6

gooly: "difference of a signal-value": I have the values from the back-test let's say: max-4-cons.-losers and avg-2-cons-lLosers. Now from the OrderHistory I count 6 cons.-losers. Now I want to know if this is statistically significant due to the numbers of trades in the OrderHistory to reduce the risk or not.

The back-test produces values that I want to use as expectant values for a statistical test.

No, I haven't seen this kind of analysis. The closest I can think of is Z-Score. You probably do need statisticians and mathematicians to tell you if this method is viable. But they do-not come around here often. Perhaps going to a forum where they hang out and throwing this question at them would be faster.

For me, exceeding the maximum#-of-consecutive-losing-positions would be a very-very big_deal. To even consider this number, I would have had to back-test the system on all_pairs and all-data which I could lay my hands on. It wouldn't be a question of "do I have enough sample from the back-tests" ... I simply cannot get anymore samples.

I'll also have to be fairly convinced that my results are not curve-fitted. I mean ... would you flip a coin 100 times and say the maximum number of consecutive heads || tails is 7. I wouldn't, I would run the simulation some very high number of times, that when ever it does happen, if it does happen, I would consider myself extremely unlucky. Its either time to re-evaluate the system or stop-trading altogether.

and wandering around randomly Do you agree that [Branch closed!] EURUSD -

Carl Schreiber 2013.12.13 20:03 #7

ubzen,

it is not only a question of curve-fitting but the market behaviour can change as well. During the EUR-crisis every statement of a 2nd-grade, local politician caused a bigger movement in the currency which has disappeared now. The market before Lehman was different to the market after Lehman for some time.

I know I (we) need statisticians or mathematicians and even your link to the Z-code is as you said close, but it does not evaluate the results of two different samples.

The important part is the question as of which sample size a specific difference to the expected values (from the back-test) does matter either to decrease or increase the risk (lots) because the trading confirms or disproves the back-test to be sure not to be either to late or to early.

Have a nice weekend.

gooly.

PS. I am surprised a little bit. MT4 exist for so many years. One of its biggest advantages is the back-test and there is nothing that uses the back-test results to confirm or to warn the live trading.

They were waiting for me to ask this question, quel honneur ;)

Different Backtesting Results on Very strange Strategy Tester And let's make a

Ubzen 2013.12.13 20:17 #8

gooly: PS. I am surprised a little bit. MT4 exist for so many years. One of its biggest advantages is the back-test and there is nothing that uses the back-test results to confirm or to warn the live trading.

They were waiting for me to ask this question, quel honneur ;)

Yeah, there really isn't any proven (mathematical) equation from back-tests which confirms live trading. Sometimes, I think mixing speculation and mathematics might be a bad combination. You have a wonderful weekend too.

FORTS: Inefficient Transaction Fee Any rookie question, so A variant of the

New comment