Machine learning in trading: theory, models, practice and algo-trading - page 1101

 
Alexander_K2:

Yeah. That's the trick... Otherwise, we would have bought a bigger purse a long time ago...

I think it's time for you to find an outlet in Hilbert space, where you yourself come from, instead of draconian 2-dimensional increments in search of patterns in this primitive world of pixels :)

 
Maxim Dmitrievsky:

I think it's time you found an outlet in Hilbert space, where you yourself come from, rather than draconian 2-dimensional increments looking for patterns in this primitive world of pixels :)

Agreed.

I'll wait for the neuroGraal signal. I believe it.

 
FxTrader562:


Buddy, when are you going to make the Grail?

 
Alexander_K2:

Once again I publish the charts:

GBPJPY 2018.

Upper - the price itself.

Lower - sum of CLOSE M1 increments in the sliding window - week. In fact, detrended price with expectation = 0 and variance = practically const. A perpetual flat, so to speak...

Maybe someone can use it...

You don't understand, I just need to mark the flat, on a normal chart without any magic tricks


 

Found it on a neighboring forum:


https://www.elitetrader.com/et/threads/which-one-of-you-guys-made-this-reddit-post.323908/

Some final words

I am a professional quantitative portfolio manager, who has been in the industry for a very long time, and works on the bleeding edge of ML and applied mathematics with focus on the capital markets - I manage $100mn+ these days. I created this account to write on /r/algotrading so that I can interact with a few people on this sub, but as I have seen, this sub is filled with amateurs and it is just annoying reading the feeds most days. I am going to delete my account and I wanted to leave a few points that I hope with help a few people here,

BTC and other crypto-coins are nothing more than another asset. Stop putting it on a pedestal or thinking its anything different.

ML is super hard when applying to financial markets, and its not something anyone can figure out very easily. Most amateurs can play around with RNNs and have a descent strategy, but don't think its going to give you anything extraordinary. It's just another tool in your toolbox to create a strategy.

ML can be used to make some amazing automated trading systems, but it won't be possible for 99.999% of people. People have been doing ML for trading for a very very very very very very long time. You are being exposed to it just now because there are lots of tools and lots of resource that wasn't accessible before. Do not think taking tensorflow, sklearn, <insert library name here> and it will magically make you money. It takes a very long time, ie. decades to get anything automated to the level most of your dreamers think.

Most of you are software engineers here. Stop thinking like one. Writing a new shiny backtesting tool or trading framework is not going to do anything but waste your time. Stop talking about languages, it really doesn't matter. Work on your alpha. Yea, its the thing that you don't know how to build, work on that. Trading frameworks come after.

Anything that works on the intraday time-frame is considered HFT. Stop thinking that its only low-latency stuff, its basically what timeframe most of you are trying to make money in. People can do this, but, you need to find that thing that most of you avoid - alpha. Most people can't succeed here, so most of you, do yourselves a favor, trade daily+ timeframes, it will save you some frustration.

If you have capital, make a portfolio of a few nice assets. Start with management accounting principals and work from there to figure out what makes one asset worth more than another.
Stop asking people where to begin, how their stuff works. MONEY is involved here, no one will help you with anything. No one is going to tell you anything more than what I have said in the few points above. And the people who tell you things, are usually negative such as as TA is bullshit or ML won't work or HFT is only latency sensitive stuff - well, most of them are idiots who don't know what they are talking about. Let me tell you clear and simple here - TA is not bullshit, it's just mathematical transforms and features that MIGHT contain predictive power, ML can be used very well to make a lot of money, and HFT is anything on the sub-daily timeframe and a lot of strategies are not latency sensitive.

Lastly, there are VERY smart people in the world, who have spent their entire lives studying, building and creating technological and scientific advances more than most of the people here can fathom. These people work in this industry and make a ton of money. I am happy that you saw some documentary of how a lot of people made money in the 70s and 80s trading and you want to be like them. Sorry, the world is different, with the availability of information and higher education standards, the bar to be good in this industry is very very very very high. So, you need to be a good scientist or have that mentality today to be good in this industry. Its great you want to be like the best of this industry, so start with being humble.
Anyways. Good luck and goodbye.

- xxzam




Sync & corrections by n17t01 www. addic7ed. com

Last Will

I am a professional quantitative portfolio manager with a very long time in this industry, particularly in machine learning and applied mathematics with a focus on capital markets, today I manage over $100 million. I created this account to write about r and algotrading so that I can contact several people on this topic, but, this thread is crowded with amateurs, which is often annoying. I'm going to delete my account and want to leave a few last words that I hope will help some people.

Bitcoin and other altcoins are nothing more than just another asset. Stop putting them on a pedestal or thinking of them as something fundamentally different from other financial instruments.

Machine learning is extremely difficult to apply to the financial markets, it's not something anyone can easily do. Most beginners play with RNN and get strategies that fail, do not think that this will give you anything extraordinary. It is just another tool in your toolbox to build trading systems.

ML can be used to get great trading systems, but that is not possible for 99.999% of people. People have been using ML for algotrading for a very, very, very, very, very long time. You're doing it now because there are a lot of resources and tools that were not available to the public before. Don't think that taking tensorflow, sklearn, <insert library name here> and it will magically make you a lot of money. It takes a very long time, that is, decades, to reach the level of quality and automation you dream of.

Most of you here are programmers. Stop thinking the same way. Writing a new radiant backtester or trading platform will do nothing more than waste your time. Stop thinking about languages, it really doesn't matter. Work on Alpha. Yes, that's the thing you don't know how to do, work on it. Leave the trading frameworks for later.

Anything that works on intraday time frames can be considered high frequency trading. Stop thinking of HFT as nanosecond trading extremely sensitive to order execution speed, it's just another time frame for making money. Lots of people do it, but you have to find what most people avoid - Alpha. Most people won't succeed at HFT, so do yourself a favor, trade medium term timeframes(days, weeks), it will probably save you from quick disappointment.

If you have capital, make a small portfolio of a few good assets. Start with management accounting principles and work from there to figure out what makes one asset better than another.

Stop asking the public where to start and how their algorithms work. There is MONEY involved, no one is going to help you with this. No one will tell you anything more valuable than I have said in previous paragraphs. And the people who will trash technical analysis, say that ML doesn't work and HFT only depends on order execution speed, well, most of them are idiots who don't know what they're talking about. Let me be clear here that TA is not crap, it's just mathematical transformations of data and signs that MAY contain predictive power, ML can be very productively used and move a lot of money, and HFT is just trading on an intraday scale where many strategies are not very sensitive to time lags.

Finally, there are VERY smart people on earth who spend their lives studying, creating and building scientific and technological advancements more than most other people can comprehend. These people work in this industry and make a lot of money. I'm glad you watched the documentary of how people traded in the 70's and 80's and you want to be like them. Sorry but the world has changed, the threshold for entering this field is unattainable. So you have to be a great scientist or have similar mental ability to be in this industry. It's good that you want to be like the best in this field, but you have to start with humility.

Anyway, good luck and goodbye.

 
mytarmailS:

Look, I'm not too up on this stuff, to be honest I'm not up on it at all ... but you're saying that the market is random, and it's not.

The market movements depend on the actions of market participants, on purchases and sales, right?

Of course it is, it turns out that participants make random deals, they lack reason and logic, so they are not people but atoms in boiling water that do something chaotically

Maxim, are your deals on the market random?

No, you train the net on the history and using the statistical advantage you open deals, I'm telling you, everybody does it, this is the so-called "market memory".

And there are also levels in the market, for example when you buy something at 100 the price squeezed and you were not closed by the stop at 99 and the price is now at 75 and you are sitting in a loss praying for the price to come back, when it comes back to 100 you have to close your buy, then you make a sale and the price falls. That's the level, not the extrema.

Of course, you won't be alone there because the crowd would enter at the same points.

The levels no one takes into account when they do tests for Brownian motion or not.

This is all true, but the factors are much more than those you announced, and those of them, which are speculative, of which you speak, have negligible influence, especially in forex, if we talk only about "pure" fundamental and the hierarchy of levels of access to insider information, As well as purely random large transactions, which often occur regardless of the market situation, you can add here different "anti-trade" strategies that work with capital able to withdraw stops and knock out levels not from suckers with DTs but on the interbank, then everything becomes quite "effective" and almost random, as it is observed in real life.

 
mytarmailS:

We do not know, I agree, but indirectly we can try to identify ...

1) the big player is a counter-agent of the crowd, he won't go in without the crowd, there is nobody to open and close the position with

2) The crowd works on the statistics "market memory" according to the principle - do it now, it was profitable to do it in the past

This is the moment when we can play with indirectly identify areas where the crowd will do something unambiguous.


For example here, the market has been saying to buy for almost the whole day, and we could already know yesterday that there will be a fall today

The green dots are overbought, the crowd was buying the whole day yesterday


It's very rare for the whole day

I have already showed the part of this indicator in work to the Wizard, but he did not take it seriously. I asked Maksim to help with MT4, but he had too busy with his business.

It is possible to make predictions, but it is difficult and not always possible.

The idea of detecting the "crowd strategy" is grey-haired and grey-haired and unfortunately it is no more effective in practice than searching the correlation of patterns in the past and extrapolating the future of "winners", what probably every algorithmic trader involved in. The reason is that "the crowd" is very inhomogeneous, so much so that it does not act coherently as we would like it most of the time, besides the statistical advantage when predicting one hour ahead direction, using a lot of data and the top algorithms ML does not exceed 2-3%, that is not statistically valid, there is nothing to play against, and the classical TA in its pure form is noise.

 
Grail:

So many words, but only water... common phrases of crowd-pleasers.

zy. I'm talking about you, not this guy from the forum.

 
Grail:

Found it on a neighboring forum:


Yeah, it's okay.

You have to work hard to achieve anything in this field. And then it depends on the innate abilities. And of course the discussion MO noubami cause emotion, the same when you remember how yourself started to learn the basics and tried to be clever

 
mytarmailS:

so many words, but only water... common phrases of crowd-pleasers.

zy. I mean you, not this guy from the forum.

Not a full 8 lines:)

I can make it even shorter: the "crowd" in the market cannot be detected by means of statistics (MO including), what to play against it

For you it is a super valuable idea so far, that's why you are poking me and emotionally condemn my criticism, but when you check it yourself, you will quietly agree.

Reason: