OpenCl and the tools for it. Reviews and impressions. - page 7

 
icas:
"That's right, that's right, isn't it? But there's another side to the coin." ("Captive of the Caucasus", C). The meta-quotes are finally "keeping up with the times".

Have they decided to keep up with iron for some reason? And iron ages the fastest.


Normal heroes always go the other way around (c) It's a song from some movie.


I personally am not in the least bit inspired by the fact that in order to get the speed with MT4-level optimization, I'll have to look for some kind of video cards. And optimization is the most problematic part in autotrading in terms of computational cost.

Moreover, if tomorrow the ironmongers will think up some other feature which will give them a more acceptable solution of the same problems, it will be a pity for the meta-quotes. After all, hardware developers are the least conservative in their ideas and at the first opportunity they will try to implement something so as to abandon the previous crutches and get ahead of the competition.

The most effective way to increase productivity of software is to improve its algorithm. All other attempts to deviate from this path, and especially those dependent on other developers, often end very miserably.

 
Reshetov:

Personally I am not at all inspired by the fact that I have to look for some widgets to achieve the speed of optimization at the MT4 level. And optimization is the most problematic part in autotrading in terms of computational cost.

Moreover, if tomorrow ironmongers think of some other feature that will give them a more acceptable solution for the same tasks, it will be a pity for the metaquotes. After all, hardware developers are the least conservative in their ideas and at the first opportunity they will try to implement something so that they could forget about previous crutches and outrun their competitors.

The most effective way to improve the performance of software is to improve its algorithm. All other attempts to deviate from this path, and especially those which are dependent on other developers, often end very poorly.


Try MQL5 Cloud Network - the speed of optimization is as high as never before seen in MT4. In massive calculations, it will be amazing how hundreds of results per second will fail when using a couple of thousand independent agents.

I'm not sure that GPU computing will bring an increase in the basic trading algorithms but we have far reaching plans. In the next builds there will be a lot of interesting things - it will be possible to build the most powerful calculation farms with data exchange and results postanalysis.

 
Renat:

I'm not sure that GPU computing will give an increase in basic trading algorithms, but we have far-reaching plans. The next builds will be very interesting - it will be possible to build the most powerful calculation farms with data exchange and results postanalysis.

Wow! Does this mean that the user will have access to the results of agents' calculations?
 
joo:
Wow! Does this mean that the user will have access to the results of the agents' calculations?

Exactly.

You can transfer large amounts of data from the agents back to the host terminal, and when the calculations are completed, automatically run the OnTesterFinalize() method of the same Expert Advisor on the terminal itself. Thus, it will be possible to perform complex post-processing of test results.

 
Renat:

Exactly.

It will be possible to transfer large amounts of data from the agents back to the host terminal, and when the calculations are completed, automatically run the OnTesterFinalize() method of the same Expert Advisor on the terminal itself. Thus, it will be possible to perform complex postprocessing of test results.

Cool!!!

Also, before it is too late. The OnTesterFinalize() method for agents would also be desirable (could be the same, but with host/agent input parameter). The idea is to create arrays in map files (e.g. cache heavy indicators) on agent machines, and delete them correctly in the end.

 
Renat:

Try MQL5 Cloud Network - the speed of optimization is as high as never before seen in MT4. It is amazing, how hundreds of results per second using a couple of thousand independent agents can fall down on massive computations.

Yes, I have already tried it on my local 2 core computer. I was disappointed with the results. The MT5 engine can hardly run on 2 cores while MT4 is optimizing very fast.

Theoretically, of course, one can launch additional computers connected to the local network and the performance will increase due to that. You can have more video cards installed in your computer which would increase performance. But it is not the solution when you have to buy a server rack in order to increase optimization speed in the tester.

Of course, remote distributed computing can also be used, although the service is not free.

But the point is that MT4 productivity is now enough for me. And since MT5 becomes more and more dependent on hardware, I don't want to migrate to such a platform.


Renat:

I'm not sure that GPU computing will give an increase in basic trading algorithms, but we have far-reaching plans.

To put it in perspective, GPU computing for anything other than computer games is not very useful so far. Here the story is quite different. Namely, there is a paradox that 3D graphics, ie images and animation is created (rendered) on the main computers processors. On the one hand would seem to produce graphics cards with powerful processors for graphics with high-speed memory, but on the other hand, all who have to do with 3D, do not buy fancy graphics, and purchase multi-core processors and conventional, but high-speed RAM, sometimes combining all this stuff into farms and render. Better yet, the results of rendering can also be seen on a fairly cheap graphics cards (there is a big role played by the quality of the image on the monitor).

It is clear, that video card manufacturers are not happy with this situation, because profits flow by, so they for their part and make attempts to control the situation under themselves. But so far very unsuccessfully. I.e. software for rendering for certain lines of video cards appeared, but if calculate, it turns out that the quality / price ratio is not in favor of the GPU. Advertising agency or printer wants to wait for rendering 3D image on a usual multi-core or on the farm, but with the highest quality, rather than to get quickly on the GPU and then try to fix bugs. Renderers are in no hurry to switch to the GPU, because in this case, their software would become dependent on videocard manufacturers, and being tied to a particular hardware would only discourage customers.

If MetaQuotes was engaged in developing farms for 3D graphics, then this step would have been understandable. But IMHO, it is a waste of time, because it would be better to wait until the graphics card manufacturers have calibrated their products, rather than push too far into the thick of things.

Although it's not for me to decide, as I'm the boss. But the point remains, i.e. there is no desire to switch to hardware-dependent software and it will not appear soon.

 
Reshetov:

Yes, I tried it already, on a local 2 core computer. The results are disappointing. I have tried it on MT5, while MT4 shows very fast optimization.

Have you tried MQL5 Cloud Network with 2000 agents on a dual core?

Or did you just run a single local pass and rest on that?

It is with the MQL5 Cloud Network that we have achieved a simple acceleration of hundreds and thousands of times. Without GPU, on any Expert Advisor. Take my old example with video: Example of calculation acceleration using MQL5 Cloud Network


Theoretically, of course, you can run additional computers on the local network and the performance will increase because of it. It is also possible to have more video cards plugged in and performance would increase. But this is not the solution, when in order to increase optimization speed in the tester, you will need to buy a server rack.

Of course, remote distributed computing can also be used, although the service is not free.

We have made it possible to instantly raise thousands of agents with one button (network warm-up time about 20-30 seconds) for quick task calculation. Instead of hundreds of hours, you can really get it done in half an hour.

Moreover, the prices are ridiculous - anyone can check them, and everyone who signs up at MQL5.com gets a bonus of $2.

This is a real revolution. Now any user can put almost any supercomputer to shrugs through the use of the cludes. And with the introduction of the GPU (a build will be available on Friday) it will be possible to beat supercomputers in packs.


But the essence of the matter is that for now MT4 productivity is quite enough for me. And as MT5 becomes more and more hardware dependent, I don't want to migrate to this platform.

The productivity is enough (or rather you assure yourself of it) for simple cases. For massive and detailed calculations there is not enough power.

The mistake with comparing MT4 vs MT5 testers is that people turn a blind eye to the vastly different level of detail and ability to accurately multicurrency test.

You missed your mark about hardware dependence too:

  • MT5 supports both 32 and 64 bit architectures (the 64 bit version is actually faster and more powerful)
  • MT5 uses all local cores
  • MT5 uses remote agents (32 and 64 bit)
  • MT5 uses the MQL5 Cloud Network when a simple notebook receives calculation results from computers several thousand times more powerful than it
  • MT5 uses hardware-independent OpenCL, which allows you to work with a variety of GPU processors

You are blatantly wishful thinking, clearly aware that you are wrong.


It is clear that videogame developers are not satisfied with this situation as profits flow by, so they make attempts to take the situation under their control. But so far very unsuccessfully. I.e. software for rendering for certain lines of video cards appeared, but if you calculate it, it turns out that the quality / price ratio is not in favor of the GPU. Advertising agency or printer wants to wait for rendering 3D image on a usual multi-core or on the farm, but with the highest quality, rather than to get quickly on the GPU and then try to fix bugs. Renderers are in no hurry to switch to the GPU, because in this case, their software will become dependent on videocard manufacturers, and being tied to a particular hardware will only discourage customers.

While I am not a GPU advocate (have been outspokenly against it for a long time), your conclusions are wrong.

There is no software for GPU, because developers have not yet believed in this direction, but not reiterating the mythical binding to a specific hardware. For the sake of decency, one should at least read what OpenCL is and what it was invented for (hardware and platform independent).

Companies are only forced to use software rendering because of inertia and established processes. Getting a company to change software is a titanic effort, often impossible for the next 3-5 years. Given the leap that GPUs have made in the last 3 years and the clearly visible emphasis by their authors on universalization, it is reasonable not only to expect improved results, but also to actively pursue new features.


If MetaQuotes was developing farms for 3D graphics, then such a step would still be understandable. And now, IMHO, it is just a waste of time, because it would be better to wait until the video card manufacturers have everything settled, rather than push too far into the thick of things.

Although it's not for me to decide, as I'm the boss. But the point remains, i.e. there is no desire to switch to hardware-dependent software and it will not appear soon.

We've waited long enough, we've also criticised and disagreed, but the time has come.

And this is the third time the repetition on 'iron-dependent software' has taken place.

 
Reshetov:

Yes, I tried it already, on a local 2 core computer. The results are disappointing. The results are disappointing. I can barely make it on MT5 using 2 cores while MT4 is fast optimizing.

Mainly, all else being equal, testing speed on MT5 is slower than on MT4 because MT5 uses better tick history modelling. It greatly reduces the speed. But, there is a hope that MQ will implement simpler tick generation methods (and variants of testing without ticks at all) like in MT4 (there are many such requests, and looking at recent innovations, there is a hope that it will be implemented).

Reshetov:

If I want to get into details, GPU computing is not very useful for anything except computer games so far. The story is quite different here. Namely, there is a paradox that 3D graphics, ie images and animation is created (rendered) on the main computers processors. On the one hand would seem to produce graphics cards with powerful processors for graphics with high-speed memory, but on the other hand, all who have to do with 3D, do not buy fancy graphics, and purchase multi-core processors and conventional, but high-speed RAM, sometimes combining all this stuff into farms and render. Better yet, the results of rendering can also be seen on a fairly cheap graphics cards (there is a big role played by the quality of the image on the monitor).

It is clear, that video card manufacturers are not happy with this situation, because profits flow by, so they, from their side, are trying to win the situation under themselves. But so far very unsuccessfully. I.e. software for rendering for certain lines of video cards appeared, but if calculate, it turns out that the quality / price ratio is not in favor of the GPU. Advertising agency or printer wants to wait for rendering 3D image on a usual multi-core or on the farm, but with the highest quality, rather than to get quickly on the GPU and then try to fix bugs. Renderers are in no hurry to switch to the GPU, because in this case, their software would become dependent on videocard manufacturers, and being tied to a particular hardware would only discourage customers.

If MetaQuotes was developing farms for 3D graphics, then this step would be understandable. But IMHO, it is a waste of time, because it would be better to wait until the video card manufacturers have everything settled, rather than push too far into the thick of things.

The situation is almost exactly the opposite. Everyone thinking about the future is switching to GPU computing. They are building entire supercomputers out of videocomputers.

GPUs nowadays calculate everything, video, audio, photos, and even strength calculations in dynamics and statics.

Unofficial information:
General information about CUDA
http://www.gpgpu.ru/
http://www.computerra.ru/interactive/423392/
http://cgm.computergraphics.ru/issues/issue16/cuda
http://www.ixbt.com/video3/cuda-1.shtml
http://www.ixbt.com/video3/cuda-2.shtml
http://www.thg.ru/graphic/nvidia_cuda_test/print.html
http://www.thg.ru/graphic/nvidia_cuda/print.html
https://ru.wikipedia.org/wiki/CUDA

CUDA documentation
http://www.uraldev.ru/articles/id/33
http://www.steps3d.narod.ru/tutorials/cuda-tutorial.html

CUDA examples and articles
http://www.codeproject.com/info/search.aspx?artkw=CUDA

Printed literature:
http://www.ozon.ru/context/detail/id/5080841/
http://www.ozon.ru/context/detail/id/5432077/


Official information:

Related technologies
http://www.khronos.org/opencl/

http://www.nvidia.ru/object/cuda_home_new_ru.html
http://nvidia.custhelp.com/cgi-bin/nvidia.cfg/php/enduser/std_alp.php

http://developer.nvidia.com/object/gpucomputing.html
http://developer.nvidia.com/object/cuda_3_2_downloads.html
http://developer.nvidia.com/page/documentation.html
http://developer.nvidia.com/object/nsight-downloads.html

CUDA code examples

http://developer.download.nvidia.com/compute/cuda/sdk/website/samples.html

Applications

http://www.nvidia.ru/object/cuda_research_ru.html

Demos from the community

http://www.nvidia.ru/object/cuda_apps_flash_new_ru.html


These are mostly CUDA links (pulled from the stash) but everything still applies to OpenCL in its entirety. Besides, the future of OpenCL looks brighter (hardware-independent technology) than that of CUDA (hardware-dependent technology).


Reshetov:
Although it's not up to me to decide, as the master is the boss. But the essence remains, i.e. the desire to switch to hardware-dependent software does not and will not soon appear.

No one is stopping you from writing programs the old fashioned way, for the CPU. All programmes will work.


SZ Renat beat me to it.

 

joo:

...

ZS Renat beat me to it.

Double strike. Information is never superfluous. :) Very informative branch, it should even be fixed, so it doesn't go down.
 
icas:
Renat, will there be OpenCL support in MT4?

Unfortunately, no.

But a big update of MetaTrader 4 is coming out next week and the new MetaTrader 4 for iPhone/iPad is coming out this week.

Reason: