Requirements for CPU when increasing the number of securities in trading. (with 300 – N thousand securities)
At the moment, OsEngine allows for trading on platforms as a WHOLE. That is, simultaneously for 600 futures on FORTS and 250 stocks on SPOT.
This post contains some tips on how to trade multiple sources simultaneously without issues and what to pay attention to.
1. The main task is parsing messages from the API.
Technically, if we disregard the load on trading logic, the main task becomes parsing the queue from the API when 500 or 1000 securities are connected.
1. A market depth update arrives – 20 transformations from string to number. 2 search operations. 5 transition operators. And this happens 1,000 - 20,000 times per second.
2. A trade comes in – 3 transformations from string to number. 2 search operations. 2 transition operators. And this happens 1,000 – 30,000 times per second.
3. And so on.
This is a serious load on the central processor.
The more securities we connect, the greater the load on the CPU in parsing messages from the API.
2. A home PC and a remote server are significantly different.
Home PCs typically have very powerful and fast central processors, which is essential for parsing messages from the API.
For example, if you want to trade on both the FORTS and SPOT platforms on MOEX at the same time, it looks something like this on a home PC:
The average CPU load is around 4%. This is two scanners looking at 15-minute intervals across platforms. 900 sources.
On a remote server costing 2-3 thousand rubles, the load will be around 30%. That is, approximately 10 times higher.
3. If the CPU load is consistently over 50%, there may be queue buildup from the API.
When we suddenly want to trade scanners on the platform but don’t have enough capacity to parse the message queue, what happens is:
1. Messages from the API queue up, and parsing can't keep up.
2. The time and information that the robots see do not correspond to reality.
3. The delay accumulates. Initially, it's 10 seconds, after an hour it’s already 2 minutes, and after a day, it can be 15 minutes.
4. We place an order, and it appears in a different market depth than what the robots see.
5. The robot operator panics! OsEngine is not working!
Overall, one could even say YES, rather than no. In such conditions, the program stops functioning adequately.
So, pay attention to CPU load if you are trading a lot of securities!
4. How to open Task Manager and check CPU load?
1. Click the “Windows” button in the corner of the screen.
2. Find the Task Manager icon there. If it's not there, you can search for “Task Manager” next to the Windows button.
3. Here it is.
If the load is always above 50%, there may be issues with queue parsing.
If the load is around 100%, problems with queue parsing are guaranteed!
5. Ensure that market depths do not lag behind what the web terminal shows.
Next. If you see that the CPU is loaded, you need to:
1. Open the market depth in OsEngine;
2. Open the market depth in your broker's web terminal;
3. Visually check for discrepancies. If there is a visible difference, proceed to the next step.
6. Solution one: Optimize the robot's logic.
It’s quite possible that the problem is not with OsEngine at all, nor with the performance of the virtual server’s CPU, but rather with your robot “overworking.” For example, it might be monitoring every single candle change (or every market depth update from 10,000 per second) and causing 98% CPU load.
This can be addressed by changing the robot’s logic and moving simple filters closer to the logic entry point and only checking market depths for 1-2 instruments that are actually needed, instead of brute-forcing through all 1,000, for instance.
7. Solution two: Increase server capacity.
Yes. If nothing else helps, you'll need to go ahead and buy a more powerful server.
If you want to trade with 900 securities at once, the machine needs to be up to the task.
8. Strange cases can also occur. The CPU is not loaded, but there is a lag.
This is theoretically possible when you have a lot of very weak CPU cores.
For example, a user "accelerated" the parsing of messages from the Transaq Connector API because they needed to trade specifically on a ten-year-old server with weak processors. However, there were many processors available, which meant that in this case, additional multithreading helped them.
And so, happily, they trade their 400 sources.
Thus, if you have a non-standard machine, and if you start experiencing "wonders" with order execution, and if you are trading more than 50 instruments, pay attention to the order book…
Wishing you successful algorithms!