How to write a trading robot - straight to the point. Exchange software: Tools for creating trading robots

Sergey writes on 08/08/2019 00:05:
>Writing any programs to order. 2 Diplomas of MEPhI (Moscow Engineering Physics Institute) - mathematician and programmer. 25 years of professional programming experience. Expert of the highest class, worked for many years abroad in companies with a worldwide reputation. Languages ​​С, С++, C#, Java, JavaScript, HTML, Pascal, Delphi, Visual Basic, Excell, Assembler, Assembler for microcontrollers (AVR, PIC, x51, x80-86). Databases - any, including Oracle, MS SQL Server, InterBase, DB2, Paradox, Access, FoxPro any, client - server. ASSEMBLER microcontrollers (all dialects) - 8080, 8085, Z80, x51, AVR, PIC, Arduino. Computer graphics 2D, 3D - OpenGL, DirectX. Creation of sites of any level of complexity.

Greetings. I am interested in writing a trace program.
Bot program for automatic bets in a bookmaker's office
Not for betfair. The program should perform a series of actions, everything should look something like this:

1. We go into the application, press the button, the necessary rates (forecasts) are parsed from the postal address (every day a new letter arrives). Perhaps we display them in a separate window or form in the program.

2. According to the received bets (forecasts), the program automatically places these bets with a certain time interval on several accounts in the bookmaker.

3. Account in the bookmaker can be either one or several. Ideally, you can simply drive in accounts into the program and tick which account to use and which not.

4. The time between bets must be random, but not less and not more than the range that can be specified in the settings. For example: the interval is set from 1 to 5 seconds. 1 bet - boomz, the second after 2 seconds, the third after 3.6, the fourth after 1.7, etc.

5. The site of the bookmaker's office also needs to be parsed into the window with the program and select only the necessary line and the necessary events. I don’t know how to implement this, the programmer must come up with. For what? Sometimes a certain team name in the forecast may differ from the name in the bookmaker. Therefore, it will not be possible to automatically place a bet. We look further.

6. If, nevertheless, some bets could not be placed automatically, the program should allow the user to place them manually as quickly as possible. Those. for example, 8 out of 10 bets are placed successfully, then a window is knocked out which bets were not made - the user in the program already sees the parsed line with the necessary events and can bet manually with one click. Those. click once on the desired bet - and the bet will be made from all accounts.

7. The bet amount is set in the settings and is the same for all bets.

8. If during the operation of the bot the odds, handicap size (i.e. the bet itself) or the maximum bet amount has changed: sometimes, it can drop from 1.9 to 1.7) b) in case of a change in the size of the handicap, give the user a window with a new handicap, where the bet and the new handicap will be visible, you can click ACCEPT or CANCEL c) in case of a change in the maximum bet (may change from 50 to 0.01 for example) also accept a new amount if it is not less than N (it is desirable to put this in the settings).
Perhaps this is not all that is needed, but it is at least 90%. Please clarify any questions during development. Of course, it will not be such that the program will need to be changed upside down, but I think some improvements will be required. Ideally, this should be a three-click program: open the program, parse bids, place bids. I will personally give the BC website to the programmer, then I will show in what format the letters come to the soap and which line from the BC needs to be parsed. Develop the program so that you can make updates in the future, we will cooperate for a long time

One of the most common questions asked by people who are just starting to get interested in algorithmic trading is “What is the best programming language for this?”. Of course, the short answer is that there is no “best” option. When choosing a tool, you should take into account the parameters of the trading strategy, the required performance, modularity, development methodology, and requirements for fault tolerance. In this article, we will talk about the main components of the architecture of an algorithmic trading system and how each of them affects the choice of programming language.

Translator's note: We often encounter skepticism about algorithmic trading. There is an opinion that this is sheer speculation, which is extremely harmful and doing this for a technical specialist, to put it mildly, is not comme il faut. Anticipating some questions in the comments, we would like to immediately give a link to the material, in which a lot of attention is paid to describing what types of traders exist in the stock markets and why EACH of them has a certain benefit at a certain point in time, as well as to a topic in which touches on the more general topic of the purpose of the exchanges themselves. And here you can read about the experience of such trading, which allowed a person with knowledge of programming to earn half a million dollars (first part, second part .) Enjoy reading!

First of all, we will look at the main elements of an algorithmic trading system, such as analytical tools, a portfolio optimizer, a risk manager, and, in fact, a trading engine. Then we will touch on the features of various trading strategies and how the choice of any of them affects the development of the entire system. In particular, we will discuss the expected frequency (speed) and trading volume.

After you have chosen some kind of trading strategy, you need to design the architecture of the entire system. This includes the choice of hardware, operating system (or several systems), and resilience to rare but potentially catastrophic events. When designing the architecture, due attention should also be paid to performance - both the speed of the system's analytical tools and the trading engine itself.

What does a trading system do?

Before choosing the “best” programming language in which our million-earning robot will work, it is necessary to determine the requirements for this language. Will the system be purely task based or will we also need risk management or a portfolio builder module? Do I need a fast backtesting module to work? For most strategies, trading systems can be divided into two categories: exploratory and generating signals.

Research strategies focus on testing performance against historical data. Testing on data collected in the past is called backtesting. The processing power of the backtesting module is affected by the amount of data and the algorithmic complexity of the strategy. In the matter of optimizing the optimization of the speed of the research strategies, the speed of the processor and the number of its cores often act as limiting factors.

If we talk about generating trading signals, then here the algorithm must understand when to buy or sell and send the appropriate orders (most often through a brokerage system) to the market. Some strategies require a high level of performance. The speed of the strategy is limited by such factors as the width of the data channel and the delay introduced by the brokerage and exchange systems (latency).

Thus, depending on which category of strategy you need, and the choice of programming language for its implementation may vary.

Type, liquidity and volume of the strategy

The type of trading strategy will affect all its subsequent device. It is necessary to evaluate in which markets it is planned to trade, the possibility of connecting external data providers, as well as the frequency of operations performed by the algorithm, and their volume. Finding a balance between ease of development and performance optimization, as well as hardware, including servers that will need to be placed in brokerage or exchange data centers, and additional equipment that may be needed (GPU, FPGA, etc.) will also be important factors.

To trade low-liquid stocks in the US markets, you will need to use completely different technologies than in the case of a high-frequency statistical arbitrage strategy in the futures market. Before proceeding with the choice of the actual programming language, you should select the data providers with which your trading strategy will work.

It is necessary to analyze the existing connectivity to vendor systems, the structure of any APIs, the speed of data delivery, and the ability to store it in case of failures. It would also be a wise decision to organize access to several such systems at the same time, which will also require separate study, since each data provider has its own technological requirements (ticker symbols of exchange instruments, etc.).

The expected frequency of trading will have a determining impact on how your system's technology stack will be implemented. Strategies that need to update data more than once a minute will require more resources to run.

In the case of strategies that require tick data, it is necessary to develop the entire system according to the performance driven design methodology. HDF5 or kdb+ are often used for these tasks.

Both an optimized backtester and a trading engine must be used to handle the redundant volumes of data that are required for HFT applications. The main candidates for the role of a programming language in such a system will be C / C ++ (possibly, in some places Assembler). High-frequency strategies will often require additional equipment, such as programmable arrays (FPGA), as well as placing servers as close as possible to the exchange core and tuning the network interfaces of the servers themselves.

Research Systems

When creating systems of this kind, it is often necessary to resort to interactive development and script automation. The first concept takes place in an IDE like Visual Studio, MatLab, or R Studio. Scenario automation is associated with a large amount of calculations for different parameters and data points. Considering all this, it is necessary to choose a language that provides excellent opportunities for testing the code, and also allows you to achieve acceptable performance when calculating strategies for different parameters.

At this stage, IDEs such as Microsoft Visual C++/C# are often used, which include various tools for debugging, code completion and working with the entire project stack (ORM, LINQ); MatLab, which is designed specifically for problems of numerical linear algebra and vector operations; R Studio, which is implemented using the statistical language R; Eclipse IDE for Linux Java and C++ and semi-proprietary IDEs like Enthought Canopy for Python which includes a variety of data analysis libraries (NumPy, SciPy, scikit-learn and pandas).

For numerical backtesting, all the tools mentioned are suitable, although due to the fact that the code will run “in the background”, it is not at all necessary to use graphical IDEs. At this stage, first of all, you should think about the speed of the system. Compiled languages ​​(like C++) are useful when the number of backtesting options is very large. In such a case, you should always be very careful with each design step, as your system may not be as fast to begin with. In the case of interpreted languages ​​like Python, high performance libraries (NumPy/pandas) are often used for backtesting.

The choice of language for implementing the backtesting module will be determined by the specific needs of your algorithm and the number of libraries available for this language (more on this below). Keep in mind, however, that the language used for the backtester and research environment may differ from the tools chosen for the Portfolio Builder, Risk Management and Trading Engine modules.

Portfolio builder and risk management

Many algorithmic traders often underestimate the importance of a portfolio builder and risk management. This is a big mistake as these funds will allow you to keep your money on the exchange. With their help, you can not only reduce the number of risky transactions, but also minimize the cost of trading operations, reducing transaction costs.

Thoughtful implementations of these components can have a significant impact on the quality and consistency of profitability. Without them, it is very difficult to create a stable strategy, because the presence of a portfolio collection mechanism and a risk manager make it easy to modify the trading system.

The task of the Portfolio Builder module is to identify a set of potentially profitable trades and make those that will bring the greatest profit - for this, many factors are analyzed (for example, volatility, asset class and sector of the company whose shares are traded). In accordance with this, the available capital is distributed among various exchange instruments.

Portfolio construction is often reduced to a linear algebra problem (like matrix factorization), which means that the performance of the mechanism largely depends on the efficiency of the implementation of linear algebra tools in the system. Popular libraries include uBLAS, LAPACK and NAG for C++. MatLab has extensive capabilities in terms of operations with matrices. Python uses NumPy/SciPy for this kind of calculation. In order for the system to support a high-quality and balanced portfolio, you will need a compiled (and well-optimized) library for working with matrices.

Another extremely important part of any algorithmic trading system is the risk management module. Risk can take many forms: increased volatility (even desirable for some strategies), increased correlations between asset classes, server outages, so-called “black swans” (events that cannot be predicted) and undetected bugs in the trading program code - and this is only a small part of the possible problems.

The risk management module tries to "foresee" the consequences of most of the above risks. Very often, statistical analysis is used for this (for example, Monte Carlo stress tests). Parallelism plays a big role in such calculations, and, in general, performance problems can be solved by simply increasing computing power.

Trading engine

The task of the system's trading engine is to receive filtered trading signals from the portfolio constructor and risk management modules, generate trading orders based on them, which are then sent to the brokerage trading system. In the case of "regular" private traders, this will most likely require an API or FIX protocol connection. Accordingly, to choose a language, it is necessary to analyze the quality of the API itself, the presence / absence of software shells for working with it, the expected frequency of transactions and the expected "slip" between the moment the order is sent to the broker's system and its appearance in the core of the exchange trading system.

The "quality" of an API consists of several elements: the quality of the documentation, the performance that the interface provides, whether you need separate software to work with it, or you can establish a connection without a GUI, etc.

Most brokerage APIs have C++ and/or Java interfaces. Usually, a community of users-clients of the broker is formed around each such tool, who help develop it and create wrappers for C #, Python, R, Excel and MatLab. However, it must be remembered that any additional plugin may contain various errors, so they should always be thoroughly tested and make sure that the developers are engaged in supporting their creation. It's best to look at how often updates have been released in recent months.

The frequency of trading operations is the most important element of the trading engine algorithm. The robot can send hundreds of orders per minute, so system performance is critical. If the system is not implemented very well, there will inevitably be significant slippage between the price at which the order should have been placed and the price at which it was actually executed. This can have a dramatic effect on profitability.

Static-typed languages ​​(see below) like C++/Java are usually best for writing a trading engine, but their use raises development time, testability, and code maintainability issues. On the other hand, dynamically typed languages ​​such as Python and Perl are now "fast enough". Make sure that all components of your system are designed with a modular approach that makes it easy to remove and add new elements to the system over time.

Architecture planning and development process

We have already discussed the components of a trading system, the importance of the parameters of the frequency of trading operations and their volume, but so far have not touched on infrastructure issues. An independent private trader or an employee of a small HFT company or fund will likely face many challenges - alpha model analysis, risk management and execution parameters, and final deployment of the system - all to be done independently. These are all important points, so before diving headlong into the discussion of programming languages, it's a good idea to discuss the optimal system architecture.

Separation of interests

One of the most important tasks in creating a trading robot is the "separation of concerns" or, in the language of software development, the division of various aspects of the trading system into modular components.

This division into components will help in the future to change / replace / add new modules to the system to improve performance, reliability or facilitate its maintenance, without having to check all the dependencies and “if anything is broken” in other places. For trading systems, this approach is the best practice. For systems that operate "at medium speeds" its implementation is highly desirable. In the case of HFT systems, some rules may need to be ignored in order to achieve even faster performance, but in general, it is worth sticking with this approach.

Creating a component map of an algorithmic trading system is a topic that deserves a separate article. However, the optimal approach here is to implement separate components for historical and real market information, data storage, access API, backtesting module, strategy parameters, portfolio builder, as well as the risk management module and the trading engine itself.

For example, if problems with the efficiency of working with the data warehouse are found (even after optimization work), then such a module can be easily replaced with almost no need to rewrite anything in the data ingestion or API access components.

Another plus of the modular scheme is that it allows you to use different programming languages ​​in different parts of the system. There is no need for a hard binding to a specific tool if the method of communication of the system components is independent. For example, they can communicate over TCP/IP, ZeroMQ, or other protocols.

For a specific example, the backtesting system can be written in C++ for high performance, while the portfolio manager and trading engine can be written in Python using SciPy and IBPy.

Thoughts on performance

Performance is important for almost any trading strategy. The higher the frequency of the trading system, the more important this factor is. “Performance” refers to many things, including algorithm execution speed, network latency, communication channel, data I/O, multithreading/parallelism, and scaling. Separate books are devoted to each of these aspects, so we will only briefly touch on them. We will now discuss architecture and specific programming languages ​​in terms of their impact on overall system performance.

Donal Knuth, one of the fathers of what we call Computer Science, said a very wise thing: "the root of all evil is premature optimization." This is true almost always, but not in the case of developing an HFT trading algorithm! If you are interested in creating a less high-frequency strategy, then the general approach in your case would be to build the system in the simplest way and start optimizing it only when bottlenecks are found.

To identify them, various profiling tools are used. You can create profiles both in MS Windows and Linux environments. To do this, there are a whole bunch of different means. Now, as agreed, we will discuss specific programming languages ​​in the context of performance.

C++, Java, Python, R, and MatLab each have high-performance libraries (both internal and external) for basic datasets and algorithmic work. C++ comes bundled with the Standard Template Librar and Python comes with NumPy/SciPy. You can find standard math problems in these libraries, and writing your own implementation is a path that is rarely ever beneficial.

An exception is when you need unique hardware and you use an algorithm that works with some proprietary extensions (like custom caches). At the same time, it must be remembered that the invention of the wheel often takes time, which can be spent with much greater benefit on the development and optimization of all parts of the trading system. Development time is priceless, especially if you're building your own system.

For a trading engine, there is often a problem with delays because the market analysis tools are usually located on the same machine. Delays can occur at any step of the execution process: databases are accessed (disk / network delays), trading signals need to be generated (OS or kernel delays), orders must be sent to the exchange (communication channels delay) and they must be processed by the core of the exchange's trading system ( exchange delays).

To create an efficient HFT system, you will have to understand the optimization at the kernel level and the optimization of data transfer processes.

Another useful tool for a developer of high-speed exchange robots is caching. The main idea of ​​caching is to store frequently requested information in such a way that it can be obtained without wasting resources. In web development, for example, caching can be used when loading data from a relational database on disk into memory. All subsequent requests for this data will no longer need to be sent to the database, which can significantly improve system performance.

For online trading, caching can also be a very useful thing. For example, you can save the current state of the portfolio to the cache and keep it there until the instruments in it are “rebalanced”, which will avoid the need to generate a list of bought and sold assets anew each time the algorithm is triggered - it can simply be updated. Such an operation requires significant CPU and I/O resources.

Unfortunately, caching is not without its problems. Reloading data in the cache, due to the volatile nature of cache stores, can also require significant infrastructure resources. Another problem is the domino effect, where high load causes multiple copies of the cache to be generated by mistake, resulting in a series of crashes.

Dynamic memory allocation is an expensive operation. Therefore, high-performance trading applications should be able to work well with memory and be able to allocate and take it at all stages of the program flow. Newer programming languages ​​such as Java, C#, or Pythong have automatic garbage collection, where memory is allocated or deallocated dynamically.

This tool is extremely useful in development because it reduces the number of errors and improves the readability of the code. However, for some HFT systems it is still better not to use standard memory management tools, but to implement your own. For example, in Java, some tweaking of the garbage collector and heap configuration can improve the performance of HFT strategies.

There are no native garbage collectors in C++, so it is necessary to manage the allocation and release of memory during the implementation of objects. This, of course, makes errors more likely to occur, but it also allows for better control of objects and heaps in specific applications. When choosing a programming language, take the trouble to learn more about how garbage collection works in it, and whether it is possible to somehow optimize the operation of this mechanism for specific scenarios.

Many operations in algorithmic trading can be parallelized, that is, make it so that different program operations are performed simultaneously. The so-called "stunningly parallel" algorithms include steps that can be performed completely independently of the other steps. Specific statistical operations, such as Monte Carlo simulations, are good examples of such parallelized algorithms, since each probability and the course of events when it occurs can be calculated without knowing other possible paths of the situation.

Other algorithms are only partially amenable to parallelization. Algorithms of this type include modeling in hydrodynamics, where the computational area can be divided into separate domains, but they still need to be connected to each other. Parallelizable algorithms obey Amdahl's law, which introduces a theoretical upper limit to the performance improvement of a parallelized algorithm in the case of N individual processes (for example, on a processor core or in a thread).

Parallelization has become an important optimization element as processor clock speeds have not increased recently and new processors contain more and more cores that can perform parallel computing. The development of graphics hardware (particularly for video games) has led to improvements in GPUs, which now contain hundreds of "cores" to perform many simultaneous operations. And the price of such GPUs has become much more affordable. High-level frameworks like Nvidia's CUDA have become widespread in science and finance.

Typically, such GPU devices are only suitable for research tasks, but there are also those (including FPGAs) that are used directly for HFT. At the moment, the vast majority of modern programming languages ​​support multithreading to some extent, which will allow you, for example, to optimize the backtester so that it uses processes that are independent of each other.

Scaling in software development refers to the ability of a system to handle ever-increasing workloads in the form of more requests, more CPU usage, and more memory allocations. In algorithmic trading, the strategy “scales”, meaning it can work with a large amount of capital and still consistently produce a positive result. A trading technology stack is scalable if it can handle large volumes and handle increased latency without bottlenecking.

Of course, systems must be designed to be scalable, but problems and bottlenecks can be hard to predict. Strict logging, profiling and monitoring will make the system more scalable. Some programming languages ​​are often described as "not scalable". In fact, those who say so simply "do not know how to cook them." The whole stack of technologies can be non-scalable, but not the language itself. Naturally, some languages ​​have better performance than others in specific cases, but one cannot say that one language is "better" than another in every sense.

As we said above, you need to share interests. In order for the system to cope with "spike" (the so-called sudden volatility that causes a large number of transactions), it is useful to create a "message queue architecture". This means that there is a message queue between the components of the trading system, so that the system hangs if a certain component cannot process many requests.

Hardware and operating systems

The hardware your trading system runs on can have a significant impact on the profitability of the algorithm. This does not even apply to exclusively high-frequency traders - bad servers can fail at any moment, and it will not matter how many transactions your robot makes if it cannot, due to the fault of the iron, perform one, but very important operation. Therefore, the choice of hardware for a trading system is extremely important. Usually, the choice is between the user's own computer, a remote server, a cloud virtual machine, or a server at a collocation (in the data center of an exchange or broker).

It is clear that the desktop option is the simplest and cheapest, largely due to the existence of a large number of user friendly operating systems (Windows, Mac OS, Ubuntu). But they also have significant disadvantages. One of the main ones is that each new OS upgrade will require patching of the trading robot, plus the computer will have to be rebooted from time to time, which is also not very good. In addition, the computing resources of a personal machine are also spent on maintaining the GUI, but they could be spent on increasing the performance of the trading system!

In addition, working at home or in the office is fraught with problems with uptime and Internet connection. The main advantage of a desktop system is the fact that additional computing power for it can be purchased for an amount much less than what is required to upgrade a server of the same speed.

A dedicated server or a cloud machine will cost you more than a desktop computer, but it will also allow you to organize a much more redundant infrastructure - including automated data backups, the ability to configure uptime and speed monitoring. They will be more difficult to administer, since at least this will require a remote connection.

For Windows servers, RDP will most likely be used, and in Unix-based systems, SSH cannot be dispensed with - there is no way to get away from the command line, which makes some development tools like Excel or MatLab inapplicable due to their impossibility to work without a graphical interface.

A server on a collocation simply means that you put your server as close as possible to the core of the exchange - in its data center, or in the broker's data center, which is located with the exchange system on the same local network. For some HFT strategies, this is the only viable option, despite being the most expensive.

The final aspect to consider when choosing software and programming language is platform independence. Is there a need to execute code on different operating systems? Or is the code designed to run on a specific processor architecture - eg x86/x64 - or will it be able to run on ARM's RISC processors as well? The answers to these questions will directly depend on the expected frequency and type of trading strategy.

Sustainability and Testing

The best way to lose a lot of money on algorithmic trading is to create an unsustainable system. Resilience includes the system's ability to respond to rare events such as broker outages (or bankruptcy), unexpected high volatility, Internet service provider (internet, data center) outages, or accidental deletion of the entire trading database. Poorly implemented architecture can undo years of successful and profitable trading in a couple of seconds. The key points of your trading system should be debugging, testing, backup, availability and monitoring.

When developing a high-quality trading system, do not even expect to spend less than 50% of the time on debugging, testing and support.

Almost all programming languages ​​come bundled with a debugger or have adequate third-party alternatives. With the debugger, you can place special breakpoints in your code that allow you to examine the behavior of the program before it crashes.

Debugging is an important tool in bug analysis, but it's mostly used in compiled languages ​​like C++ or Java, while interpreted languages ​​like Python are easier to debug. However, this language also comes with pdb, a powerful debugging tool. The Microsoft Visual C++ IDE has additional GUI debugging tools, while for Linux C++ you will need to use the gdb debugger.

Not without testing. The most modern testing paradigm is TTD or Test Driven Development, in which a test is first written that covers the desired change in the system, and then code is written under it that can pass this test.

Test-driven development is not an easy task that requires a lot of discipline. For C ++, there is a unit testing framework in Boost, in Java there is a JUnit library for the same purposes. Python also has a module for this kind of testing, which is part of the standard library. Many other languages ​​also have tools and frameworks for doing unit testing.

In a productive environment, logging is absolutely necessary and thoughtful. It is necessary to establish a process for issuing a variety of error messages and system behavior. Logs are the first thing you will start with when dealing with problems and failures. Despite the external simplicity of the task - outputting messages to a file and storing it - in fact, everything is much more complicated, and you should think over the scheme of the logging system before proceeding with its implementation.

Both Windows and Linux have a variety of logging tools and features, and programming languages ​​also come with logging libraries that will work for most situations. A reasonable solution would be to centralize all reporting information - so it will be more convenient to analyze it in the future.

The logs will give you an idea of ​​what happened in the past, and the monitoring system will provide an understanding of the current situation. Almost every aspect of your trading system can and should be monitored: disk space usage, available memory, link states, and CPU usage are all useful data for a basic understanding of the state of affairs.

In addition, it is worth monitoring purely trading metrics - abnormal volumes or prices, sudden account drawdowns and news affecting certain sectors of the economy or entire countries. The monitor should come with a module that will notify you if any of the parameters is violated. You can use a different message delivery method (email, SMS, a call from a robot to a phone) depending on the severity of a particular event.

Usually, a dedicated administrator is responsible for monitoring the system, but if you do everything yourself, you will have to resort to using various tools that will facilitate development, since there are many both paid and free, open solutions for a variety of cases.

System backup and availability is the first thing to work on. Think about the following questions: 1) if for some (terrible) reason the entire database is suddenly deleted (and there are no backups), then how will this affect the algorithm for researching and executing orders? 2) If the trading system hangs for a long period of time (if there are open positions), how will this affect the amount of money in the account and portfolio? The answers to these questions are usually frightening.

Therefore, it is imperative to develop a system for backup and further deployment of data - this is almost more important than the copy itself. Many traders do not test the saved backups, which entails no guarantees that at the right time this data will be able to “roll up” and the system will work as expected.

The same applies to work on the accessibility of the system. Despite the additional costs, be sure to take care of the availability of redundant infrastructure and redundancy - the cost of system downtime can exceed all costs by dozens of times in a few minutes.

Language selection

We have already considered many factors and aspects that affect the development of a trading system. It's time to talk about programming languages.

Type system

When choosing a programming language for the trading stack, one should not forget about the type system. The languages ​​of interest to algorithmic traders can be either dynamic or static. The latter include C++ and Java - they have type checking performed during the compilation process. In dynamic languages, this check happens on the fly without any compilation. These are, for example, Python, Perl and JavaScript

For highly accurate systems, which certainly include trading robots, compile-time type checking can be a very beneficial option, as it eliminates a large number of errors that could lead to numerical errors. On the other hand, type checking does not catch all possible bugs, so you need to spend time and handling exceptions. When using dynamic languages, startup errors often occur that would not occur with type checking in static languages. If you are still using a dynamic language, then it is worth implementing the TDD methodology and unit testing to reduce the number of possible errors.

Open source or proprietary software?

One of the main decisions that a developer of algorithmic trading software will have to make is whether to use commercial software or resort to open technologies. Each of these paths has its pros and cons. It is necessary to study how well the language is supported, how active the community is developing it, how easy it is to install and maintain, how good the documentation is on the web, and calculate any possible costs for licenses and use of the product.

The Microsoft .NET stack (including Visual C++, Visual C#) and MathWorks from MatLab are the main proprietary tools for developing trading systems. Both systems have been tested by thousands of traders around the world on various exchanges.

These software products are well-documented and have a large active community interested in the development of these tools. The .NET software allows integration with many programming languages ​​like C++, C#, and VB, as well as connecting other Microsoft products like the SQL Server database (via LINQ) quite easily. MatLab also has many plugins and libraries (some of them are paid) that can be applied to almost any area of ​​financial computing.

But there are also limitations. The main one is the price, which can be unbearable for a single trader (although Microsoft gives the basic version of Visual Studio for free). Microsoft products work well together, but integrating them with any third-party systems is not at all easy. Also, Visual Studio runs only on WIndows, which can be blamed for slower performance than a similarly powerful, well-configured Linux server. MatLab is missing some plugins that could make using this product easier.

The main disadvantage of proprietary products is the lack of access to the source code. This means that when you need ultra and mega performance, you will not be able to tweak anything in them, and therefore, you will be limited.

Open source products are also very common in the financial industry. For example Linux, MySQL/PostgreSQL, Python, R, C++ and Java are used where high performance is needed. However, none of these tools can be called "sharpened" for this particular market. Python and R contain a lot of redundant libraries with which you can perform almost any possible calculation at speeds comparable to compiled languages ​​(with certain caveats, of course).

The main benefit of using interpreted languages ​​is development speed. Python and R require far fewer lines of code to achieve similar functionality. In addition, they also often allow interactive console based development, which greatly speeds up the process of sequential development.

Considering the fact that a developer's time (especially if he is a loner) is very valuable, and the speed of everything and everyone in the HFT business is always in the first place, it is worth taking a closer look at the open technology stack. The same Python and R have an impressive community, and, due to their popularity, are well supported. In addition, there is a huge amount of documentation on them.

However, open source software often lacks commercial support, as in the case of proprietary products, and they run on much less user-friendly interfaces. On a Linux server, you will almost never see a graphical management interface, everything will have to be done through the console. And for some tasks, languages ​​like Python and R can be too slow. There are mechanisms for integration with, for example, C++ to improve speed, but this requires some experience in multilingual programming.

Even though in the world of proprietary software you can encounter problems when updating product versions, in the case of open source software, such difficulties are much more common. In general, open systems are more difficult to administer.

What's in the box

What libraries does the language contain and how good are they? This is where older languages ​​have an advantage over newer ones. C++, Java, and Python have been in development for a long time and have an extensive set of libraries for network programming, HTTP, interaction with the operating system, graphical interfaces, libraries for regular expressions, and so on.

C++ is famous for its STL (Standard Template Library), which contains many high-performance data structures. Python is known for its ability to work with almost any type of system and protocol (especially on the web) through its own standard library. R has a lot of built-in statistical and econometric tools, and MatLab is great for writing linear algebra code (this can be found, for example, in portfolio optimization engines and pricing

And technologies related to this area, but we have never talked about software with which, in fact, you can create your own trading program. Under the cut - an overview of common software tools for creating mechanical trading systems adapted to the Russian stock market.

Wealth Lab

The product of Fidelity International is one of the most powerful tools for technical analysis, development and testing of trading strategies. The built-in programming language in it is WealthScript, which has a lot in common with Pascal, the latest versions use C # and other .NET languages.

On the Russian stock market, it is used in conjunction with brokerage terminals - in Wealth-Lab, the user describes his strategy, according to which the program generates orders for transactions. With the help of special libraries for integration, these orders are then transferred to the trading terminal, from which they are executed. Objectively, such a scheme imposes quite a few restrictions, so Wealth-Lab, of course, cannot be called an ideal option for Russian exchanges.

MetaStock

Another foreign product. MetaStock contains a large library of various indicators and tools for creating your own formulas. Of the benefits - a fairly simple built-in programming language. With the help of additional modules, you can generate buy/sell orders. Like Wealth-Lab, it is used in the Russian market in conjunction with trading terminals with the help of additional libraries, which entails approximately the same problems. Also, the disadvantages include the fact that the simplicity of the built-in programming language does not allow describing complex trading strategies.

Omega Research

Technical analysis tool for creating and testing mechanical trading systems. You can write robots in the built-in Easy Language programming language (syntax is similar to Pascal). As in the case of the above two programs, it is used on the Russian stock market with the help of "gaskets". Among the minuses, respectively, is the stability of the operation of such a design, as well as the complexity of setting up Omega Research. In addition, the program works only with its own data format and does not support conversion from text files or formats of other technical analysis programs.

In addition to foreign products, there are a number of software solutions from Russian developers on the domestic stock market. And here are just a few of them.

TSLab

Like the previous project, TSLab is developing a platform for creating and launching mechanical trading systems, "sharpened" specifically for the Russian stock market. One of the essential for traders who do not have programming skills is the ability to record a trading algorithm in the form of a flowchart.

stocksharp

Free (in the basic version) StockSharp open source platform and products based on it (S#. Studio). As the name implies, you can program in C#. Of the benefits - the ability to connect to various trading terminals and brokerage systems.

livetrade

The product line of the St. Petersburg company Cofite. Thanks to the API, using the LiveTrade Terminal trading terminal, you can run robots implemented on the .NET platform. It is possible to connect to trading terminals and systems of several Russian brokers (including the ITinvest system using the SmartCOM API). In addition, Cofite has a Robotlab product that is entirely dedicated to building robots. As in the case of TSLab, trading algorithms can be implemented using a visual constructor. The resulting robot block diagram can then be run in the terminal.

SmartX

The SmartX trading terminal is not an ordinary terminal in the usual sense, but an integrated software product, which, among other things, includes the built-in scripting programming language TradeScript - a vector language that was created by the American company Modulus Financial Engineering (USA) specifically for creating trading robots.

Among the interesting functions of the terminal are:

• The ability to back-test trading strategies - testing the robot on historical data. At the same time, these archived data do not need to be loaded from other (often paid) sources - they are loaded automatically by the terminal.
• Ability to build algorithms based on tick data.
• Another interesting feature is the ability to test a trading strategy "on the fly" using current exchange data, but without placing an order on the exchange itself - the time of a virtual transaction, the price and the resulting profitability will be displayed in a separate window.

The scripting language is quite easy to learn, and you can start programming simple robots within a couple of hours after getting to know it. In addition, many algorithms are similar in writing to Metastock, so if the user was previously familiar with this program, then he will practically not have to relearn.

An example of a simple TradeScript robot:

Buy Signals # Buy if volatility is down and volume and price are trending up HistoricalVolatility(CLOSE, 15, 252, 2)< REF(HistoricalVolatility(CLOSE, 15, 365, 2), 15) AND TREND(VOLUME, 5) = UP AND TREND(CLOSE, 40) = UP Sell Signals # Продаем, если волатильность снижается, а объем и цена имеют нисходящий тренд HistoricalVolatility(CLOSE, 15, 252, 2) < REF(HistoricalVolatility(CLOSE, 15, 365, 2), 15) AND TREND(VOLUME, 5) = UP AND TREND(CLOSE, 40) = DOWN
The advantage of this method of creating robots is that, unlike the same Wealth-Lab and Metastock, there is no need to create complex structures and use "pads" in the form of additional libraries to transfer orders to the terminal - everything is built-in and immediately connected to the brokerage trading system.

In addition, the user can run as many simultaneously running algorithms as the processor clock speed and computer memory allow. Considering the large number of words and operands of the scripting language, this means the ability to create any number of complex trading strategies.

smartcom

The API of our brokerage system (more details in), with which you can create trading robots of any complexity. There is an additional plugin for AmiBroker, which greatly facilitates data analysis.

That's all for today. In the following topics, we will talk in more detail about writing trading robots and give examples of specific mechanical systems created using various tools. Thank you for your attention!

In this article, I will tell you in detail and show you how to write a trading robot, which was discussed in the previous article. This will be the first and easiest lesson on creating automatic exchange systems. But, despite this, having done exactly all the actions described by me, the result will be a real trading robot.

I will not pour water, but I will go straight to the point. So, first of all, you need to download and install Delphi 7. You can do this from the link - . After that, open the folder with the shell program. The program can be viewed and ordered at the following link:

We are interested in the TradeRobot.dpr file. Open it and see what is shown in the picture below.

The window that says "Robot" is Form1, a Windows window to put it simply. This is how it looks in the developer. On it we need to place the controls and controls of the robot, such as: two buttons and three small row windows. The first button will be responsible for connecting with QUIK, the second for starting and stopping our robot. The lines of the windows will display information about the volumes of last buys and sells and the current status of the program.

To place the necessary elements on the form, you need to click on the button in the Standard tab.

Then click anywhere on the form. Delphi will offer to rename the button and we will use it. Let's call it ConnectButton.

In the same way, let's create a second button and name it StartButton.

Now let's create 3 row windows. To do this, in the same Standard tab, select the appropriate icon (as in the picture) and place it on the form.

So we repeat three times and call them as follows:
- top left BuyVolumeEdit
- top right SaleVolumeEdit
- bottom StatusEdit

Now you need to change the labels on the elements. This is done in the ObjectInspector window, for buttons in the Caption line, for window-lines in the Text line.

It should turn out like the picture below.

Now we need to write a script for each button. To do this, double-click on the Connect to QUIK button and the code will open, as in the picture below.

Insert the following text before begin:

Var EMsg: Array of char; EMsgSz:DWord; ExtEC,rez:LongInt; path:string; dOrderNum:double; massage:string;

Insert the following between begin and end:

EMsgSz:=255; ExtEC:=0; rez:=-1; path:= "C:\Quik-Junior-Zerich\"; FillChar(EMsg, SizeOf(EMsg), 0); rez:=connect(PChar(path), ExtEC, EMsg, EMsgSz); Massage:= ResultToString(rez and 255); if Massage = "EXECUTE" then begin StatusEdit.Text:= "Connected..."; Status:= "Connected..."; end else begin StatusEdit.Text:= "Failed..."; Status:= "Failed..."; end;

As a result, you should get the following:

The red underline marks the path to QUIK. If you have a different one, then change the path in this place.

Now let's move on to the START button. Double click on the script and paste the following between begin and end:

If StartButton.Caption = "START" then begin Status:= "Started"; StartButton.Caption:= "STOP"; end else begin Status:= "Stopped"; StartButton.Caption:= "START"; end; StatusEdit.Text:= Status;

It should turn out like this:

Now let's write global variables. To do this, let's find a piece of code:

And paste the following code:

DataTable: array of array of string; BuyVolume, SaleVolume: Real; Status: String;

Now in the ObjectInspector we select Form1 and in the Events tab we find the onshow line. Double click to open the code. Insert the following between begin and end:

SetLength(DataTable,2000000,10);

Now we need to find a place in the code, as in the picture below:

After var i, j: integer; paste the following:

path,transaction:string; dOrderNum:double;

After begin is the following:
StatusEdit.Text:= Status;

After If(Data.Cells<>"0") and (Data.Cells<>"0,00") then begin paste the code:

DataTable:= Data.Cells; if DataTable<>"" then begin BuyVolume:= StrToFloat(DataTable); SaleVolume:= StrToFloat(DataTable); If StartButton.Caption = "STOP" then begin if (BuyVolume > 10) and (Status<>"Purchase") then begin transaction:="ACTION=NEW_ORDER; TRANS_ID=1; CLASSCODE= SPBFUT; SECCODE= SiU6; ACCOUNT= SPBFUT00553; CLIENT_CODE= ; TYPE=M; OPERATION= B; QUANTITY=1; PRICE=0;" ; Status:= "Purchasing"; dOrderNum:=0; send_async_transaction_test(transaction, dOrderNum); end; if (SaleVolume > 10) and (Status<>"Sale") then begin transaction:="ACTION=NEW_ORDER; TRANS_ID=1; CLASSCODE= SPBFUT; SECCODE= SiU6; ACCOUNT= SPBFUT00553; CLIENT_CODE= ; TYPE=M; OPERATION= S; QUANTITY=1; PRICE=0;" ; Status:= "Sale"; dOrderNum:=0; send_async_transaction_test(transaction, dOrderNum); end; end; end; BuyVolumeEdit.Text:= DataTable; SaleVolumeEdit.Text:= DataTable;

It should turn out like this:

ATTENTION!

In lines like (there are two of them):

Transaction:="ACTION=NEW_ORDER; TRANS_ID=1; CLASSCODE= SPBFUT; SECCODE= SiU6; ACCOUNT= SPBFUT00553; CLIENT_CODE= ; TYPE=M; OPERATION= B; QUANTITY=1; PRICE=0;";

in the ACCOUNT= parameter, you need to specify your account. Now my SPBFUT00553 is listed there.

Everything, the robot is written. Now it remains to compile it, i.e. turn into a finished EXE application. To do this, click on the green Run (or F9) button, similar to Play.

The trader's brain functions in an unusual way. A person who is far from financial markets acts rationally and predictably. When an individual who thinks exclusively in candlestick combinations comes into play, the consequences are much more difficult to predict. As a private matter, knocked down life priorities and increased impulsiveness are well-known vices of traders. The very process of trading becomes an end in itself, and the biological computer in the bowels of the skull enters an endless cycle, hanging on the abstract idea of ​​​​an idyllic existence and its own yacht, moored at the pier of the mansion's embankment.

Let's start with the fact that the mere idea of ​​automating your own trading strategy does not provide either a carefree existence or getting rid of the trader's pressing problems. If you have no idea how the market works, investing time and money into building a robot will not fill your bank account with gold galleons, but rather return you to the brutal reality of the typical middle class.

So time wasted is experience. Time is money. Roughly speaking, spending your time, you buy experience. But you, as a reasonable person, will probably want to save an extra penny, not making mistakes where you can get around the rake, especially if there is such an opportunity. At this point, I will have to ask for a little of your time in order to tell you how to actually prepare for the question of creating money printing machines trading robot, and, of course, how not to make a lot of mistakes.

1. We start developing a forex robot with an idea

Developing a robot for trading in the markets is a continuous search for ideas and, in particular, the search for the Holy Grail. Contrary to the claims of unenlightened citizens, the Grail is found quite simply - let's say "thank you" to technological progress. Try to google "Holy Grail Forex" - most likely, your dream adviser is on the first page of the search results, and it is absolutely free.

Actually, this story could have ended. If the perfect Expert Advisor already exists, why keep looking? But then the second question arises: why, if the ideal adviser has already been found, is there still a shortage of drinking water in Africa? If the answer is obvious to you, congratulations, you have passed the first stage of becoming a reasonable trader, having overcome the level of a beginner or an algo-fanatic who does not see the prospects for his development, and is focused only on the result.

I am sure that the idea of ​​your Expert Advisor is absolutely unique and has no analogues. But, in any case, it is foolish not to try to find similar developments by other authors - the scale of the research done is difficult to assess until you come across it face to face. Humanity is moving towards a technological singularity, and new developments are being introduced so quickly that it is often not possible to comprehend all the material laid out. If there are still no such developments, and it is impossible to test the idea without conducting your own research, it's time to move on to the next stage.

2. 4 important questions before development

But before that, try to answer the following questions for yourself:

• What is your trading idea?
• What tasks will the adviser solve, and what objective advantage can this give in trading?
• Perhaps you need a graphical indicator or a helper script?
• How feasible is the idea technically, and can you independently assess its complexity?

These are very important questions that will not only save you a lot of time, but will also bring you a little closer to understanding your own idea. Surely, you did not come to the market to distribute money to those in need, but you have other goals.

Ideally, you should thoroughly know and understand how the trading platform you have chosen functions and the strategy on which, in fact, you will earn. Any vague definition or omission of seemingly unimportant moments will turn out badly at the next stages of development, and will hurt the trading account. Do not be afraid to resurrect long-forgotten trading ideas, but do not try to play on some ready-made solutions. The market is very volatile and, by definition, is in constant motion, and competition is constantly tightening.

Beginners in the foreign exchange market often have a lot of questions, the answers to which are not entirely obvious, or are just a matter of inexperience thereof. As a novice trader, it is enough to read one book by Bill Williams to replace the concept of trading chaos with a trading order for life. Then such people wonder why not to buy here, but not to sell here, and order an adviser from the programmers for the intersection of two moving averages. But at the initial stages of training, this can be completely left to the mercy of ready-made development tools. One of these is the MQL5 Wizard, a wizard for creating Expert Advisors for MetaTrader 5.

For the implementation of simple things, the Master is ideal. You do not need to have any magical abilities - programming skills are not required here. A few simple steps, and the adviser is ready. A ready-made advisor can be tested in the same MetaTrader terminal. Keep a harvester for the production of simple grails.

So, for starters, launch MetaTrader 5 and go to the application editor (F4 button). On the top menu, click the "Create" button and in the dialog that appears, select the generation of a new adviser.

Adding trading signal modules. Modules can be both standard and developed by the community (you can also participate in this process).

Actually, all standard signals are well described in the online help on the official website. You can also download custom ones there.

Having chosen a suitable indicator, you need to specify its input parameters. Here you can also indicate the weight of the signal in relation to others, if there are many of them.

A regular trailing stop or closing a deal based on the indicator values ​​is your choice.

There is also a choice of a fixed lot, or the volume of the transaction as a percentage of the deposit. For the most risky newcomers, there is also one with its geometric progression of the lot, which can bring you millions for a couple of extra passes in the strategy tester.

Actually, even if you are not a member of the Metatrader clan, do not worry, the constructor for the trading platform you need has already been written. The point, again, is in the possibilities and desire. The moral is simple: do not try to implement what has already been implemented a long time ago. Ignoring the work of others can be costly, it is not at all necessary to invent science anew.

So, you already have some trading experience, you have tested your idea in trading or have an accurate idea of ​​how it will work in the real market. So, it's time to move on to writing the formal rules of your strategy. Often at this step, problems begin for most followers of the humanities - people are faced with an insurmountable wall of misunderstanding, when an absolutely perfectly worked out scheme in their head stops working when translated into a technical language.

This problem is so widespread that it has created an entire industry for translating abstract ideas into the formal language of clearly structured rules, naturally, not for free. In fact, with minimal preparation, you will not encounter such complexity simply because you understand the mechanism of the gears of your Forex robot to the smallest detail, but development will still require some work.

In fact, a correctly drafted task is already half the work. However, this is not the programmer's job. If you ask a programmer to write a task for you, be prepared to run into misunderstanding at best and resentment in the vast majority of situations. It is the developer of the idea that is responsible for its exact formalization, and not the implementer in any way - if you like, the translator into the language of low-level commands.

An example of a bad task for creating a trading advisor is its absence. The very intention of the customer can be understandable: there is an indicator, you need to write a trading robot, but then what?

"Try to delve into the process of work and everything that directly concerns the implementation of the idea of ​​your adviser"

How to close a deal? How to accompany her? What volumes to choose? What to do with ? You need to understand that the programmer does not have psychic abilities. If you find one, be sure to let me know, such people should not disappear.

Seeing the terms of reference for the creation of a work without the necessary details, the freelancer will make his own interpretation and, in some cases, can really guess the real intentions of the customer. But, as a rule, such appeals end with two offended parties and an appeal to arbitration with further termination of the contract.

Morality: the more details you provide in the assignment, the easier it will be to understand, comprehend and, in the end, implement.

7. Choice of artist

If you have no experience with freelance exchanges, try to imagine a sub-equatorial version of hell. In fact, not everything is so scary, but some nuances are still worth considering.

Firstly, the pricing policy when writing an Expert Advisor can be extremely uncertain. There is a certain minimum that an adequate programmer will not undertake, but at the same time, the value of this minimum floats a lot and at times the minimum becomes the maximum (yes, this also happens).

You should not be too generous either, this immediately suggests that a person has money and you will definitely be torn off three skins for some simple task. If you don't understand the cost of a task, try getting it from people you trust. Communicate with experienced developers, even if you are not going to order anything from them. In the future, this will help to identify who you can contact and who you should not contact.

Secondly, do not be afraid to ask the developers of your Expert Advisor what points are incomprehensible to him, which require clarification, and what, in fact, is the complexity of one or another part of the robot. It would seem that everything in the terms of reference is clear to you, and yet the performers often come across a fundamental misunderstanding.

“It is in your interest to control the process of creating your dream Expert Advisor. Ask for milestones, participate in the beta test, and don't hesitate to get technical details."

One typical example is a misunderstanding of file formats and their meaning. Like: “I have a .dll library here (.exe file or .ex4 program), could you quickly fix this and that, and I’ll throw you a pretty penny for it? Not? And why? Vasily made me an adviser for 5 bucks, and you are a bad developer!

Morality: try to delve into the process of work and everything that directly concerns the implementation of your idea. This will not only help you better understand the work of the robot yourself, but will also further simplify the process of communication with the performers. There are always at least two parties involved in the work, and if you intend to make trading your main profession, patience must be preserved - both yours and those around you.

The handover stage is the most difficult. Rest assured, each performer drinks a soothing broth out of grief, marking with a black marker the day of the calendar, when he is once again asked to correct an order that has already exhausted all deadlines. Like the universe, there is no limit to perfection, but at a certain moment you need to get together and say - the task is completed.

I understand that for a trader, the light reflected from the golden mountains illuminates the creation of advanced technical thought and the fruit of inner genius in its immensely beautiful presentation. How so, because an adviser that does not earn money simply does not fit in the head, and creates an image of work in progress. How can you pay for something that doesn't work?

On the part of the performer, the work is hung up exactly when all the points of the order are completed. The terms of reference for a programmer is a clearly drawn path, to deviate from which means to think for the customer. Sometimes this path leads to a dead end, like the road from the former capital of Russia to liberal Europe. That is, at the stage of accepting work, it turns out that the task, it turns out, was not at all, but in a radically opposite way.

If the task is small, the performer may well make concessions and correct the work in accordance with the new TOR. If the work has been done on a large scale, it is too late to change anything. A common problem is total detachment during the development process, for example, refusal to participate in beta testing: “Why did you send me an Expert Advisor with errors, what else should I test for you?” or: “How do I know how to run this tester of yours, am I a programmer or something?”.

For an adequate person, such statements cut the ear, while the performer falls into depression. Development of a trading robot is, first of all, work. If you are looking for a ready-made solution, there is a shop of ready-made Expert Advisors, where the Expert Advisor will be tested and polished for you, documenting its historical success along the way. In this case, contacting a freelance exchange does not make sense.

Morality: Do not give everything to chance. It is in your interest to control the process of creating your dream Expert Advisor. Ask for milestones, participate in the beta test, and don't hesitate to get technical details.

Conclusion

Great figures like Albert Hofmann will not let you lie. The experience of creating your own product is invaluable. If you came to the foreign exchange market of your own free will, believe me, this is for a long time.

The study of patterns, the search for solutions and their implementation forever change the perception of the world, making its picture more complete, and life takes on additional meaning.

In fact, developing your own Expert Advisor is an extremely interesting process. Even if you are not going to link your career with trading in the financial markets, the knowledge gained can be applied in other areas. Trade and development is what the entire modern economy rests on, and today you have the opportunity to experience it personally.