In this post, we outline the architecture of our latest trading bot, built using Python, Backtrader, Telegram and Django
What this bot does …
This trading bot monitors over 1000 markets daily to identify buy and sell signals.
The triggers are generated by a number of relatively sophisticated Technical Analysis patterns which are overlaid onto the market candles fetched from a number of exchange APIs.
The bot then ‘paper trades’ (so it does not place live trades on the exchange) on these markets and monitoring positions. It makes the orders available to a separate live trading bot via an API for live trading on an exchange where required.
The bot has a web front-end for our collaborators showing current status, technical analysis charts for each market, open orders, recent paper trades, etc.
The bot also uses the Telegram API so that trade messages and orders can be posted there to give real-time updates.
The diagram shows how the various pieces fit together in this bot architecture.
Database — the brain
The PostgreSQL database sits at the core of the bot, this holds all the market data, calculated indicators, assets, statuses, orders, trades and results history. It is like the brain at the centre of the bot and everything feeds into this.
It is important for us to use a database rather than storing information in memory so that it can continue to work if the server restarts and also so that the data can be made available to a web app.
Originally we set up the bot using a SQLite database using SQLAlchemy as the Python API, but the volume of market data and indicator was causing performance issues. We switched to a PostgreSQL database backend managed by Django.
Web app front-end
The web app front-end enables collaborators to monitor the progress of the bot and delve into any of the 1000+ assets it is monitoring. There is a lot of human interaction required to tune and select the right algorithms.
The web front-end mirrors the core Bot database. We have separate screens for each market, open orders, completed trades, and performance analysis, etc.
There are many web front-end frameworks that could be used for this. We used the Python Django framework as we know it well, and it has a good framework for setting up the database structure itself. In our bot, Django manages the database tables.
This bot is set up to scan the markets and trigger paper trades only. It does not live trade on exchanges itself. This was a design choice with two key benefits:
- The bot can scan a large number of markets including ones that we may not want to live trade, therefore collecting data and insight we can use to improve the algorithms
- The bot does not need to hold any exchange API keys (and therefore be a security risk) as the market data is available on public APIs
To live trade, a separate, more secure, bot is used. This live trading bot fetches the proposed orders from the paper trading bot via this REST API. It can then trade securely with the relevant exchanges.
The Django REST framework provides a simple way create a REST API quickly in the Django project.
Whilst the web app gives a lot of information for analysis, we also wanted a mechanism to give real-time updates on orders and trades. This is provided through Telegram.
The Python Telegram API is simple to set up and with just a few lines of code, we can use it to post order messages or trade messages to our smartphones via the Telegram app and the Telegram bot we set up for this purpose.
Market data and technical analysis (TA) indicators
In the previous two iterations of our bots, we have used the exchange APIs to fetch market data, and then used python code to calculate the indicators we wished to use.
However this latest version of the bot has a more sophisticated set of technical analysis (TA) indicators. This was starting to become code spaghetti, so we moved to use the open-source Backtrader module for Python. This is a good system that contains many of the standard TA indicators. It also enables us to create our own custom indicators in a structured way.
Backtrader can interface directly with the public exchange APIs and fetch the candle data needed for the TA indicators. We then store the candles and the indicators in the PostgreSQL database so that our algorithms can use them.
A nice feature of Backtrader is that it can create stock charts with the indicators plotted onto the candles. This is built upon Matplotlib so is customisable. We used this to create the charts needed that can be displayed to users in the web app.
Trading algorithms and order management
The trading algorithms themselves use the latest indicators stored in the database to trigger entries or manage trade exits. Algorithms vary for different strategies and different markets.
This is where the real profits and losses happen. The algorithms are developed on charts by human traders and then we code them into the bot and backtest them thoroughly using the bot.
The bot also has a module to manage the paper orders, fill trades, and set up stop-losses. This is needed so that the bot can fully paper trade and we can backtest effectively and monitor performance.
In both cases, the algorithms and the order management was built bottom-up in Python.
Jobs and scheduling
A functioning trading bot requires a number of processes to be regularly run throughout the day. The main jobs include:
- Scanning markets to fetch latest candle data, run the indicator jobs, store to database and then check the algorithms
- Revalidating any markets that failed last run
- Checking the exchanges from time to time for new assets
- Backtest runs for specific markets to test new algorithms
These jobs were created as python scripts which have been scheduled using CRON on the deployment server so that they can be run on a set schedule.
There are many good ways to structure the architecture of a trading bot. Our approach is just one of those set ups, drawn from our experiences having developed two other, less successful, bots in the past.
The Trading Bot Series covers our project, building a complete trading bot using Python, exchange APIs, Backtrader and Telegram