System Messages

This forum is for members of the Apiary Investment Fund. To learn more about how to get involved,
sign up now for a 14 day free trial or fill out the form below and preview the training right now!

Genetic Algorithm Optimization of Trading Strategies

 
Back to Forums
 
 
dbaechtel

Genetic Algorithm Optimization of Trading Strategies

I am having some success with using Genetic Algorithm Optimization on trading strategies.
See: https://en.wikipedia.org/wiki/Genetic_algorithm

How does this work? Technically, Genetic Algorithm Optimizers find optimal parameters to maximize a given fitness function for a given system.

In FOREX, what is the fitness function to be maximized? Maximum Profitability of a trading strategy over the long haul.

What are the parameters? In any trading strategy you have made choices of several trading parameters: timeframe, choice of indicators, indicator parameters, conditions to open a trade, conditions to close a trade, when to take a profit, stoploss limit, etc. Up till now, you have just guessed what the best parameters might be. It is possible for a Genetic Algorithm to find optimal settings for all of these parameters that will maximize the return Profit over a set of trial data.

What is the trial data? If your trading strategy parameters were optimized to return a maximum Profit for the last 1 or 2 years of historical market data, then, even though market characteristics change occasionally over time, you would think that your optimized trading strategy would do well under current (live) market conditions.

Where do you get historical market data to use for the GA optimizer? The easiest free source that I have found is https://tickstory.com/. You can get bar and tick data for many currency pairs for periods from 1 month to 5 years on various timeframes, down to M1 data.

How does the GA optimizer work? You construct a Chromosome of all of the parameters that need to be optimized. You also create a fitness function that tells the GA how good(fit) a given a Chromosome is. Then the GA creates many chromosomes using random numbers at first, and then using goal seeking, mutation, momentum, and gene crossover functions, the GA modifies these trial Chromosomes to seek a Chromosome that maximizes the fitness function.

Home does this work in the Forex case? Let's say you want to Optimize a three SMA Trading Strategy. The 3 SMA trading strategy uses 3 SMA indicators of increasing periods on a selected timeframe and opens a contract when all 3 indicators agree (up or down), and closes an open contract when the middle SMA indicator goes in the opposite direction of the open contract. For additional risk management, the strategy may also use a trailing stoploss limit that will limit the loss on any open contract.

What are the parameters to be optimized?
* trading timefrsame
* SMA period(P1)
* SMA period (P2)
* SMA period (P3)
* Stoploss limit.

Wouldn't you like to know the values of the parameters that maximized the Profit for the EUR/USD over the last 2 years?
Would you like to know how those optimized parameters have changed since to 2016 presidential election until today?
Would you also like to have a unique set of optimized parameters for the USD/JPY or any other currency pair?

How do we do this for the 3 SMA Trading example?
1. capture the historical data for the desired currency pair and the desired period of time (6M, 1Y, 2Y, 5Y).
2. construct the Chromosome:
* T = timeframe selection in minutes: 1, 5, 10, 15, 30, 60.
* since P1 < P2 < P3: let P1=A, P2+A+B, P3 =A+B+C in ticks. A, B and C can be in the range of 1 to 300 ticks.
* S = stoploss in .1 pips
So the Chromosome becomes T:A:B:C:S.
3. Now, the GA is run, and every Generation, the GA will request that the application evaluate candidate Chromosomes (set of parameters) for fitness. At this point the application uses the candidate parameters and runs the trading strategy against the historical data and accumulates the total Profit/Loss generated by the trading strategy, using those parameters, over the entire historical period.
4. With this Fitness (Profit/Loss) evaluation of the candidate Chromosomes, the GA will use the Genetic optimization functions to locate the Chromosome with the parameters that will provide the best Fitness (Profit) over the test (historical) data.
5. Now you have a set optimized parameters that worked best to maximize the Profit over the test period and should work well in the current period. The test runs may also give you an idea of the Profit and drawdown ranges to be expected.
6. You can use the optimized set of parameters to trade manually, or plug them into an Expert Adviser (EA) to trade automatically 24 hours per day. With a set of optimized parameters specific to each currency pair, You can trade multiple currency pairs simultaneously using optimized strategies.
7. You can use the GA optimizer periodically with updated historical data to:
* fine tune the trading parameters to adjust for changing market behavior,
* evaluate changes to your trading strategy,
* evaluate different trading strategies.

Is anyone interested in this technique?
Any questions?
Is anyone interested in more details?

Sat, 12/23/2017 - 9:30pm
 
Norma Jenner

Very interested. I'd missed this post.

 
cuttoheal

DB,

As a human biologist, I like the logic.
Does it translate to trading?

Tom

 
Rookie

I like the concept, I'd bet Jeff could make some great input ideas. Fram my view as long as I can download the data directly into the tester and then run it automatically at variable speeds it would be a wonderful tool to test ideas out on, twerking on the spot and re-running. I would enjoy useing it quite a bit.

 
dbaechtel

cuttoheal,

Did you ever guess what the best SL or TP limits might be for your trading strategy?

Well, I take a trading strategy, then I take all of the numerical trading parameter values, like SL, TP, MA period, etc and fashion them into a Chromosome. I construct a Fitness function that lets the GAO (Genetic Algorithm Optimizer) select which Chromosome is best. It looks like:
Fitness = (numContracts > 0) ? profitLoss * 2
+ 9 * (double)(numWins - numLoss)
+ 18 * maxDrawdown
: -100000 - numLoss / 100;

I have added interfaces into my Alveo Expert Advisor(EA) so that I can feed a years worth of Historical Price data into the EA, one bar at a time, just like Alveo does when it calls an EA in realtime. The GAO then runs through 15000 possible Chromosomes each Generation, testing each one by calling my EA, and selects 500 of the best fit. Then the GAO takes those 500, uses them in Pairs as parents to foster modified children Chromosomes and inserts them back into the Population of 15000 again and test all of the Population again. The GAO repeats this process for 100 or more generations until the single Best Chromosome remains unchanged for 40 generations (is stable).

At the end of this, I get the Best Chromosome that maximizes the Fitness function over the 1.5 million or more tests that were performed. I can take the Best Chromosome and convert it back into the Best set of trading parameters that satisfy the conditions in the Fitness function over the Historical period (12 months of 2017) tested.

The assumption is that the Best set of trading parameters that gave the best performance over the recent historical period, then they should perform well in current markets.

During this process, I learn alot about the trading strategy EA, and I can manipulate the Fitness function to get results that I prefer.
Once I have good trading parameters, I can have my EA use those trading parameters directly for trading in the markets.

In this way my trading strategy EA is "tuned" to provide optimized trading performance in the markets.

I will have the details of this process in the second edition of eBook on Automated Trading.
Send me a note if you are interested.

 
Charles Moeller

Seems like a beneficial, but horrendous, process that is simplified for the user by software.

 
sk34trader

Hi D,
Sounds very awarding but complicated. Wish there was an elementary way of understanding and applying step by step, level by level. Wish I could understand it at your level, someday.
Thanks for sharing.

 
dbaechtel

sk34trader,

Take a look at https://en.wikipedia.org/wiki/Genetic_algorithm for the basics.

If you have a system to test (your Expert Adviser) and a system to test it in (you simulating environment) and some Historical data (e.g. data from Tickstory) then you can set up a Genetic Algorithm Optimizer to run your system in the simulation environment, feeding it Historical data just like the real market would. If you identify the parameters to optimize (e.g. Stoploss and TakeProfit limits) and provide a fitness equation on how to judge the "fitness" of the result to meet your goals (e.g. maximize Profitability), then the GAO can iterate through millions of possible values of your parameters and tell you which values will maximize your fitness equation (goal).

I have an environment that I have written in Microsoft's Visual Studio that does just this. The GAO can run through 12 months worth of M1 data (360000 bars) through a target EA is less than 1/5 of a second. It needs to be able to do this if it is going to test about 1.5 million possible trading parameters values.

The GAO works well and provides answers that are not easily obtainable through other means.

 
dbaechtel

sk34trader,

Be aware that there is a difference between backtesting and optimization.

In Backtesting, if you specify a trading strategy and one set of trading parameter values, the backtest will tell you how well it did over a specific timeframe (e.g. the last 12 months).

In Optimization, the goal is to find the best set of parameters that will fit your specified goals (e,g, maximum Profitability + minimum Risk). Optimization runs more than 1 million backtests, searching for the best parameter values each time.

I don't know of anyone else that is doing Forex trading optimization at this time.

 
dbaechtel

Gold III

Achieved.png
 
joniup

Congrats........

 
burton

dbaechtel, congrats...i'm sure that feels really good. well done! =)

 
burton

also, thank for the ga info dbaechtel...definitely going to use what you shared.

 
sanjasamja

There are serious issues with every backtester and optimizer. While when building a trading strategy it is good to use a backtester to determine if you throw a strategy away or not, using a backtester to measure performace of a strategy or even genetic optimizations to optimize the paramater settings for a strategy, never really work. if you should decide to choose a an optimized "set" of parameters after looking at the "alleged" performance characteristics, you appy it again to same chart, and see a difference in its PnL, DD, or perhaps you see that its ratio of wins/losses has improved - BEWARE! THIS DOES NOT HAVE MUCH BEARING WHATSOEVER ON A LIVE TRADING ACCOUNT. 99.99% of the time, genetic optimizations will "over-fit" and as such their remedy to this is to apply various monte carlo methods. However, those really don't work either. Ever developed a strategy that gave tremendous results on the backtester, then the second you pop it on a live account it gets eaten alive? That is because the live markets are entrenched with many other things that happen, and with trading experience you learn to build in safety precuations or you lose your money quickly. Safety precautions like volume and spread filters. While I am just learning to trade, my fiance has been trading for 17 years and the past 5 years jnon-stop 24/7 building new advanced models for trading. I have learned that the live market is not a market as we would expect, we are not trading against other traders in FOREX, we are not trading against foreign exchanges of currency affectingthe market, we are not even trading against liquidity pools taking the other side, we are trading against a market flooded with super advanced algorithms, millions of them, moving the market up or down, and for that reason alone, you should never stay in a trade for more that 15 bars and if possible trade on the M1 and M5, they are the most accurate to trade on, and the general rule of thumb I am learning is get in and get out of the trade as quickly as possible. Automated trading can be very successful if you have a lot of experience and know precisely how to counter the unforeseen by keeping your balance and not losing your money, through sophisticated money management systems, using advanced dynamic indicators, and gut instinct. My only hope is if you read this, you will consult with an expert algorithmic trader before trying to launch your own and do not trust in backtesters or optimizers unless you really know what you are doing with them.

 
burton

sanjasamja,

definitely a great read...some great points shared.

the thesis it clearly has a bias, which is to be expected. still, not all algo are designed to trade, some algos in fact are there to distract while the real trade is captured and executed. the world of algo trading is an ecosystem within itself...i probably should clarify, hft algos (i.e., high frequency trading).

not sure that i would compare automation are a retail traders setup to hft alogs.

the existence of algos does not necessarily make it more difficult for the retail trader...algos, by design, will have a bias, it's part of the human dynamic, which is part of the algo creation. also, the ecosystem of the overall currency market is to complex and it's scale to great not to have algos create unforeseen opportunities by their very action.

nothing, of any meaningful scale, happens without creating a market ripple.

15 bars, lower time frames, okay, if that works for you. there is no one time frame that is more accurate than the next. it's a question of how it is being traded.

one of the bigger discussions in the forum is the idea of "holding trades", not all trades can be held and to do so artificially is to introduce risk. i know, this is obvious to some, what may not be obvious is that with a well timed entry and a trade that is aligned with the market...that position can actually be held and by doing so provide the trader a rich opportunity to trade the middle of the move at a substantially reduced risk. what's the point...

positioning, the stop loss, the take profit are arguably more important than the number of bars to hold a trade and or the time frame being traded.

there is no one way that "is best"...there are only different approaches to the same challenge...

"how to be consistently profitable"

again, some really great information and insights with regard to developing and testing a given setup, really appreciate the read.

thank you