This paper presents a novel approach for determining the best combination of groynes in terms of their number, lengths and positions for controlling bank erosion. The vulnerable bank is considered to be protected if a very small value of water flow speed is achieved on the near bank area. A linked simulation–optimization model is developed in this regard which minimizes the total construction cost of the groyne project. At the same time, a constraint in terms of low flow speed in a predefined zone is incorporated, which helps in bank erosion prevention. In the simulation model, the depth-averaged shallow water equations are solved using a finite difference scheme. The optimization problem is formulated in three different approaches to tackle different types of in situ field problems. Genetic algorithm (GA) is used to solve the optimization problem. The proposed optimization model is used in two hypothetical test cases including one straight channel and one meandering channel. The results obtained with all the three formulations are found to be logical and establish the potential of the present model for application in real cases.