Analyzing Evolutionary Algorithms (Natural Computing Series)
Author: Jansen, Thomas
Date: 2013-01-24
Publisher: Springer Berlin Heidelberg

Thomas JansenNatural Computing SeriesAnalyzing Evolutionary Algorithms2013The Computer Science Perspective10.1007/978-3-642-17339-4© Springer-Verlag Berlin Heidelberg 2013

Natural Computing Series

Series EditorsG. Rozenberg, Th. Bäck, A. E. Eiben, J. N. Kok and H. P. Spaink

Leiden Center for Natural Computing

For further volumes: http://www.springer.com/series/4190

Thomas Jansen

Analyzing Evolutionary AlgorithmsThe Computer Science Perspective

Thomas Jansen

Department of Computer Science, University College Cork, Cork, Ireland

ISSN 1619-7127

ISBN 978-3-642-17338-7e-ISBN 978-3-642-17339-4

Springer Heidelberg New York Dordrecht London

Library of Congress Control Number: 2012954385

© Springer-Verlag Berlin Heidelberg 2013

ACM Computing Classification: F.2, F.1, G.1, I.2

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

I dedicate this book to Ingo Wegener, my Doktorvater.

Preface

Evolutionary algorithms are general randomized search heuristics, inspired by the concept of natural evolution. They are easy to implement and easy to apply, thought to be robust problem solvers, popular in circumstances where there is no good problem-specific algorithm known and where there is not enough time or expertise to develop one. Their development and application is motivated by practical needs. Therefore, the theory of evolutionary algorithms seems to be a contradiction. In fact, many people are very skeptical toward evolutionary algorithm theory and reject it as impractical, useless, or even just plain wrong. Admittedly, there were times when evolutionary algorithm theory was dominated by unfounded claims, risky hypotheses, and overgeneralizations. But evolutionary algorithm theory has seen a change in the last two decades, a change that itself was more revolutionary than evolutionary and that has lead to a rigorous, sound, and arguably useful theory. At the heart of this novel theory for evolutionary algorithms is the insight that evolutionary algorithms are in fact randomized algorithms and that consequently they should be analyzed as randomized algorithms. This establishes the study of evolutionary algorithms as an important topic in the area of computer science. Analyzing evolutionary algorithms as randomized algorithms is by far the most important and fruitful form of theory in the area of evolutionary computation today.

The analysis of evolutionary algorithms is a lively and very active field of research. Many different and tremendously useful tools and methods for analyzing evolutionary algorithms have been and continue to be developed. In this book, an introduction to this field of research is presented that makes it accessible by introducing the most important and fundamental analytical methods, presenting them in a rigorous manner, including complete proofs, and demonstrating how they are applied by means of a number of instructive examples. This should enable anyone to enter this fascinating and fruitful field of research: people interested in further developing the theory of randomized search heuristics, practitioners in evolutionary computation who appreciate the importance of a solid theoretical foundation, and people who teach evolutionary computation and prefer solid and proven facts over rules of thumb. Since there is nothing more practical than a good theory, it is hoped that this book on analyzing evolutionary algorithms is seen as not merely a theoretical exercise but as a useful guide into practical evolutionary computation.

Many people have contributed to this book in different forms. I avoid missing any of them by extending my sincere thanks to all of them without naming them: the research groups at the TU Dortmund, George Mason University, the Max-Planck-Institut für Informatik, and the University of Birmingham and everybody who participated in discussions at Dagstuhl seminars about evolutionary algorithm theory, at FOGA workshops, ThRaSH workshops, workshops at GECCO and PPSN, or conferences—they all deserve to be acknowledged for contributing insights, expertise, and motivation. Moreover, I gladly acknowledge the generous support by the Deutsche Forschungsgemeinschaft (DFG), the German Academic Exchange Service (DAAD), and Science Foundation Ireland (SFI).

Thomas Jansen

Cork, Ireland

Contents

1 Introduction 1

1.1 Overview 4

1.2 Remarks 5

2 Evolutionary Algorithms and Other Randomized Search Heuristics 7

2.1 Modules of Evolutionary Algorithms 9

2.1.1 Initialization 9

2.1.2 Selection 10

2.1.3 Mutation 12

2.1.4 Crossover 14

2.1.5 Termination Criterion 16

2.2 Parameters of Evolutionary Algorithms 17

2.3 Typical Evolutionary Algorithms 19

2.4 Other Simple Randomized Search Heuristics 22

2.5 Design of Evolutionary Algorithms 24

2.6 Remarks 29

3 Theoretical Perspectives on Evolutionary Algorithms 31

3.1 Approaches Based on Markov Chains 32

3.2 Schema Theory 37

3.3 Run Time Analysis 41

3.4 Remarks 43

4 General Limits in Black-Box Optimization 45

4.1 No Free Lunch 50

4.2 Black-Box Complexity 62

4.3 Remarks 83

5 Methods for the Analysis of Evolutionary Algorithms 85

5.1 Fitness-Based Partitions 86

5.2 A General Lower Bound for Mutation-Based Algorithms 100

5.3 Typical Events 108

5.4 Drift Analysis for Lower Bounds 112

5.5 Drift Analysis for Upper Bounds 123

5.6 Typical Runs 129

5.7 Delay Sequences 135

5.8 Random Family Trees 142

5.9 Remarks 153

6 Select Topics in the Analysis of Evolutionary Algorithms 157

6.1 Crossover 157

6.2 Mutation 177

6.3 Cooperative Coevolution 201

6.4 Combinatorial Optimization Problems 216

6.5 Remarks 235

A Fundamentals237
A.1 Landau Notation237
A.2 Tail Estimations238
A.3 Martingales and Applications240
A.4 Remarks243
References245
Index253

Thomas JansenNatural Computing SeriesAnalyzing Evolutionary Algorithms2013The Computer Science Perspective10.1007/978-3-642-17339-4_1© Springer-Verlag Berlin Heidelberg 2013

1. Introduction

Thomas Jansen^1

(1)

Department of Computer Science, University College Cork, Cork, Ireland

Abstract

As the name evolutionary algorithm (EA) suggests, these algorithms have some link to biology. In fact, their inventors were inspired by the idea of natural evolution. There can be many different reasons for mimicking the process of natural evolution by an algorithm and investigating the behavior of this derived algorithm. From a biologist’s point of view the idea is to formalize the model of evolution one has in mind, and to synthesize this idea into an algorithm. When in subsequent experiments the algorithm behaves like the natural archetype, the model of evolution may be called established. If the algorithm behaves unexpectedly and differently, the model of evolution is shown to be false.

As the name evolutionary algorithm (EA) suggests, these algorithms have some link to biology. In fact, their inventors were inspired by the idea of natural evolution. There can be many different reasons for mimicking the process of natural evolution by an algorithm and investigating the behavior of this derived algorithm. From a biologist’s point of view the idea is to formalize the model of evolution one has in mind, and to synthesize this idea into an algorithm. When in subsequent experiments the algorithm behaves like the natural archetype, the model of evolution may be called established. If the algorithm behaves unexpectedly and differently, the model of evolution is shown to be false. So, for biology evolutionary algorithms can be used as an experimental tool for investigating natural evolution. From an engineering point of view, natural evolution may appear to be some kind of ‘improving’ process. Building an algorithm that mimics this process one may hope to obtain some kind of ‘improving’ algorithm. Thus, engineers may use evolutionary algorithms anywhere where something shall be improved—hence everywhere. The general approach of applying methods and mimicking systems found in nature in the design of technology is known as bionics, and evolutionary algorithms are merely one example. The most typical task evolutionary algorithms are applied to is, in fact, optimization. This is true even though evolutionary algorithms almost never come with a guarantee to find an optimal solution in reasonable time. They come, however, with the promise to find reasonably good solutions in acceptable time.

A totally different point of view is offered by the perspective of computer science. For a computer scientist, an evolutionary algorithm is an instance of a randomized algorithm. This implies that it should be designed, described, implemented, and analyzed as randomized algorithms are. The questions computer scientists ask about algorithms are centered around the algorithm’s correctness, its (expected) run time, and possible improvements. Adopting this point of view comes with the advantage that it does not matter at all what our personal opinion with respect to natural evolution is. If we accept the idea of natural evolution, considering evolutionary algorithms is an interesting endeavor in itself. But it is also perfectly sensible to reject the idea of natural evolution and still accept that evolutionary algorithms have been designed and implemented and that it therefore makes sense for a computer scientist to consider them as algorithms. What we do accept from biology, however, is part of the vocabulary. This is part of a well-established tradition, and there is no point in introducing new terms as long as the well-known terms make sense.

Knowing that evolutionary algorithms stem from biology enables us to understand why some people find these algorithms such a fascinating field of study and why they hope to gain some understanding of evolution, genetics and life itself by these studies. It is not ours to judge such hopes. We defer such thinking to the fields of biology and philosophy. Instead, we deal with these algorithms simply as algorithms and care solely about their analysis as randomized algorithms. For computer scientists, such an endeavor is equally fascinating and fulfilling.

Our main focus will be the analysis of the expected optimization time of evolutionary algorithms, i.e., the time needed on average to solve some concrete optimization problem. The most satisfying and appropriate result one can deliver is a rigorously proven theorem about this time. This implies that we use a mostly theoretical approach to the analysis of evolutionary algorithms that is similar in spirit and methods to the analysis of ‘classical’ randomized algorithms. One main difference is that usually randomized algorithms are designed for some specific problem and with analysis in mind. Nowadays, it is virtually impossible to publish a new problem-specific algorithm without a formal analysis proving it useful, i.e., at least comparable to the best-known algorithms for this problem. Therefore, one strives not only for efficient algorithms but also for efficient algorithms that are particularly easy to analyze. This is different for evolutionary algorithms. They are general randomized search heuristics that are designed with no particular problem in mind. Neither biologists nor engineers care about the analysis of the expected optimization time when designing new evolutionary algorithms. Therefore, it comes as no surprise that evolutionary algorithms are especially difficult to analyze. This motivates why we deal with particularly simple evolutionary algorithms and do not hesitate to introduce artificial optimization problems that facilitate the analysis. Such artificial problems have the advantage of having a clear structure so they are easy to describe and understand. One has an intuitive understanding of the problem and therefore an intuitive idea of how a specific evolutionary algorithm will behave on this problem. This makes it either very satisfying to prove this intuition correct or very instructive to understand where and why it fails. Moreover, it enables us to teach evolutionary algorithms in class in a rigorous and well-structured way.

Emphasizing theory does not at all imply that we deprecate practical application of evolutionary algorithms in any way. On the contrary, it goes without saying that only the practical application of evolutionary algorithms justifies the study of these algorithms. If nobody was using evolutionary algorithms in practice, there would be no point in doing theory on evolutionary algorithms. We prefer to study evolutionary algorithms from a theoretical point of view for several reasons. First of all, we believe that theoretical insights help design and adapt evolutionary algorithms for practical problems in a more informed way. Furthermore, knowing about the potentials and limitations of evolutionary algorithms enables us to decide when it is wiser to use different methods. Finally, teaching proven theorems and well-structured examples in class is much easier and more convincing than teaching a bunch of beliefs and rules of thumb.

For computer scientists it is helpful to realize the fundamental difference between evolutionary algorithms and the usual problem-specific algorithms and make this difference explicit. As one learns in classes about design and analysis of (randomized) algorithms, the first thing to observe is the problem to solve. In practice, it is probably necessary to come up with some formalization or model of the problem before it can be dealt with by means of some algorithm. In textbooks, problems typically already come in formalized form. With this formal problem in mind, one starts looking for an efficient algorithm, either using an already existing one or designing an entirely new one. After defining this algorithm, one performs verification, analysis, implementation, and tests. This process may be repeated until some satisfactory algorithm is found.

For evolutionary algorithms things are completely different. Based on a good idea, one designs and implements an evolutionary algorithm. Then one would like to find a problem that can be solved with the help of this algorithm. In practical applications this means that there is a whole variety of different randomized search heuristics, among them evolutionary algorithms, and one starts by representing the problem in a way that can be dealt with by such a heuristic algorithm. Most often a process of repeated modifications of the evolutionary algorithm (or other randomized search heuristics) begins until satisfactory results are produced. What makes evolutionary algorithms particularly attractive is the ease by which they can be applied to problems of very different kinds. They come as off-the-shelf tools and require very little work and expertise. Typically, they consume more computation time than problem-specific algorithms. But since experts are expensive and computation time becomes cheaper and cheaper, such algorithms are an attractive alternative to problem-specific solutions. Note that in this process the formal analysis of evolutionary algorithms is nowhere included. Therefore, it comes as no surprise that analysis turns out to be so difficult. It is, however, worthwhile when it provides us with insights that enable us to find better evolutionary algorithms in shorter time.

The way evolutionary algorithms are usually designed implies that they do not make explicit assumptions about the problem to solve and do not exploit the properties of the problem at hand. They act as if they had no access to the problem instance at all, as if the concrete problem instance was hidden from the algorithm. This scenario where the problem instance is not known and information about it can only be gathered by sampling points in the solution space and obtaining function values as measures of the solution’s quality is called black-box optimization. Black-box optimization is different from the classical optimization scenario in fundamental ways. These differences have important consequences with respect to both the algorithms we design and the difficulty of the optimization problems themselves.

A problem-specific algorithm is designed for one particular problem. We expect from such an algorithm that it solves this problem correctly and efficiently. Moreover, we expect to have proven statements about the algorithm’s worst-case run time and the worst-case quality of the solutions it delivers. We are willing to accept that the algorithm is difficult to design, that it may be complex and hard to understand, that its implementation may be tedious and time-consuming. Our expectations with respect to evolutionary algorithms are entirely different. We expect an evolutionary algorithm to be ‘ready’, to be easy to implement and to adapt to a new problem. Moreover, we expect it to be easy to use. We hope that it will deliver a solution of acceptable quality in reasonable time, but we are willing to accept that this is not guaranteed in any way.

These different expectations imply where evolutionary algorithms should be used and, perhaps more even more important, where they should not be used. If a problem is not well understood, if no problem-specific algorithm is known and there is not sufficient time or expertise to design one, then evolutionary algorithms can be used. In particular, if there is computation power in abundance or enough computation time to spare, they are most definitely worth a try. If, on the other hand, one is dealing with a well-known problem where good problem-specific algorithms are known, it would be foolish not to apply such a problem-specific algorithm and try an evolutionary algorithm instead. We cannot hope that evolutionary algorithms beat problem-specific algorithms for any problem. Even if an evolutionary algorithm turns out to be the best currently known algorithm for some problem, it is probably possible to understand why this evolutionary algorithm performs so well for this problem and utilize this knowledge to design an even better problem-specific algorithm. Designing this algorithm in a careful way may even enable us to prove upper bounds on its run time and lower bounds on the solution quality, something that was not available for the evolutionary algorithm.

1.1 Overview

In addition to this introduction, the book contains five more chapters and an appendix. In case of need, some fundamentals that can usually assumed to be known when someone has some background in algorithms and their analysis (which includes some background in probability since this is required for the analysis of randomized algorithms) can be found in Appendix A. It is suggested to glance over Appendix A before continuing to read the other chapters.

Chapter 2 defines evolutionary algorithms, parameters, and notation and discusses the design of new evolutionary algorithms. The sections that cover the different modules of evolutionary algorithms (Sect. 2.1 ), define their parameters and to some degree notation (Sect. 2.2 ) and present some simple and common evolutionary algorithms (Sect. 2.3 ) are required in order to follow the rest of the book. The section about other simple randomized search heuristics (Sect. 2.4 ) is also very useful. Reading the section about the design of evolutionary algorithms (Sect. 2.5 ) is recommended but optional.

Chapter 3 describes different theoretical perspectives that can be assumed when considering evolutionary algorithms. It aims at putting the approach taken here into perspective and can be read later. However, the short Sect. 3.3 that discusses the benefits of our approach is probably a useful motivation.

Chapter 4 does not directly deal with evolutionary algorithms. It considers the field of black-box optimization and discusses fundamental limitations, providing an understanding of what can never be achieved by any randomized search heuristic. Knowing about these boundaries of the feasible is highly useful and therefore reading this chapter is highly recommended. However, it is not strictly necessary for the rest of the book and may be omitted if the interest is more in understanding existing evolutionary algorithms and less in designing one’s own.

The main part of the book is Chap. 5. It introduces and discusses different methods for the analysis of evolutionary algorithms, illustrating them with many examples where actual simple evolutionary algorithms are analyzed on different important fitness functions that serve as test beds, exemplify important situations and properties, and help to compare evolutionary algorithms in a rigorous way. The chapter starts with the most fundamental and important methods, always preferring applicability when confronted with the choice between an accessible version or a more complex variant that yields somewhat stronger results. The sections about delay sequences (Sect. 5.7 ) and random family trees (Sect. 5.8 ) concentrate on methods that are to some degree more focused on specific situations and therefore less widely applicable.

Chapter 6 considers four topics in evolutionary computation in some detail and applies the methods described in Chap. 5. The sections deal with the topics of crossover operators (Sect. 6.1 ), mutation operators (Sect. 6.2), cooperative coevolution (Sect. 6.3 ), and combinatorial optimization (Sect. 6.4 ), respectively. They can be read independently. The section on mutation (Sect. 6.2 ) contains discussions of the choice of the mutation probability, a simple dynamic scheme for setting the mutation probability that helps the user to avoid this choice, and a discussion of the design of a problem-specific mutation operator. These parts of Sect. 6.2 covering these three different aspects of mutation can again be read fairly independently.

The main goal of the book is to introduce the reader to the analysis of evolutionary algorithms as randomized algorithms, to convince readers that this is the most useful and sensible form of doing theory on evolutionary algorithms, and to enable the reader to join the continuing research efforts in this area. Being accessible is paramount, which implies that nowhere are the most elaborate (and therefore complicated) variants of methods described. Clearly, at the research front the most current methods should be known and applied. To facilitate this, pointers to literature are provided at the end of each chapter. It is hoped that being provided with an accessible description of the most fundamental methods and being helped to find more advanced methods and applications is found to be a useful way to start the journey in one of the most fascinating and fruitful fields of research in computer science.

1.2 Remarks

It goes without saying that many people have contributed to the subjects discussed in this book. There are, however, no references in the text. This is meant to increase the readability of the text. Since this is not a monograph concerned with the historic development of the field of evolutionary computation, there is not even an attempt to give proper references to all relevant sources. It is, however, clearly desirable to study original work in order to gain a deeper understanding. To facilitate this, pointers to selected relevant parts of the literature are given in a remarks section at the end of each chapter.

In this book, the emphasis is on theoretical analysis of evolutionary algorithms. Readers interested in the history of evolutionary computation are advised to consider the book by Fogel [40]. For readers mainly interested in practical applications the books by Ashlock [5], De Jong [16], and Eiben and Smith [37] are probably more accessible. The book by Neumann and Witt [94] has a strong emphasis on combinatorial optimization (a topic that we only touch on in Sect. 6.4) and covers multiobjective evolutionary optimization as well as ant colony optimization. The book edited by Auger and Doerr [6] is a collection of contributions to different topics by different authors, necessarily less coherent, but in the same spirit as this book and definitely a useful addition.

References

5.

D. Ashlock, Evolutionary Computation for Modeling and Optimization (Springer, Berlin, 2006)

6.

A. Auger, B. Doerr (eds.), Theory of Randomized Search Heuristics (World Scientific, Singapore, 2011)

16.

K.A. De Jong, Evolutionary Computation: A Unified Approach (MIT, Cambridge, 2006)

37.

A. Eiben, J. Smith, Introduction to Evolutionary Computing (Springer, Berlin, 2007). Corrected 2nd printing

40.

D.B. Fogel, Evolutionary Computation: The Fossil Record (IEEE, Piscataway, 1998)

94.

F. Neumann, C. Witt, Bioinspired Computation in Combinatorial Optimization: Algorithms and Their Computational Complexity (Springer, Berlin, 2010)

Thomas JansenNatural Computing SeriesAnalyzing Evolutionary Algorithms2013The Computer Science Perspective10.1007/978-3-642-17339-4_2© Springer-Verlag Berlin Heidelberg 2013

2. Evolutionary Algorithms and Other Randomized Search Heuristics

Thomas Jansen^1

(1)

Department of Computer Science, University College Cork, Cork, Ireland

Abstract

In our description of evolutionary algorithms we make use of terms that stem from biology, hinting at the roots of evolutionary algorithms. We adhere to these standard notions as long as they do not collide with standard notions in computer science. Evolutionary algorithms are structurally very simple. They work in rounds that are called generations. Evolutionary algorithms operate on some search spaceS, where S is a set. Points are assigned some quality via a function f.

In our description of evolutionary algorithms we make use of terms that stem from biology, hinting at the roots of evolutionary algorithms. We adhere to these standard notions as long as they do not collide with standard notions in computer science. Evolutionary algorithms are structurally very simple. They work in rounds that are called generations. Evolutionary algorithms operate on some search spaceS, where S is a set. Points are assigned some quality via a function f. In the context of optimization, f is called an objective function. In the context of evolutionary algorithms, it is usually called fitness function. Sometimes one distinguishes between a fitness function f and an objective function f′, where the fitness f is utilized by the evolutionary algorithm directly whereas the objective function f′ is in some sense closer to the problem at hand. We will introduce this distinction later when we discuss how we can adapt evolutionary algorithms to a given problem (see Sect. 2.5). For now we are only dealing with the fitness function and do not care if there is an objective function f′ ‘wrapped around.’ The set R is the set of possible fitness values. It may be an arbitrary set; most often it is (some subset of) .

An evolutionary algorithm operates on a collection of points from the search space, called a populationP. The members of the population, i.e., some points in the search space, are called individuals. We use to denote the size of the population, i.e., μ =  | P | . A population is a multiset over S, i.e., it may contain multiple copies of individuals. Since the population changes from generation to generation, we denote the t{ th} population by P t . Choosing the first population P 0 in the beginning is called initialization. Usually for each member x of the population its fitness f(x) is computed and stored. If fitness values are stored this starts with the initialization, where it is done for the complete population. The first step in each generation is to select some individuals from the population that will be used to create new points in the search space. These individuals are referred to as parents. Their selection is called selection for reproduction. Often this selection is done fitness-based: the chances of individuals to become parents increase with their fitness. Then some random variation is applied to the parents where small changes are more likely than large changes. We distinguish two kinds of variation operators. Variation operators that take one parent as input and randomly create one individual are called mutation. Variation operators that take at least two parents as input and randomly create (at least) one individual are called crossover. In any case, the newly created individuals are called offspring. It is not unusual to combine different variation operators, e.g., create an offspring via crossover and use this offspring as input for a mutation operator. Usually, the intermediate step (the offspring before mutation) is discarded; but this may be different in some evolutionary algorithms. There is hardly any limit to creativity when designing evolutionary algorithms and thus there are no really strict rules. After creating the offspring population most often there is some kind of selection for replacement. The reason is that the size of the population is typically not changed during the whole run. Thus, the new population P t + 1 is selected from the old population P t and the newly generated offspring. When describing this selection as selection for replacement, i.e., selecting which individuals are not going to be present in the next population P t + 1, it differs from selection for reproduction in preferring individuals with smaller fitness. One may as well describe this selection step as selection for survival where those individuals are selected that will be member of the next generation. In this description there is no conceptual difference to selection for reproduction. After the new generation P t + 1 is produced (and the old population P t is discarded), it is checked whether some termination criterion is met. If so, some output is produced and the algorithm terminates. Otherwise the next generation starts with the selection for reproduction. This evolutionary cycle is summarized in Fig. 2.1. Note that we do not explicitly include evaluating the fitness of individuals in the evolutionary cycle. We silently assume that whenever the fitness of an individual is needed (usually in selection) the value is either available or will be computed ‘on the fly.’

Fig. 2.1

Outline of a generic evolutionary algorithm. Arrows show control flow, dashed arrows show data flow

Now we describe concrete realizations for each of the six modules: initialization, selection for reproduction, mutation, crossover, selection for replacement, and termination criterion. Since variation operators depend on the structure of the individuals, we have to discuss different search spaces first.

In principle, the search space may be any set S. Often it is structured as Cartesian product of some other sets, i.e., . The two most important cases for both theory and practical applications are and . In applications, combinations of some discrete or even Boolean search space and some real search space are also common. Another important search space is the permutation set, i.e., . The standard search spaces {0, 1} n , , and S n all have in common that points in these search spaces have natural descriptions (and therefore natural implementations) of constant length: bit strings of length n, vectors of n real numbers, and n natural numbers, respectively. This is quite different from search spaces where natural descriptions have varying length. One example of such search spaces is the set of all trees that correspond to arithmetic terms. Most of the time, we will not deal with such search spaces where individuals are more flexible and have no fixed length. They are most often used in evolutionary algorithms that are counted to the subfield of genetic programming.

2.1 Modules of Evolutionary Algorithms

We describe the different modules that can be combined and ‘plugged into’ the algorithmic framework depicted in Fig. 2.1 to obtain a complete evolutionary algorithm. Where necessary, we give different descriptions for the three standard search spaces {0, 1} n , , and S n . Our descriptions are formal and specific. This does not exclude the existence of quite different variants in actual evolutionary algorithms. As already pointed out in the introduction, there is hardly a limit to the creativity of designers of evolutionary algorithms. The descriptions given here will, however, be used throughout this text.

Initialization

In most cases, initialization is done ‘purely at random.’ For the search spaces {0, 1} n and S n this means uniformly at random. In , initialization is most often done uniformly in some restricted part of the search space.

In applications, it may make more sense to start with ‘promising’ individuals that were obtained by means of some heuristic, possibly in some previous run of the evolutionary algorithm. It has to be noted, though, that many evolutionary algorithms suffer if the members of a population are too similar to each other.

When dealing with problems with restrictions, it often makes sense to initialize the population with individuals that respect all restrictions. There are variation operators with the property that by using such feasible solutions as input only feasible points in the search space are produced as offspring. This may improve the performance of an evolutionary algorithm significantly. It is most useful in problems where feasible solutions are difficult to find.

For tests and studies, it may make sense to start in some carefully chosen region of the search space or with individuals having some property of interest. This way one may observe and analyze how the evolutionary algorithm under consideration behaves under circumstances that may be unlikely to evolve in a normal run using plain random initialization.

Selection

Selection appears twice in the evolutionary cycle (Fig. 2.1): as selection for reproduction and as selection for replacement. Since selection for replacement can as well be described as selection for survival, then coinciding with selection for reproduction, we describe all selection mechanisms in a way appropriate for this kind of selection. If selection for replacement is desired, analogous remarks apply but ‘the sign of fitness’ changes: whereas larger fitness values are to be preferred in selection for reproduction, we prefer smaller fitness values in selection for replacement (since we are maximizing fitness).

Often, selection is based on the fitness of the individuals alone. Some variants do additionally take other properties of the individuals or even the population into account. All variants have in common that they do not select in favor of lower fitness values. While one may in principle devise such selection variants, we discard those variants as unreasonable—essentially being designed not in accordance with the spirit of evolutionary computation.

There are two quite different ways how selection may be performed. One way is to select single individuals and repeat this as many times as selected individuals are needed. This applies to almost all selection methods presented here. Such selection methods can be described by the underlying probability distribution. For each individual one provides the probability to be selected. Given these probabilities, selection is usually performed independently with replacement. The other way is selecting all individuals that are needed in one batch. This implies that no single individual can be selected more often than once. This corresponds to performing selection without replacement.

Uniform selection

Select an individual uniformly at random. This is the weakest form of selection that we still consider to be reasonable.

Fitness-proportional selection

This selection method assumes that all fitness values are positive. An individual s in the current population P is selected with probability .

The most obvious disadvantage of fitness-proportional selection is the direct and strong dependence on the fitness values. This implies that changing the fitness function f to f + c for some constant c (from an optimization point of view not really a change) changes the selection probabilities observably.

If differences between fitness values in the population are very large, fitness-proportional selection behaves almost deterministically. This may be something that is not wanted. If, on the other hand, differences between fitness values are very small, fitness-proportional selection behaves similar to uniform selection. This is typical for situations later in the run of an evolutionary algorithm: as all fitness values become larger, the relative differences become smaller and fitness-proportional selection becomes increasingly similar to uniform selection.

Variants of fitness-proportional selection

Since fitness-proportional selection has such obvious drawbacks, there are a number of variants of fitness-proportional selection that aim at adjusting the selection mechanism and avoiding its difficulties without disregarding the main idea.

Scaled fitness-proportional selection

Use fitness-proportional selection but replace the fitness function f by a scaled and translated version , where m and b are parameters. Sometimes these parameters are chosen adaptively depending on the current population. Clearly, this does not really solve the problem. With respect to , all our critical remarks about fitness-proportional selection still apply. Moreover, it may be difficult to find appropriate values for m and b.

Boltzmann selection

Use fitness-proportional selection but replace the fitness function f by e f ∕ T where T is a parameter (called temperature) that allows one to vary the influence of the actual fitness values. Typically, T varies with time, it is rather large in the beginning and gradually lowered.

Boltzmann selection comes with the immediate advantage that additive changes of the fitness function, i.e., going from f to f + c, have no influence on the probabilities to be selected. We observe that

holds for each constant c.

Rank selection

Use fitness-proportional selection but replace the fitness value of individual s ∈ P by its rank, i.e., its position in the list of all individuals of the current population sorted descending with respect to fitness, ties being broken uniformly at random.

It is not difficult to see that each direct dependence on the concrete fitness values is removed here. In particular, this selection method does not change its characteristics during a run as fitness-proportional selection does.

Tournament selection

This selection method comes with a parameter , called tournament size. The selection works by first selecting k individuals uniformly at random. Then the one with maximal fitness is selected. Ties are broken uniformly at random.

The main advantage of tournament selection is that it works in a very local fashion. There is no need to compute the actual fitness values for all members of the population. Only those individuals selected for the tournament need to be evaluated. Another advantage is that the parameter k allows for very simple tuning of the selection strength. Clearly, the larger k is, the more competitive tournament selection becomes. All advantages with respect to independence of the concrete fitness values that we listed for rank selection apply here, too. Finally, tournament selection guarantees that the k − 1 worst members of the population are never selected.

Truncation selection

This is the only selection method that we discuss where all individuals are selected in one batch. The selection is simple: the individuals are selected descending with respect to fitness, and ties are broken uniformly at random. If k individuals are to be selected, the individuals with ranks in are selected.

We introduce two special variants of truncation selection that are both used for selection for replacement only. If all the individuals to be selected are from the offspring population (ignoring the parent population), we call the selection comma-selection. If we select λ offspring from a population of size μ, we write this as (μ, λ). If the individuals to be selected are from both the parent population and the offspring population, we call the selection plus-selection and write (μ+λ).

Selection mechanisms taking the actual individuals into consideration are typically sensitive to the similarity of the individuals. If this is done the selection mechanism tends to aim at increasing diversity in the population by avoiding the selection of individuals that are very similar. We will discuss some concrete examples later when we perform concrete analyses. Another type of selection mechanism takes the ‘age’ of individuals into account. Inspired from nature, something like a lifespan for individuals may be introduced. The most basic form of this mechanism uses age to break ties: if a parent individual and an offspring individual have the same fitness value, typically the offspring individual is favored.

Mutation

Mutation operators depend on the structure of the individuals and thus on the search space. All mutation operators have in common that they tend to create rather small changes. We describe mutation operators for the three standard search spaces {0, 1} n , , and S n .

Mutation operators for {0, 1} n

The parent is x ∈ {0, 1} n with length n. The ith bit in x is denoted by x[i]. We want to favor small changes to x and measure the size of changes by using Hamming distance as our metric. The Hamming distance of two individuals x, y ∈ {0, 1} n is simply the number of positions where x and y differ, i.e., .

Standard bit mutation

The offspring y is created as a copy of x, where for each bit the value of the bit is inverted (a bit-flip occurs) independently with probability p m . The parameter p m is called the mutation probability. The expected number of positions where the parent x and its offspring y differ equals . Since we want to favor small changes, we need p m  ∈ (0, 1 ∕ 2]. Note that p m  = 1 ∕ 2 implies that y is drawn from {0, 1} n independently of x uniformly at random. The most common mutation probability is p m  = 1 ∕ n, flipping just one bit on average.

b -bit mutations

The offspring y is created as a copy of x where the value of exactly b bits is inverted. The positions of these b bits are chosen uniformly at random; is a parameter. Typically, b is quite small: b = 1 is a common choice. In comparison with standard bit mutations with mutation probability p m  = 1 ∕ n, 1-bit mutations have less variance and facilitate analysis. But it has to be noted that the differences in performance induced by changing from 1-bit mutations to standard bit mutations with p m  = 1 ∕ n can be enormous.

Mutation operators for

Most often, the offspring is created from its parent by adding some vector . In principle, m may be chosen in arbitrary ways. In simple evolutionary algorithms, each component of m is chosen in the same way, i.e., independently and identically distributed. Thus, we now describe the way of choosing one component . Since we want to favor small changes, we typically choose m′ as a random variable with mean value 0. We distinguish bounded from unbounded mutations. In bounded mutations, m′ is chosen from some interval [a, b], often uniformly from [ − a, a] for some fixed . More commonly used are unbounded mutation operators, where m′ is not bounded. For these unbounded mutations, however, the probability typically decreases strictly with the absolute value. The most common choice makes use of a normal distribution (Gaussian mutations), where we have as probability density function with parameter σ. We know that the mean is 0 and the standard deviation is σ. In some sense, the choice of σ determines the size of the mutation. Typically, σ is not fixed but varies during a run. The idea is to have σ large when far away from optimal points and small when close by. Often, one chooses σ = 1 fixed and uses an additional parameter to scale the step size, using instead of m′.

Mutation operators for S n

For permutations a number of quite different mutation operators have been devised. Which ones make sense clearly depends on the fitness function and, more generally, the problem at hand. Some mutation operators are designed for specific problems. Here, we concentrate on only a few general mutation operators that can be used in different contexts.

Exchange

Choose with i ≠ j uniformly at random. Generate the offspring y from its parent x by copying x and exchanging i and j.

Jump

Choose with uniformly at random. Generate the offspring y from its parent x by copying x, moving i to position j, and shifting the other elements accordingly.

Combination of exchange and jump

Both mutation operators for permutations, exchange and jump, have in common that they act quite locally. They are not capable of generating arbitrary permutations in a single mutation. But it is often desirable to have this ability since it makes sure that an algorithm cannot become trapped in a local optimum. Thus, we let ourselves be inspired by standard bit mutations with mutation probability p m  = 1 ∕ n. For these mutations, the number of mutated bits is approximately Poisson distributed with parameter 1. Here, we choose according to a Poisson distribution with parameter 1, i.e., . Then we perform k + 1 local operations, choosing exchange or jump each time independently with equal probability.

Crossover

Crossover operators cannot be designed independently of the search space. In this way they are similar to mutation operators. The difference from mutation is that more than one parent is used. Most crossover operators make use of two parents, which is clearly close to the natural paradigm. But there are also crossover operators in use that operate on many more parents.

The idea of crossover is to generate an offspring that is in some way similar to its parents. We will define all crossover operators in a way that they produce exactly one offspring.

Crossover operators for {0, 1} n

Let be the two parents of length n. For crossover operators for {0, 1} n that make use of two parents, it is not unusual to produce two offspring. Such crossover operators produce one offspring by assembling pieces of the two parents. The second offspring is created by assembling exactly the unused pieces of the two parents. This way, for each position the numbers of 0-bits and 1-bits in the two offspring equal these numbers in their parents. Here we stick to describing the construction of one offspring only.

k -point crossover

Select k different positions from uniformly at random. Let these positions be . Then the offspring y is defined by copying the first p 1 bits from x 1, the second from x 2, the next bits from x 1, and so on, alternating between x 1 and x 2. This method can be visualized as having the two parent bit strings cut into pieces after each p i { th} position. Then the offspring is the concatenation of pieces taken alternately from the two parents. An example for n = 9, k = 3, and p 1 = 2, p 2 = 5, p 3 = 6 can be seen here.

We observe that the offspring is equal to both parents at all positions where the parents agree. Usually, only very small numbers of crossover points are used. The most common forms of k-point crossover are 2-point crossover and even 1-point crossover.

Uniform crossover

The offspring is created from its parents by copying their bits in the following way. For each bit the value is copied from one of the parents and the decision among the parents is made independently for each bit and with equal probability. As is the case for k-point crossover, the offspring is equal to both parents at all positions where the parents agree. From the set of all individuals with this property, each is generated with equal probability by uniform crossover. This way the number of possible offspring is usually much larger for uniform crossover than for k-point crossover.

Gene pool crossover

This crossover operates on an arbitrary number of parents. It is not unusual to use the complete population as parents. If we use parents , the offspring is defined by setting y[i] = 1 with probability , and y[i] = 0 otherwise.

Crossover for

For it is possible to have k-point crossover and uniform crossover very similar to the corresponding operators for {0, 1} n . Instead of copying bits, real numbers are copied from the parents. In addition, there is another type of crossover that makes use of the natural interpretation of individuals from as vectors.

Arithmetic crossover

This crossover operates on an arbitrary number of parents like gene pool crossover. Again, it is not unusual to use the complete population as parents. If we use parents , the offspring is created as weighted sum , where the parameters α i sum up to 1, i.e., . Typically, one sets α i  = 1 ∕ m for all . This is called intermediate recombination and defines the offspring to be the center of mass of its parents. Note that this is the only variation operator that has no random component.

Crossover for S n

There is a variety of crossover operators for permutations. Almost all of them create an offspring based on exactly two parents. Most often, two positions are selected uniformly at random and the elements between these two positions in the first parent are reordered according to the ordering in the second parent. Examples include order crossover, partially mapped crossover (PMX), and cycle crossover (CX). We name these examples without giving precise definitions. The reason is that for permutations no successful standard crossover with convincing performance across a wide variety of different permutation problems is known. It is worth mentioning that for specific applications custom-tailored crossover operators have been suggested. Examples are edge recombination and inver-over, both designed for the traveling salesperson problem (TSP).

x 1[2]

x 1[3]

x 1[4]

x 1[5]

x 1[6]

x 1[7]

x 1[8]

x 1[9]

x 2 = x 2[1]

x 2[2]

x 2[3]

x 2[4]

x 2[5]

x 2[6]

x 2[7]

x 2[8]

x 2[9]

y = x 1[1]

x 1[2]

x 2[3]

x 2[4]

x 2[5]

x 1[6]

x 2[7]

x 2[8]

x 2[9]

Termination Criterion

The termination criterion is the final step in the evolutionary cycle (see Fig. 2.1). It decides if the algorithm is to be stopped or if another generation is to be started. In the case of stopping, usually an output is produced. In most cases evolutionary algorithms are used for maximizing the fitness function f, and consequently some x ∈ S with maximal fitness among all visited search points is presented. To achieve this, the current best is usually stored in addition to and independent of the current population. Termination criteria can be more or less flexible. We describe different classes of termination criteria without going into detail. For our goals it is helpful to consider the most basic and simple termination criterion, i.e., avoiding the issue altogether by using no termination criterion at all.

Adaptive termination criteria

These termination criteria may take anything into account. Being completely unrestricted implies that they are the most flexible way of deciding about termination. This may depend on directly observable properties like the population and its fitness values as well as on statistics based on such properties. A typical example for such an adaptive termination criterion is the number of generations since the last improvement. Also, simpler criteria, like stopping once the best fitness value found is beyond some predetermined value, fall into this category.

Fixed termination criteria

In practice, in many cases much simpler stopping criteria are used. Fixed termination criteria stop the algorithm after a predefined number of steps or computations regardless of the actual run. Concrete examples include stopping after a predefined number of generations or a predefined number of fitness evaluations.

No termination criterion

When we consider evolutionary algorithms from a theoretical point of view we avoid the topic of choosing a termination criterion and simply let the evolutionary cycle continue without stopping. In a formal sense, we let the algorithms run forever. What we are interested in is the first point of time T when a global maximum of f is found, where time is measured by the number of function evaluations. We call this point of time the optimization time and are mostly interested in its mean value. For practical purposes this is an important measure. It tells us how long we have to wait, on average, in order to find an optimal solution.

2.2 Parameters of Evolutionary Algorithms

It is already apparent that evolutionary algorithms have a number of parameters that need to be set. Apart from the parameters belonging to some module as defined in the previous section, there are also more global parameters. We list these parameters here and briefly discuss their role. The main purpose is to introduce the notation that is used throughout the text. Note that in the evolutionary computation community there is no common notation in use. Here, we only introduce the most basic and important parameters. Some evolutionary algorithms make use of additional parameters. These will be introduced as needed.

Dimension of the search space n

We use n to measure the size of the search space for {0, 1} n , , and S n . Clearly, n is not really a parameter of the evolutionary algorithm but a property of the search space and thus the fitness function. When analyzing (randomized) algorithms, one usually considers the performance of the algorithm depending on the size of the input. Most often, results are expressed in an asymptotic form (using the Landau notation, see Appendix A.1) and one assumes that the size of the input grows to infinity. For us, n plays the role of the input size. Therefore, we analyze the performance of evolutionary algorithms for . Another important role of the dimension of the search space is to determine the value of parameters of the evolutionary algorithms. Often such parameters are fixed with respect to n. One example is the mutation probability p m , which is most often set to p m  = 1 ∕ n.

Population size μ

The number of individuals in the population, in particular right after initialization, is called the population size μ. There are no clear rules for setting the population size. Clearly, μ needs to be bounded above in a reasonable way. If we adopt the usual perspective of complexity theory, it makes sense to have μ polynomially bounded, i.e., . Typical choices include or , but also much smaller populations like or even μ = 1 are not uncommon.

Offspring population size λ

The number of offspring generated in each generation is called the offspring population size. Clearly, is reasonable. Again, a wide variety of different values for the offspring population size are commonly used. In particular, λ = 1 is a very common choice (even more common than μ = 1), but also having (like ) is not unusual. Of course, what values for λ make sense also depends on the selection mechanism employed.

Crossover probability p c

We denote the probability that crossover is used to produce an offspring as crossover probability p c . Any choice p c  ∈ [0, 1] is possible; the most often recommended choices are quite large constant values like p c  ∈ [0. 5, 0. 8]. In some evolutionary algorithms either crossover or mutation is applied. In these cases we apply crossover with probability p c and consequently mutation with probability 1 − p c . Here we do not do that. Instead, we decide about the application of crossover and mutation independently and have offspring created by means of crossover undergo a subsequent mutation with a certain probability that is called the probability for mutation.

Probability for mutation

Probability for mutation is the probability to produce an offspring via application of a mutation operator. We mention this parameter without actually using it. In the algorithms that we consider we always set the probability for mutation to 1. It is important not to mix this up with the mutation probability p m , a parameter of standard bit mutation. Using the most commonly used mutation probability p m  = 1 ∕ n implies that with probability no bit is flipped. Thus, there is no need to ensure that there are steps without actual mutations by introducing a probability for mutation. It is important to note that some authors use ‘mutation probability’ to denote the probability for mutation and introduce the notion of ‘mutation strength’ or ‘mutation rate’ for our mutation probability.

There are different ways of setting the parameters in evolutionary algorithms. The simplest and most common way is to set all parameters to some fixed values in advance and keep the parameters unchanged during the run. Usually, one experiments a little to find settings that lead to acceptable performance. It is not clear, however, that such static parameter settings are able to deliver satisfactory results at all. It may well be the case that in the beginning of the run the fitness function presents the evolutionary algorithm with a landscape that makes certain values for some parameters mandatory while at some later time the population searches in other regions of the search space where other values for the parameters are to be preferred. This motivates having the parameter settings vary with time. This way of setting the parameters is called dynamic parameter setting. In implementations, the number of generations or the number of fitness evaluations are used as measure for time. While dynamic parameter setting is quite common for some randomized search heuristics, it is seldom used in evolutionary algorithms. One example, however, that we already mentioned is the temperature T in Boltzmann selection. In evolutionary algorithms, if parameter values are to change during the run, in most cases a more complicated mechanism is used. In principle, any information available may be used to decide upon the new values of parameters. Typical examples of the kind of information used include the current population, the population’s distribution in the search space, the current fitness values, or measures based on a history of the population. A concrete example would be the number of generations since an offspring with better fitness than its parent has been produced. A common strategy is to increase the mutation probability if such an event has not occurred for a predefined number of generations. Generally, any method depending on any individual created and any random experiment performed during the complete run is permissible. This complex way of setting the parameters is called adaptive parameter settings.

We observe that these methods of setting the parameters form a hierarchy as depicted in Fig. 2.2. Adaptive parameter settings form the most flexible and largest class of methods for setting the parameters. Dynamic parameter settings are restricted methods that may only depend on the time but not on the many other available data. It may therefore be adequately described as a special case of adaptive parameter setting. In the same way, static parameter settings can be described as a special case of dynamic parameter settings, where the function that depends on time is constant. One may say that static parameter settings are a degenerate form of dynamic parameter settings.

Fig. 2.2

Hierarchy of parameter setting methods

One special way of adaptively setting the parameters deserves to be mentioned: self-adaptive parameter settings. Self-adaptation means that the parameter settings are evolved together with the population in some way. Typically, the parameter values are encoded as part of the individuals, formally increasing the size of the search space. We can describe this by having our new individuals live in S ×Q, where S is the original search space and Q is the set of possible parameter settings. Note that the fitness is still defined on S, and therefore the parameter settings have no direct influence on the fitness. In each generation, first variation operators are applied to the parameter settings. Then these offspring parameter values are used to parameterize the variation operators applied to points in S. Selection is done in the usual way. The idea is that good values for the parameters have the tendency to create good offspring. Since good offspring are typically selected, good values for the parameters indirectly have better chances to survive and be present in the next generation. This is the reason why the parameter values have to be subject to variation first. Otherwise we have random changes of parameter values that have no influence at all on the selection, and self-adaptation would probably not work that well.

2.3 Typical Evolutionary Algorithms

The ‘invention’ of evolutionary algorithms dates back to the 1960s. Different research groups suggested independently of each other quite similar randomized search heuristics all inspired by natural evolution. All had the idea to devise interesting and useful new algorithms by mimicking natural processes in algorithmic form. For us it is not difficult to identify these different algorithms as variants of the same algorithm family, as different kinds of evolutionary algorithms. And yet it was a long and sometimes difficult process for the researchers involved to recognize that others developed very similar ideas. This can still be seen today and is apparent in different names for certain kinds of evolutionary algorithms. One may consider having separate names for these different variants useful since it seems to allow us to describe rather complicated evolutionary algorithms in a very short and precise way. This, however, is actually not an accurate description of the status quo. Nobody really uses the algorithmic variants devised nearly 50 years ago. There are so many variations of all kinds of evolutionary algorithms around that it is practically impossible to infer the details of an evolutionary algorithm just from its name. Since the historic names are still around, we give a short description and describe typical variants. We conclude this section by giving precise definitions of an evolutionary algorithm that we will consider in great detail in the chapter on methods for the analysis of evolutionary algorithms (Chap. 5). We describe all algorithms in a way that make them suitable for maximization of a fitness function f. This agrees with the intuitive idea that fitness should be maximized. Clearly, minimization of − f is equivalent to maximization of f and thus considering only maximization is no restriction.

In the United States, Holland [50] devised genetic algorithms (GAs). Most often they are defined for the search space {0, 1} n ; the variation operator that is considered to be the most important one is crossover. Often k-point crossover with k ∈ {1, 2} is used with quite high probability, i.e., . Mutation, typically standard bit mutations with small mutation probability, is considered to be a kind of ‘background operator’ that merely generates enough diversity in the population to facilitate crossover. Selection for reproduction is usually done using fitness-proportional selection. In his original work Holland does not mention selection for replacement; in fact, (μ, μ) selection is applied.

In Germany, Schwefel [114] and Rechenberg [107] devised evolution strategies (ESs). Most often they are defined for the search space ; the most important variation operator is mutation. Often there is no crossover at all, and if crossover is to be used it is usually intermediate crossover. In the beginning there was no explicit discussion about selection for reproduction; uniform selection is used for this purpose. Selection for replacement is usually implemented either as comma- or plus-selection. For evolution strategies, non-static ways of setting the parameters have been in use almost from the very beginning. In particular, self-adaptive methods were used very early on.

In the United States, Fogel et al. [41] devised evolutionary programming (EP). Originally, the set of finite state automata was used as search space. This may be seen as the most ambitious approach. The complex structure of the search space led to an emphasis of mutation (of course, mutation operators suitable for mutating finite state automata had to be developed). Later developments led evolutionary programming increasingly close to evolution strategies.

Quite some time later, Koza [73, 74] and Koza et al. [75, 76] introduced genetic programming (GP). Genetic programming uses evolutionary algorithms that are modeled after genetic algorithms but use a different search space (often the set of all s-expressions) and aim at developing programs as individuals.

Considering the different streams that together led to the class of evolutionary algorithms, it comes as no surprise that there are many different variants of evolutionary algorithms. Some variants, however, are more common than others. We consider some of the more basic, well-known, and therefore in some sense important ones here.

One algorithm that is often singled out is the so-called simple GA. It is a genetic algorithm operating on the search space {0, 1} n , fitness-proportional selection for reproduction, 1-point crossover with crossover probability p c  ∈ [0. 5, 0. 9], standard bit mutations with mutation probability p m  ≤ 1 ∕ n, and (μ, μ) selection for replacement. Sometimes crossover is described as producing two offspring but this varies from author to author. The simple GA was first described by Goldberg [46] and later analyzed by Vose [125].

An evolutionary algorithm that we are going to analyze in considerable detail is a simple algorithm that we call (μ+λ) EA. It operates on the search space {0, 1} n , has population size μ, offspring population size λ, uniform selection for reproduction, no crossover, standard bit mutation with mutation probability p m  = 1 ∕ n, and (μ+λ)-selection for replacement. Since this algorithm is central for our analysis, we give a precise formal definition.

The (μ+λ) EA implements plus-selection with a slight preference for the offspring: if parents and offspring have equal fitness, the offspring is preferred. So, if we have three parents x 1, x 2, x 3 with fitness values f(x 1) = 2, f(x 2) = 7, f(x 3) = 4, and three offspring y 1, y 2, y 3 with fitness values f(y 1) = 4, f(y 2) = 6, f(y 3) = 1, then the ordering is x 2, y 2, y 1, x 3, x 1, y 3.

We know that evolutionary algorithms are particularly difficult to analyze since they are not designed with analysis in mind. Thus, it makes sense to start with particularly simple evolutionary algorithms. This way we can hope to have manageable objects of studies and develop tools for their analysis that turn out to be suitable for the analysis of more complex evolutionary algorithms, too. This motivates considering the (μ+λ) EA with minimum population size and offspring population size, i.e., μ = λ = 1. These settings lead to a simpler formal description of the (1 + 1) EA.

Note that we described these two evolutionary algorithms without any stopping criterion. Remember that this is the most common approach when analyzing evolutionary algorithms. We are interested in the optimization time T, i.e., for the (1 + 1) EA.

2.4 Other Simple Randomized Search Heuristics

Evolutionary algorithms are by no means the only kind of general randomized search heuristics. There is a plethora of different randomized search heuristics ranging from very simple (like pure random search) to quite complicated and sophisticated (like particle swarm optimization or ant colony optimization). Since our interest is in evolutionary algorithms, we will not discuss other randomized search heuristics in great detail. It makes sense, however, to compare the performance of evolutionary algorithms with that of other randomized search heuristics in order to get a better understanding of their specific strengths and weaknesses. We consider five of the simpler randomized search heuristics for this purpose. Another motivation for discussing these other randomized search heuristics is to develop an understanding of the way a borderline could be drawn separating evolutionary algorithms from other randomized search heuristics. Due to the flexibility of the algorithmic concept ‘evolutionary algorithm’, it is impossible to come to a clear and indisputable distinction. But we will be able to give reasons for calling some simple randomized search algorithm like the (1 + 1) EA an evolutionary algorithm while we consider quite similar algorithms to be of a different kind. Finally, structurally simpler randomized search heuristics can serve as a kind of stepping stone. Proving results about their behavior can provide valuable insights about how a proof for the more complex evolutionary algorithms can be obtained. We describe all randomized search heuristics for the maximization of some function .

Pure random search

Search starts with some x ∈ {0, 1} n chosen uniformly at random. In each step, another point y ∈ {0, 1} n is chosen uniformly at random that replaces x.

This kind of blind search may, of course, be described as an (1, 1) EA with standard bit mutation with mutation probability p m  = 1 ∕ 2. But this would be misleading. It differs in two important points from evolutionary algorithms: the search is not guided by the fitness values encountered, and the search has no locality whatsoever.

Pure random search is a very simple and almost always very bad search heuristic, of course. Its only advantage is that it is extremely easy to analyze. We may use it as a very weak competitor. If an evolutionary algorithm is not even able to clearly outperform pure random search, it is definitely not doing much good for the fitness function under consideration.

Random local search

Search starts with some x ∈ {0, 1} n chosen uniformly at random. In each step, another point y ∈ N(x) is chosen uniformly at random where N(x) denotes some neighborhood of x. Then, y replaces x if holds.

Often, the neighborhood is used. We call this the direct Hamming neighborhood. Sometimes, larger neighborhoods like are more useful. Almost always one has neighborhoods of at most polynomial size, i.e., .

Random local search with the direct Hamming neighborhood, i.e., the neighborhood N(x) consists of only the Hamming neighbors of x, could be described as (1 + 1) EA with 1-bit mutations instead of standard bit mutations. This difference, however, is crucial. Random local search can be trapped in local optima of the fitness landscape where all Hamming neighbors have smaller fitness values. Random local search cannot move anywhere from such a point. For the (1 + 1) EA, no such traps exist. It can reach any point in the search space by one mutation with positive (yet very small) probability. This ability to perform global search is typical for evolutionary algorithms. This is why we draw a borderline here.

Nevertheless, random local search and the (1 + 1) EA (as defined as Algorithm 2) are very similar. Often random local search is much easier to analyze. Since the performance of random local search and the (1 + 1) EA is often similar, analyzing random local search can be a helpful first step toward an analysis of the (1 + 1) EA.

Iterated local search

In iterated local search we carry out several local search runs subsequently. Each random local search is carried out as described above. Every time the search gets stuck in a local optimum it is restarted with some new starting point x chosen uniformly at random from {0, 1} n .

We note that iterated local search requires considering the complete neighborhood N(x) in order to decide whether the search got stuck. Clearly, this implies increased computation time for something that we consider one ‘round’ of the algorithm. However, this extension to random local search is so simple and yet such a significant improvement that it is worth mentioning. Moreover, such restarts may be added to any kind of random search heuristics; in particular they may be used in combination with evolutionary algorithms. There, however, it is more difficult to find an appropriate criterion for triggering a restart.

Metropolis algorithm

Search starts with some x ∈ {0, 1} n chosen uniformly at random. In each step another point y ∈ N(x) is chosen uniformly at random, where N(x) denotes some neighborhood of x. Then, y replaces x with probability .

As for random local search, the most common neighborhood consists just of the Hamming neighbors of x. The parameter is called temperature; it is fixed in advance and held constant during the complete run. The term equals 1 if holds. So, improvements in fitness are always accepted here. This coincides with random local search. But for f(y) < f(x), the two search heuristics differ. While such a move from x to y is never done with random local search, it may be done in the Metropolis algorithm. The probability, however, depends on the parameter T and the difference in fitness values f(x) − f(y). With increasing difference the probability for such a step decreases exponentially. The selection mechanism helps prevent getting stuck in local optima.

Simulated annealing

Simulated annealing is almost identical to the Metropolis algorithm, but the fixed temperature T is replaced by some function that is called annealing schedule and that depends on time, i.e., on the current generation.

Clearly, we may describe the Metropolis algorithm as simulated annealing with fixed temperature. Thus, simulated annealing is an example of an algorithm making use of a dynamic parameter-setting scheme, whereas the Metropolis algorithm is the same algorithm but utilizing a static parameter setting.

Usually, one uses strictly decreasing annealing schedules T. In fact, simulated annealing is inspired by the process of annealing in metallurgy. There, metal that is heated beyond its recrystallization temperature is cooled sufficiently slowly such that it is allowed to enter an energy-minimizing state. When using simulated annealing for finding points with large fitness values in the search space, the idea is the following. In the beginning, fitness values are quite bad, and it should be simple to escape from local optima. This is analogous to using a high temperature. Later on, the algorithm has probably found much more promising regions of the search space that should not be left easily. This is analogous to lower temperatures. Given a certain optimization problem, to find an appropriate annealing schedule is as crucial for the performance of simulated annealing as it is difficult to achieve.

2.5 Design of Evolutionary Algorithms

Our main focus when considering evolutionary algorithms is analysis. We aim at considering common evolutionary algorithms and finding out how they perform on different problems. When one wants to apply evolutionary algorithms, the perspective is necessarily different. In this case, one wants to design an evolutionary algorithm that is appropriate and efficient for a given problem class. We consider this situation here and discuss aspects that stem from a theoretical perspective and that should be taken into account.

We restricted our description of modules for evolutionary algorithms to the three search spaces {0, 1} n , , and S n . In principle, we could even restrict the discussion to the search space {0, 1} n . Evolutionary algorithms are (like all algorithms nowadays) implemented on computers using binary representations internally. Thus, any nonbinary data like real numbers has to be mapped to a binary representation at some level. One may therefor argue that we may as well ‘lift’ this mapping to the topmost level, perform it ourselves, and deal only with binary representations from now on. However, such reasoning does usually not take the complexity of the actual mappings involved into account. It is similar to arguing that a modern computer is in principle nothing more than a finite automata with a huge state space. While this is true in a formal sense, we know that it is much more appropriate to describe a modern computer as a Turing machine. Properties and complexity are better captured this way. Similarly, it makes more sense to consider evolutionary algorithms on different search spaces.

We are interested in evolutionary algorithms because they have proven to be very useful in many practical applications. Without practical applications there would be no point in doing theory. For some practical problems it is actually possible to find an appropriate formalization as function , , or where f is to be maximized. One particularly good example is the satisfiability problem SAT that is of practically importance in many applications and at the same time one of the most prominent problems in theoretical computer science. If we consider SAT for n variables it is easy for any concrete SAT instance to describe a corresponding function that yields the number of clauses satisfied by an assignment x ∈ {0, 1} n of the n variables. Similarly, for each instance of the traveling salesperson problem (TSP) with n cities, we can define a corresponding function that yields the length of the tour described by a permutation π for each permutation π ∈ S n . But here we encounter a first (easy to solve) problem. While evolutionary algorithms aim at maximizing a fitness function, the function f { TSP instance} needs to be minimized. As discussed above, this can easily be fixed by replacing f { TSP instance} by − f { TSP instance}. But this yields a fitness function with only negative function values, making, for example, the use of fitness proportional selection impossible. A different way of dealing with a minimization problem would be to replace the kind of selection employed, using selection for replacement where selection for reproduction is intended, and vice versa. But this would in some sense be against one of the main ideas of evolutionary algorithms. Evolutionary algorithms are attractive to practitioners since there is no need for modifications of the algorithm, since they are easy off-the-shelf solutions to a wide variety of different problems. Such ad hoc modifications of the algorithm become even more complicated if the fitness function is not defined over one of the standard search spaces. Adopting a novel kind of search space implies that new variation operators need to be defined. Then the design of new evolutionary algorithms becomes as complicated and as time consuming as the design and implementation of any new problem-specific search heuristic. Thus, it is advisable to follow a different route. In the following, we discuss such a solution to our problem that, from the point of view of computer science, is more structured than ad hoc modifications of the algorithm. Moreover, it has the additional charming property of having a corresponding mechanism in the natural paradigm.

Let us assume that we are dealing with an optimization problem that is modeled as either maximization or minimization of some function . Here, A may be an arbitrary set, B is some set that allows for some way of evaluating solutions. Thus, there needs to be at least a partial ordering defined on B. We want to solve this optimization problem by means of some evolutionary algorithm. In order to do so, we define two mappings and , where S is the search space of our evolutionary algorithm. The idea is to have the search space S equal to some standard search space so that we have an evolutionary algorithm operating on S ready to use. We define a fitness function f for our evolutionary algorithm by . This implies that in order to compute the fitness for some point s ∈ S, we first map s via h 1 to h 1(s) ∈ A, compute its value g(h 1(s)) ∈ B, and, by means of h 2, map this value to a fitness value .

Following the natural paradigm, the set S is often called phenotype space and A is called genotype space. Accordingly, h 1 is called genotype–phenotype mapping. The idea is that our ‘genetic’ algorithms operate on the genotypes, whereas in nature fitness (in the sense of survivability) is, of course, expressed on the level of phenotypes. So having some mapping from the genotypes to the phenotypes that is involved in determining the fitness of some individual is something that we may see in nature.

In principle, we are free to choose h 1 and h 2 any way we like. Obviously, we have to make sensible choices in order to arrive at an evolutionary algorithm that works well for our problem. Probably the most basic requirement is that h 2 is a function that needs to be maximized in order to find optimal solutions to g. Moreover, h 1 and h 2 need to be computable efficiently, evaluation via h 2 needs to have a good correspondence to evaluation via g, and h 1 needs to map to as much of A as possible. If we choose h 1 in a unfavorable way, it may happen that optimal solutions in A have no preimage in S and thus cannot be found be the evolutionary algorithm at all.

All this advice is basically trivial. In practice, however, it may be highly nontrivial to follow this advice. Nevertheless, we discuss even more guidelines that all aim at delivering a well-functioning evolutionary algorithm. These guidelines come with the advantage of being less trivial: they are useful advice that is substantial and could hardly be found with just a few minutes of thinking. While following them in practice may be difficult, it pays to know about them in order to avoid making mistakes that have been made many times before by others.

As we pointed out when discussing different variation operators, the main idea in evolutionary algorithms is to search for promising new search points quite close to the points of the current population. Since our variation operators work in genotype space S but fitness assessment is done in phenotype space A, it is desirable that small changes in genotype space correspond to small changes in phenotype space. If there is no such correspondence between changes in S and A, we have departed (at least implicitly, perhaps unknowingly) from the idea of evolutionary algorithms. It is possible that the evolutionary algorithm that we obtain still works—but from a fundamental point of view it should not. In order to make our ideas more precise, we need a way of measuring changes. This is done by means of some metric d. For the sake of completeness, we recall the definition of a metric.

Definition 2.1.

For a set M a mapping is called a metric on M if it has the following three properties:

1.

Positivity

2.

Symmetry

3.

Triangle inequality

We assume that some metric d A for A is known. This metric is a formal way to express domain knowledge that a user has. For candidate solutions to the practical problem , the user is expected to be able to describe their similarity. If our genotype–phenotype mapping h 1 happens to be injective, then we obtain a metric d S on S by means of h 1 via for all x, y ∈ S. If h 1 is not injective, we cannot define d S this way since this would violate the positivity constraint. In this case a metric d S that reflects d A has to be defined some other way. From now on we simply assume that some metric d S on S is defined, and we demand that for all x, x′, x′ ∈ S

holds. We call this property monotonicity. Clearly, monotonicity guarantees the connection between the magnitudes of changes that we desire.

Based on this metric d S , we can now formalize our requirements for variation operators. This helps not only to check whether our genotype–phenotype mapping h 1 and the metric d S are appropriate when applying some evolutionary algorithm to some practical optimization problem. It also enables us to discuss the appropriateness of newly designed variation operators in an objective way.

For our discussion we want to describe mutation and crossover as randomized functions that we define in the following way. Assume we have a mapping , where the image r(x) ∈ Y depends not only on x ∈ X but also on some random experiment, i.e., r(x) is a random variable. The random experiment is defined on some probability space (Ω, p). We define the mapping r as , where r(x, ω) = y if applying r to x together with ω ∈ Ω as result of a random experiment yields y. For x ∈ X and y ∈ Y , we let . We simplify our notation by omitting ω and assume that the probability space (Ω, p) is clear from the context.

For the sake of clarity, we discuss a simple example and consider 1-bit mutation as a randomized mapping . Since we flip exactly one bit and choose the position of this bit uniformly at random, (Ω, p) with and p(i) = 1 ∕ n for all i ∈ Ω is an appropriate probability space. For x ∈ {0, 1} n and i ∈ Ω we define , where we use the following notation. For b ∈ {0, 1} and let b j denote the concatenation of j times the letter b. In particular, let b 0 denote the empty word. Thus, holds. The operation applied on {0, 1} n stands for bitwise XOR, where for a, b ∈ {0, 1} XOR is defined via { XOR}(a, b) = a + b − 2ab. It is easy to see that this way m corresponds exactly to our understanding of 1-bit mutation.

Using this notation as we did in the example, we can proceed and define our requirements for variation operators. We begin with mutation operators that we describe as randomized mappings .

We want a mutation operator m to favor small changes. We express this by demanding that

holds. If x′ is closer to x than x′ is, than it should be more likely to obtain x′ as offspring of x then x′.

A second reasonable requirement is to have mutation operators not induce a search bias. We want the search to be guided by the fitness values which are taken into account by selection. This can be expressed via

in a formal and precise way.

For crossover, we can proceed in a similar way. Crossover is described as a randomized mapping . We consider crossover operating on two parents and producing exactly one offspring. It is not difficult to generalize this to crossover operators using more parents. We express the idea that an offspring should be similar to its parent by having

for crossover. The distance from the offspring to any of its parents is bounded above by the distance of the parents.

Clearly, we do not want crossover to induce a search bias. We express this by requiring that

holds. The offspring is required to be symmetric with respect to its parents in terms of distance.

It is important to understand that these requirements are not ‘true’ in the sense that following these rules necessarily leads to better evolutionary algorithms or that violating a requirement implies poorer performance. And yet they should be taken seriously since they have the benefit of formalizing our intuitive understanding of evolutionary computation. Following them helps us not to depart from the paradigm of evolutionary computation. When applied during the creative act of defining new variation operators, they guide us and help us by presenting us with a formalism that leads us to more objectively justified and, in any case, more conscious design decisions. Moreover, they facilitate the formal analysis of evolutionary algorithms that are designed respecting these guidelines.

2.6 Remarks

While the description of the evolutionary cycle gives an accurate picture of the structure of evolutionary algorithms, our actual list of modules for evolutionary algorithms is very short and contains only the most basic examples. This is due to our emphasis on the analysis of evolutionary algorithms. A much more comprehensive view is presented in [9] and, more recently, in [110]. With respect to the mutation operators for permutations, it is worth mentioning that jump and exchange, while quite general in nature, have been designed for the problem of sorting [132].

With respect to tournament selection we remarked that there is no need to have all members of the current population evaluated as is the case for all selection mechanisms. Poli [98] suggests an evolutionary algorithm that takes advantage of this fact and reduces the number of function evaluations if the tournament size is small and the number of generations is not too large. This is one concrete example where the explicit inclusion of function evaluations within the evolutionary cycle would contradict an evolutionary algorithm as suggested to improve efficiency.

The analytical framework we pursue is inspired by the classical analysis (and design) of efficient randomized algorithms. Excellent textbooks providing introduction and overview include [15, 88].

In practical applications, finding good parameter settings is crucial. Bartz-Beielstein [10] provides a framework for a systematic approach to this problem. The classification of mechanisms to set and control parameters during a run (presented in Fig. 2.2) is structured in a way that a hierarchy is formed. This makes self-adaptation a special case of adaptive parameter settings. Historically, self-adaptation has been described as an alternative to adaptive parameter settings (see [8, 38]). While this may be useful to make a point and propagate self-adaptive parameter settings as ‘natural’ for evolutionary algorithms, it is, logically speaking, misleading.

Local search is a very well known and popular randomized search heuristic by itself. It is not covered in this text in any depth. Interested readers may want to consult [2, 83] for a more appropriate exposition of local search. The Metropolis algorithm and simulated annealing are also two popular randomized search heuristics. Useful references include [1, 55, 63, 72, 82]. The design methodology for evolutionary algorithms based on a metric was first presented by Droste and Wiesmann [29].

References

1.

E. Aarts, J. Korst, Simulated Annealing and Boltzmann Machines: A Stochastic Approach to Combinatorial Optimization and Neural Computing (Wiley, New York, 1998)

2.

E. Aarts, J.K. Lenstra (eds.), Local Search in Combinatorial Optimization (Princeton University Press, Princeton, 2003)MATH

8.

T. Bäck, An overview of parameter control methods by self-adaptation in evolutionary algorithms. Fundam. Inform. 35(1–4), 51–66 (1998)MATH

9.

T. Bäck, D.B. Fogel, Z. Michalewicz (eds.), Handbook of Evolutionary Computation (IOP Publishing/Oxford University Press, Bristol/Oxford, 1997)MATH

10.

T. Bartz-Beielstein, Experimental Research in Evolutionary Computation: The New Experimentalism (Springer, Berlin, 2006)MATH

15.

T.H. Cormen, C.E. Leiserson, R.L. Rivest, C. Stein, Introduction to Algorithms, 2nd edn. (MIT, Cambridge, 2001)MATH

29.

S. Droste, D. Wiesmann, On the design of problem-specific evolutionary algorithms, in Advances in Evolutionary Computing, ed. by A. Ghosh, S. Tsutsui (Springer, Berlin, 2003), pp. 153–173CrossRef

38.

A. Eiben, R. Hinterding, Z. Michalewicz, Parameter control in evolutionary algorithms. IEEE Trans. Evol. Comput. 3(2), 124–141 (1999)CrossRef

41.

L.J. Fogel, A.J. Owens, M.J. Walsh, Artificial Intelligence Through Simulated Evolution (Wiley, New York, 1966)MATH

46.

D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning (Addison-Wesley, Reading, 1989)MATH

50.

J. Holland, Adaptation in Natural and Artificial Systems (University of Michigan Press, Ann Arbor, 1975)

55.

T. Jansen, Simulated annealing, in Theory of Randomized Search Heuristics, ed. by A. Auger, B. Doerr (World Scientific, Singapore, 2011), pp. 171–196CrossRef

63.

T. Jansen, I. Wegener, A comparison of simulated annealing with simple evolutionary algorithms on pseudo-Boolean functions of unitation. Theor. Comput. Sci. 386, 73–93 (2007)MathSciNetMATHCrossRef

72.

S. Kirkpatrick, C.D. Gelatt, M.P. Vecchi, Optimization by simulated annealing. Science 220(4598), 671–680 (1983)MathSciNetMATHCrossRef

73.

J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (MIT, Cambridge, 1992)MATH

74.

J.R. Koza, Genetic Programming II: Automatic Discovery of Resuable Programs (MIT, Cambridge, 1994)

75.

J.R. Koza, F.H. Bennett III, D. Andre, M.A. Keane, Genetic Programming III: Darwinian Invention and Problem Solving (Morgan Kaufmann, San Francisco, 1999)MATH

76.

J.R. Koza, M.A. Keane, M.J. Streeter, W. Mydlowec, J. Yu, G. Lanza, Genetic Programming IV: Routine Human-Competitive Machine Intelligence (Kluwer Academic, New York, 2003)MATH

82.

N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)CrossRef

83.

W. Michiels, E. Aarts, J. Korst, Theoretical Aspects of Local Search (Springer, Berlin, 2007)MATH

88.

R. Motwani, P. Raghavan, Randomized Algorithms (Cambridge University Press, Cambridge, 1995)MATH

98.

R. Poli, Tournament selection, iterated coupon-collection problem, and backward-chaining evolutionary algorithms, in Foundations of Genetic Algorithms (FOGA 2005), Aizu-Wakamatsu, ed. by A.H. Wright, M.D. Vose, K.A. De Jong, L.M. Schmitt. Lecture Notes in Computer Science, vol. 3469 (Springer, Berlin, 2005), pp. 132–155

107.

I. Rechenberg, Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der Evolution (Frommann-Holzboog, Stuttgart-Bad Cannstatt, 1973)

110.

G. Rozenberg, T. Bäck, J.N. Kok (eds.), Handbook of Natural Computing (Springer, Berlin, 2012)MATH

114.

H.-P. Schwefel, Evolution and Optimum Seeking (Wiley, New York, 1995)

125.

M.D. Vose, The Simple Genetic Algorithm: Foundations and Theory (MIT, Cambridge, 1999)MATH

132.

I. Wegener, J. Scharnow, K. Tinnefeld, The analysis of evolutionary algorithms on sorting and shortest paths problems. J. Math. Model. Algorithms 3, 349–366 (2004)MathSciNetMATHCrossRef

Thomas JansenNatural Computing SeriesAnalyzing Evolutionary Algorithms2013The Computer Science Perspective10.1007/978-3-642-17339-4_3© Springer-Verlag Berlin Heidelberg 2013

3. Theoretical Perspectives on Evolutionary Algorithms

Thomas Jansen^1

(1)

Department of Computer Science, University College Cork, Cork, Ireland

Abstract

There have been a number of different attempts at building a theory for evolutionary algorithms that delivers interesting and meaningful results. All these theories have in common that they tend to start with rather simple evolutionary algorithms since even those turn out to be particularly difficult to analyze. This chapter is devoted to the presentation of some of these approaches. The choice of which branches of evolutionary computation theory to cover is mostly governed by the influence on current research in this area.

There have been a number of different attempts at building a theory for evolutionary algorithms that delivers interesting and meaningful results. All these theories have in common that they tend to start with rather simple evolutionary algorithms since even those turn out to be particularly difficult to analyze. This chapter is devoted to the presentation of some of these approaches. The choice of which branches of evolutionary computation theory to cover is mostly governed by the influence on current research in this area.

The first two approaches that we discuss in some detail have in common that they use a precise and specific description of evolutionary algorithm runs as their basis. Let us assume that we start a theoretical approach by fixing some specific evolutionary algorithm as our object of study, e.g., the simple genetic algorithm. We set all its parameters to some fixed values so that we have some concrete population size μ, some concrete crossover probability p c , and some concrete mutation probability p m . Remember that using the simple GA implies that we employ fitness-proportional selection for reproduction, 1-point crossover, and standard bit mutations. In addition to these choices, we fix some concrete fitness function . Considering all these details as fixed during the course of our analysis, what else needs to be considered to have a precise and complete description of a run of this simple GA on this fitness function f? Clearly, only the sequence of populations needs to be taken into account. This motivates that we concentrate on the way the current population P t of our simple GA can be described.

The third approach is quite different. It is based in spirit on design and analysis of efficient algorithms. While appreciating the differences between problem-specific algorithms and general search heuristics the same basic notions for measuring their efficiency is used. Since this third approach is the one that we concentrate on in the remainder of this book we only give a very brief introduction in this chapter, which is restricted to brief overviews.

3.1 Approaches Based on Markov Chains

Markov chains are a classical tool for the formal, mathematical description of random processes. The description of a Markov chain consists of two main elements. One is a set Z containing all states that the Markov chain can be in. In the context of evolutionary algorithms we can restrict our attention to finite state sets Z and Markov chains that work in discrete time steps. In each step the Markov chain moves from its current to another state. The second main ingredient are the probabilities that govern this movement. The defining property of Markov chains is that the probability for moving from the current state i ∈ Z to another state j ∈ Z depends only on the current state and not on the history of states visited so far. Given the state space Z, all these transition probabilities, and some initial probability distribution over Z, we can compute precisely the probability for being in any state at any point of time.

Presenting evolutionary algorithms using Markov chains is motivated by the hope to benefit from the many results that are available about Markov chains. Moving with respect to the description from an algorithmic point of view to a more mathematical perspective requires us to give a different description of the ingredients of an evolutionary algorithm.

For any finite search space S we may choose an arbitrary ordering of its elements so that we have . Then we can describe a population by stating how many individuals equal to s 0 are in the population, how many individuals equal to s 1 are in the population, and so on up to . Instead of giving the numbers, we can divide these numbers of occurrences by the population size μ so that we get the fraction p i of the population equal to s i for all i. Using this description we have with for all and . From the point of view of computer science, this is a strange description to choose. For the population, something that has size μ (and is thus reasonably small), we choose a description of size , a description so large that for practical purposes it cannot be written down. This is obvious when one thinks of practical applications of evolutionary algorithms. If the search space S is so small that enumeration of S (and, equivalently, writing something down that has size ) is a reasonable option, then it makes no sense to apply evolutionary algorithms at all. Just enumerate the search space and use an optimal solution that cannot be missed. In spite of these objections, from a mathematical point of view this description is well-defined and reasonable. We continue to use it for the moment and do not care about practicability.

So, we care about , the set of all populations of size μ over the finite search space S. What is the size of Z? Clearly, | Z | is larger than just , since we may have multiple copies of points in the search space in the population. On the other hand, , since there is no ordering of the elements in the population. It is, however, easy to see the correct answer if you have the following picture of a population in mind. A population can be described as bins where μ balls are placed into these bins. We may draw some concrete example for and μ = 4 as , where we use as a symbol for an individual and the bins are symbolized using | as bounds of the bins. So, in our example, there is one individual in the bin for s 0, the bins for s 1, s 3, s 4, and s 6 are empty, the bin for s 2 contains two individuals and the bin for s 5 one individual. We see that a population is uniquely defined by a string of length , where the first and last character are fixed to | . Thus, by choosing μ out of the possible positions for the μ members of the population, we fix a population. We see that holds.

We want to describe a run of the simple GA. Assume that we know the current population P t . Is this sufficient information to know P t + 1? Obviously not, since the simple GA is a randomized algorithm the subsequent population P t + 1 is a random variable, and we cannot know its value in advance. However, we can know its probability distribution. Since the algorithm and all parameters (including the fitness function f) are fixed and we know the current population P t , we can—at least in principle—compute the complete probability distribution of the next generation P t + 1. Assume that we have in addition knowledge about the complete history of the run, i.e., we know . Does this help us any further in knowing something about P t + 1? Clearly, this is not the case. All information is carried in the current population, and its history has no relevance for the creation of the next generation. This property that the next step depends on the current state alone and not on the older history is known as the Markov property. We realize that the simple GA can be described as a Markov chain. Now we want to obtain a concrete Markov chain describing it.

We are concerned with a Markov chain that is a discrete random process living in a state space Z. This Markov chain can be completely described by giving a description of the distribution of the initial state P 0 and the probability for arriving at some state j from some state i (with i, j ∈ Z). For the simple GA these transition probabilities do not depend on the current time step t. Markov chains with this special property are called time homogeneous. Using the notation we developed, the Markov property is stated in the following way.

We use for the transition probabilities to simplify our notation.

Remember that we are dealing with {0, 1} n as search space, so that holds. For some population k ∈ Z and some individual y ∈ { 0, 1} n let m k (y) denote the number of individuals in k that are equal to y. Note that this does not introduce some unknown value. When k is given, it is trivial to determine m k (y) for any y ∈ { 0, 1} n . Clearly, holds for any k ∈ Z and any y ∈ { 0, 1} n . Moreover, we have for any population k ∈ Z.

The simple GA generates an offspring population of size μ by producing μ offspring independently and identically distributed. Let, for a population k ∈ Z and an individual y ∈ { 0, 1} n , p k (y) denote the probability that in one such step y is created as offspring based on the population k. We will determine the value of p k (y) later.

In order to simplify our notation we identify y ∈ { 0, 1} n with , the number that is represented by y in standard binary coding. This way we may write m k (y) for some .

Since the simple GA utilizes (μ, μ) selection, the transition probability p i, j is given by the probability to produce the μ offspring forming population j based on the population i. This yields the following equation.

Considering the binomial coefficients from left to right, starting with

we see that most terms are canceled out, and we obtain

as the equation for the transition probability p i, j .

We still have to determine the probability to create an offspring y ∈ {0, 1} n based on a population k ∈ Z, p k (y). Clearly, creating offspring starts with fitness-proportional selection. Let s k (x) denote the probability to select x ∈ { 0, 1} n from the population k ∈ Z. According to the definition of fitness-proportional selection,

holds. Selection for reproduction is followed by variation. In the case of the simple GA this is 1-point crossover with probability p c with subsequent standard bit mutations with mutation probability p m and, alternatively (with the remaining probability 1 − p c ), standard bit mutations alone, again with mutation probability p m . Let c(x, y, z) denote the probability to create offspring z from parents x and y. Let m(x, y) denote the probability to create offspring y from parent x. It is easy to see that

holds, since the bits, where parent x and offspring y differ, need to flip, whereas the other bits must not flip. For 1-point crossover, things are a little more involved. There are n − 1 potential crossover points, each selected with probability . For each crossover point c, the offspring z is created if the first parent x and the offspring z agree in the first c bits while the second parent y and z agree in the remaining n − c bits. Thus,

holds. Combining these results we obtain

and have completed the description of the simple GA as a Markov chain.

If one is mathematically inclined it may feel tempting to say that the simple GA is this Markov chain. But that would be an exaggeration. The simple GA is a random process that may be described in different forms. One way is its description as an algorithm, and a different way is its description as a Markov chain. In Sect. 3.2 we will discuss yet another equivalent description. But these are all merely different representations of the same underlying object that is not altered by its representations. It is due to personal preferences and educational background that mathematicians tend to identify it with a Markov chain. Computer scientists tend, for the very same reasons, to identify it with a randomized algorithm. There is no point in arguing that the simple GA is the one or the other. We only care for such different descriptions as they provide different points of views and help us in different ways to gain insight into the functioning of the simple GA.

So what have we gained by describing the simple GA as a Markov chain? In principle, it enables us to apply the rich and well-developed theory of Markov chains to the analysis of this evolutionary algorithm. Since we may describe other evolutionary algorithms in a similar way, we opened evolutionary algorithms to the methods known from and used for the analysis of Markov chains. But there are two problems that make the practical use of this step questionable. First, as we have seen in its derivation, the Markov chain even for this simple evolutionary algorithms is exceedingly complicated. While, in principle, we can write it down in a formal way and do exact computations for fixed values of the parameters (namely n, f, μ, p c , and p m ), there is little hope to do anything useful in the general form. The second issue arises from the nature of the majority of results in Markov chain theory. Most of these results are concerned with the long-term behavior, with stationary distributions in particular. This is often quite different from our interests that are concerned with the behavior in the first, at most polynomial, number of steps. Some Markov chains approach their stationary distributions quite quickly. Such Markov chains are called rapidly mixing. It is, however, currently an open question which combinations of evolutionary algorithms and fitness functions lead to such rapidly mixing Markov chains. Therefore, at least currently, describing evolutionary algorithms as Markov chains is of little practical use since the resulting Markov chains are too complicated to be analyzed.

We already noted that is the set of all populations of size μ. So the populations live in , the simplex. Moreover, they form a regular lattice in Λ and this lattice becomes increasingly dense with increasing population size μ. In the limit, for μ → ∞, the set of populations Z becomes Λ. This motivates us to study a simple GA with infinite population size. Using the Markov chain describing the simple GA with finite population size μ as a starting point, now μ → ∞ yields a well-defined process that we can study. It no longer corresponds to the evolutionary algorithm. Yet, it can be rewarding to study it.

First of all, we see that this infinite population model of an evolutionary algorithm is easier to analyze. Clearly, for the simple GA with finite population size, the expected next generation is a point in Λ. With μ → ∞ the process becomes deterministic and it becomes easier to follow it over time. Moreover, we obtain a very elegant description of the model as a generational function obtained as a combination of selection , described as mapping populations to probability distributions over populations (both points in Λ), and variation (or mixing) , also described as such a mapping. These mappings clearly show the tight connection between the infinite population model and the evolutionary algorithm that we are really interested in. Given some population P ∈ Λ, we can compute the probability distribution of the next generation as G(P). Thus, we may implement the simple GA by computing the probability distribution via the Markov chain and sampling the population according to this distribution. While this is obviously a much less efficient implementation then the usual one (described in Sect. 2.3), it hints at another reason why we should be interested in results on the infinite population model. In the infinite population case, the expected next generation of the simple GA becomes the next generation in a deterministic way. With a finite population size μ, the probability for deviations from this expected next generation decreases with increasing μ. Thus, for sufficiently large population sizes μ, the finite population will follow the path of the infinite population model in Λ quite closely with probability close to 1. The crucial question is, of course, how large we need μ to be. Unfortunately, this question is currently open for any real evolutionary algorithm. Thus, currently we can describe infinite population models as a research direction that comes with the promise of delivering very useful and interesting insights—but currently it still has to prove that it really can live up to this promise.

3.2 Schema Theory

Theory for genetic algorithms used to be dominated by an approach known as schema theory. Since {0, 1} n is the search space most often used in connection with genetic algorithms, it makes sense to present this theory in a form matching this search space. Moreover, we will develop it for the same evolutionary algorithm that we considered when discussing a Markov chain-based approach, thesimple genetic algorithm. Clearly, generalizations are possible.

A schema is a string that represents a subset of {0, 1} n in the following way. We say that some x ∈ { 0, 1} n is represented by a schema s ∈ { 0, 1, ∗ } n if holds for all positions . Thus, the letter is used as a kind of wildcard matching 0-bits and 1-bits. Since schemata represent sets of points in the search space we consider a schema to be a set in this sense and thus write x ∈ s if x is represented by s. Moreover, we may write to refer to the set of all individuals in the population P t matching the schema s. We see that schemata are a very specific way of characterizing subsets of the search space {0, 1} n . Some subsets can be described very easily by schemata; consider, for example, the set of all strings that start with a 0-bit and end with a 1-bit, i.e., . Other subsets, however, are difficult to describe this way, e.g., the set of all strings containing exactly two 1-bits. The reason for choosing schemata as a way to structure the search space becomes obvious when developing schema theorems.

For some schema , we count the number of strings in the current population P t matching this schema. We write m(s, P t ) to denote this number and have . Clearly, for a given population P t and a fixed schema s, m(s, P t ) is easy to calculate—it is only a matter of counting. What we are interested in is how m(s, P t ) changes with t. Since, clearly, m(s, P t + 1) is a random variable, we choose to consider its expected value . We compute this expectation based on the current population P t , so what we really compute is . Sometimes this is described as if m(s, P t + 1) was derived based on the current number m(s, P t ), which could be expressed by . This, however, may be incorrect. It is incorrect in those cases where knowledge of m(s, P t ) is insufficient to compute the expected value of m(s, P t + 1) without any additional assumption.

For a start, we consider the simple GA with , i.e., with fitness-proportional selection, only. Using our new notation we have that the probability to select some y ∈ s from the current population P t equals . Due to the (μ, μ)-selection employed, the subsequent population consists of μ individuals that are independently and identically distributed, and we obtain

where the last equation is obviously correct but seems to be poorly motivated at first sight. The motivation is the desire to obtain formulas that provide us with some intuitive understanding of what is going on. To this end we define

as the average fitness of population P t . Similarly, we define

as average fitness of the schema s in P t . Using this notation our very simple schema theorem becomes

If we introduce mutation and have p m  > 0, things change. On the one hand, individuals selected from P t matching s may get lost since they become mutated in a way that they do not match s anymore. On the other hand, individuals selected from P t not matching s may become mutated in a way that they do match s. Since this is rather complicated to describe exactly, we will for the moment just consider the loss in the number of individuals matching s introduced by mutation. This is justified by the fact that the mutation probability p m is usually rather small.

For x ∈ { 0, 1} n we denote the random result of standard bit mutation applied to x by { mut}(x). Clearly, depends on the number of positions in s that are different from . We call this number the order o(s) of the schema s and therefore have . With this notation we obtain

as improved schema theorem (taking into account fitness-proportional selection and mutation), since we need the o(s) positions different from in s not to be mutated in the offspring. This result is pessimistic in the sense that it does not take into account the possibility of constructing individuals matching s. This explains why we went from an equation to a lower bound.

We continue in this direction and now additionally take crossover into account. Let

be the defining length of the schema s with o(s) > 0. Again, we decide to ignore the possibility of schema construction and look for an upper bound on the probability to ‘fall out of a schema.’ For crossover, a second parent is selected with fitness-proportional selection. If the second parent matches the schema too, 1-point crossover is guaranteed to yield an offspring matching the schema. If the second parent does not match s, we can still be certain that the offspring matches the schema if the crossover point does not fall between the leftmost and rightmost position different from . Thus, in order not to match the parent we have to decide to use crossover (with probability p c ), select a second parent not matching the schema (with probability ), and select a crossover point between the outermost positions different from (with probability ). Putting all things together we obtain

(3.1)

as schema theorem for the simple GA.

While the result is obviously correct, it is quite strange. Since it is entirely pessimistic with respect to the variation operators, it seems to suggest that the best we can do is not to use any variation (have ) and rely on selection only. This clearly makes no sense at all and would be an overinterpretation. But neglecting all constructive aspects of the variation operators leads to serious doubts on the explanatory power of such schema theorems.

Setting aside these doubts for a moment, we reconsider the schema theorem for the simple GA we just derived. We see that the expected number of copies of schema s in the next generation increases with increasing average fitness , decreasing order o(s), and decreasing defining length l(s). We can therefore expect to see a clear increase in the number of copies of fit, short schemata of low order. Such schemata are calledbuilding blocks, and thebuilding block hypothesis states that genetic algorithms function by assembling such building blocks and constructing solutions out of these building blocks. It leads to the idea that genetic algorithms are particularly well suited for fitness functions where global optima can be constructed that way. While intuitively appealing, this hypothesis clearly is much too vague to be considered a theory. Moreover, since it stems from a schema theorem that only takes into account destructive effects of variation operators and neglects schema construction, its relation to the simple genetic algorithm that we consider is rather weak.

This main weakness of the schema theorem as derived up to this point, not taking into account the probability of constructing individuals fitting to a schema by means of variation, however, can be remedied. This should not come as a surprise. We saw in Sect. 3.1 when discussing modeling evolutionary algorithms as Markov chains that it is not only possible to give a precise and exact description of an evolutionary algorithm as a Markov chain, it was not even particularly difficult to do so, just somewhat cumbersome. Doing the very same thing in another notation is clearly a feasible task. Here, however, we definitely need to know more than just m(s, P t ) in order to be able to say something about .

We reconsider our results from the section on the modeling using Markov chains (Sect. 3.1) and remember that we derived an exact formula for the probability to create an offspring y ∈ { 0, 1} n given that the actual population is k ∈ Z, p k (y). Thus we obtain

(3.2)

as exact schema theorem. Following the ideas of the schema theorem as derived above, this can be represented in an equivalent form as

(3.3)

where f { effective} is called the effective fitness of schema s in the population P t . Clearly, f { effective} needs to be derived in a way that as given in Eq. (3.3) coincides with as given in Eq. (3.2). We will not go into the details as this is just a tedious and rather boring exercise that does not yield any new insights.

We observe that the exact schema theorem as given in Eq. (3.2) or Eq. (3.3) cannot be changed to without being in a state of notational sin: it is in general impossible to derive without the exact knowledge of P t . Knowing only m(s, P t ) is in general insufficient. This is different from the situation with the ‘classical’ schema theorem as given in Eq. (3.1). There we do not need exact knowledge about P t ; knowing m(s, P t ), , and is sufficient. Since here we need to know P t and compute a value for , it is in general not possible to iterate this for more than one generation. In principle, one could compute information for sufficiently many schemata so that the complete probability distribution of P t + 1 can be computed. This is possible since exact schema theorems are in some sense equivalent to the Markov chain approach described in Sect. 3.1.

This equivalence between exact schema theorems and Markov chains makes it difficult to see in which way exact schema theorems can be useful. In some sense they provide exactly the same information in a much more complicated form. It is conceivable that there may be occasions when one is really interested in specific aspects that happen to be easily expressible as schemata. Then schema theory may be a convenient notation. In general, however, it is not a way that is likely to provide us with useful insights that cannot be obtained easier in other ways.

3.3 Run Time Analysis

What the Markov chain approach (Sect. 3.1) and schema theory (Sect. 3.2) have in common is that they consider evolutionary algorithms as random processes. They aim at giving a description of these random processes such that they can be analyzed as other random processes (that one may find in natural as well as in application settings). From the perspective of computer science, this is a rather strange way of dealing with evolutionary algorithms. It is much more natural and almost obvious that evolutionary algorithms should be considered as algorithms.

When computer scientists deal with algorithms that are applied as optimization tools they are mainly concerned with two aspects. Is the algorithm correct, i.e., does it always find an optimum with probability converging to 1 with increasing run time? And if this is the case, how long does it take? The second question referring to the run time may be concerned with the worst-case behavior or the average-case behavior. For randomized algorithms most often the expected run time is analyzed.

We adopt this perspective and consider the ‘run time’ of evolutionary algorithms. If we want to consider run time in the classical sense we need to decide on a stopping criterion. Clearly, the question whether the evolutionary algorithm under consideration actually is optimizing the objective function at hand, i.e., finds an optimum almost surely, depends heavily on the choice of the stopping criterion. Since stopping criteria are a difficult subject on their own, this complicates the analysis considerably. We can avoid this complication completely by neglecting the choice of a stopping criterion and let the algorithm (in a formal sense) run forever. Instead of considering the run time we now focus on the optimization time, i.e., the time needed to find an optimal solution. Note that we cannot reasonably ask the evolutionary algorithm to recognize that a global optimum is found. This task, if solved at all, falls into the responsibility of the module realizing the stopping criterion—which we just removed. We should be aware of the fact that this simplifies the task of optimization significantly. If we think of a classical optimization method like branch and bound algorithms, it may well be the case that such an optimization algorithm finds an optimal solution rather quickly and after that needs very long to actually prove that it is optimal. Our perspective, however, is a perfectly reasonable way to compare different evolutionary algorithms or, more generally, different randomized search heuristics. We will detail more ways that our perspective differs from the classical optimization perspective in the next chapter, where we discuss general limitations.

From the area of efficient algorithms and complexity theory we adopt the approach of an asymptotic analysis of the run time for growing input length. We use a logarithmic measure of the size of the search space as substitute for the length of the input since evolutionary algorithms do not really have an input. For the search space {0, 1} n this is the length of the bit strings n. For the search space of all permutations of n elements S n this is the number of elements n. If we considered the search space  this would be the dimension n. Thus, we are less interested in the concrete optimization time of an evolutionary algorithm for a concrete fitness function and much more interested how this optimization time scales with growing n. Clearly, this requires the fitness function f to be defined for all values of n, or at least infinitely many.

A second important aspect that we adopt from the area of efficient algorithms and complexity theory is the use of Landau notation to describe the optimization time of evolutionary algorithms (see Appendix A.1 for a definition). This implies that we agree not to perform an analysis on all tiny and tedious details and concentrate instead on the most important aspects. It simplifies the task of analyzing evolutionary algorithms in a way that it becomes feasible. It helps by drawing a clear picture showing the important aspects while hiding the irrelevant details. Making simplifications and neglecting details ultimately means making errors. So we have to be very careful if we want to present reliable proven results in the end. We introduce errors in a very careful and rigorous way. When making estimations we always make clear if we prove upper or lower bounds on the truth. We never make any unproven assumptions or any simplifications where we cannot control the effect introduced by it.

When analyzing the ‘run time’ of algorithms it is almost never the case that actual ‘wall clock time’ is considered. Usually one concentrates on the number of basic steps some model of computation makes and uses this number as measure of time. When analyzing randomized search heuristics like evolutionary algorithms, one considers almost always even more abstract notions of time. Similar to the approach taken in black-box complexity, a very common measure is the number of times the fitness function is evaluated. This approach is usually justified by mentioning that evolutionary algorithms tend to be algorithmically simple so that each step can be carried out quickly. Often this makes function evaluations the most costly operation. Sometimes even coarser measures such as the number of generations are used. While this abstraction from the actual implementation is useful, one has to be careful that the measure of time that is analyzed is actually a good measure for the real run time of the evolutionary algorithm under consideration.

Our analysis will start with considering very simple evolutionary algorithms on very simple example functions. This will enable us to develop and describe methods for analyzing evolutionary algorithms that will prove to be valuable and useful for much more complex evolutionary algorithms and problems. These methods are developed and presented in Chap. 5. Before we go there, however, we discuss the exact optimization scenario that we consider and general limitations for the performance of any optimizer working within this scenario.

3.4 Remarks

Many researchers have applied the theory of Markov chains to study evolutionary algorithms; one of the earliest noteworthy approaches is due to Nix and Vose [95]. Here we name only Michael Vose, who authored a beautifully written formal treatment of genetic algorithms [125]. A different noteworthy approach is to apply to evolutionary algorithms the theory of rapidly mixing Markov chains [116]. There are concrete examples [124], but this kind of theory has never proved to be very successful or influential. It provides, however, useful hints and should not be overlooked. Vose’s approach considers evolutionary algorithms with populations of infinite size. This raises the question of what size a finite population should have so that the infinite population model is a good approximation. Rabani, Rabinovich, and Sinclair provide an answer for crossover-based systems [106]. The wonderfully elegant computation of the number of different populations of size μ can be found in Feller’s [39] beautiful and highly recommended book on probability theory.

Schema theory was initiated by Holland [50] and made popular by Goldberg [46], who also tried to make this kind of theory more practical by expressing his understanding as the building block hypothesis. Exact schema theory was developed much later, and important contributions have been made by Stephens and Waelbroeck [117] as well as Poli with different coauthors [97, 99–101]. Poli was also pioneering in extending exact schema theory to the field of genetic programming, where the search space is more complex and usually infinite in size, the set of all labeled trees is a typical example. Individuals in these search spaces often represent programs, and it is easy to see that the formulation of schema theorems becomes more complex here. For an overview as well as a comprehensive treatment the book by Langdon and Poli [77] can be recommended. Readers interested in genetic programming in a more practical sense are referred to the Field Guide to Genetic Programming [102]. It is worth mentioning that both schema theory and in particular Vose’s Markov chain approach concentrate on the simple genetic algorithm described by Goldberg [46].

Analyzing the optimization time of evolutionary algorithms can be traced back at least to Mühlenbein [90]. Since Chap. 5 is devoted to this topic we only point to helpful and informative overview papers by Wegener and others [11, 127–129] and refer the interested reader to Chap. 5. Almost all publications in this area use the number of times the fitness functions is evaluated as their measure of time. Jansen and Zarges [68] discuss limits of this approach.

In this chapter we covered only three approaches to a theory of evolutionary algorithms, namely approaches directly based on Markov chain analysis, schema theory, and run time analysis. This selection is governed by the visibility and influence that different theory approaches currently have. It is by no means complete, and other approaches definitely can be mentioned. One example is the work by Mühlenbein and others [89, 91], where population genetics is used as a starting point for the analysis. Another noteworthy example is the application of methods from statistical mechanics. The idea is to use macroscopic statistics to give a summary of an evolutionary algorithm’s state and keep track of these statistical values. In cases where the statistics are insufficient for precise calculations typically the assumption of the population having maximum entropy is made. An introduction including pointers to more references is given by Prügel-Bennett and Rogers [105] and Shapiro [115].

What we have not discussed at all is the topic of ‘global convergence’, i.e., the question whether an evolutionary algorithms finds a global optimum to an arbitrary problem with probability converging to 1 as the number of generations goes to infinity. While it is sometimes considered to be important that an optimization tool can be proven to be ‘correct’, in this sense this question is of no real practical relevance. We do not care about optimization in general but in practice. Optimization tools are only useful if they optimize in a polynomial number of steps. This, however, cannot be achieved for all problems anyway, as we will see in the next chapter. Moreover, as we deal with finite search spaces ‘global convergence’ is not an issue at all. Clearly, it suffices to stop the optimization method after some time and enumerate the complete search space afterwards. Obviously, this turns any optimization method into one that guarantees ‘global convergence.’ Readers interested in what type of evolutionary algorithms are ‘correct’ optimizers in this sense are referred to [3].

References

3.

A. Agapie, Genetic algorithms: minimal conditions for convergence, in Third European Conference on Artificial Evolution, Nimes. Lecture Notes in Computer Science, vol. 1363 (Springer, Berlin, 1997), pp. 183–206

11.

H.-G. Beyer, H.-P. Schwefel, I. Wegener, How to analyse evolutionary algorithms. Theor. Comput. Sci. 287, 101–130 (2002)MathSciNetMATHCrossRef

39.

W. Feller, An Introduction to Probability Theory and Its Applications. Volume I (Wiley, New York, 1957)

46.

D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning (Addison-Wesley, Reading, 1989)MATH

50.

J. Holland, Adaptation in Natural and Artificial Systems (University of Michigan Press, Ann Arbor, 1975)

68.

T. Jansen, C. Zarges, Analysis of evolutionary algorithms: from computational complexity analysis to algorithm engineering, in 11th ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA 2011), Schwarzenberg, ed. by H.-G. Beyer, W.B. Langdon (ACM, New York, 2011), pp. 1–14

77.

W.B. Langdon, R. Poli, Foundations of Genetic Programming (Springer, Berlin, 2002)MATHCrossRef

89.

H. Mühlenbein, Parallel genetic algorithms, population genetics, and combinatorial optimization, in Parallelism, Learning, Evolution, Neubiberg, ed. by J. Becker, I. Eisele, F. Mündemann. Lecture Notes in Computer Science, vol. 565 (Springer, Berlin, 1991), pp. 398–406

90.

H. Mühlenbein, How genetic algorithms really work: mutation and hillclimbing, in Proceedings of the 2nd International Conference on Parallel Problem Solving from Nature (PPSN II), Brussels (Elsevier, Amsterdam, 1992), pp. 15–26

91.

H. Mühlenbein, D. Schlierkamp-Voosen, The science of breeding and its application to the breeder genetic algorithm (BGA). Evol. Comput. 1(4), 335–360 (1993)CrossRef

95.

A.E. Nix, M.D. Vose, Modelling genetic algorithms with Markov chains. Ann. Math. Artif. Intell. 5(1), 77–88 (1992)MathSciNetCrossRef

97.

R. Poli, Exact schema theory for genetic programming and variable-length genetic algorithms with one-point crossover. Genet. Program. Evolvable Mach. 2(2), 123–163 (2001)MathSciNetMATHCrossRef

99.

R. Poli, N.F. McPhee, General schema theory for genetic programming with subtree-swapping crossover: part I. Evol. Comput. 11(1), 53–66 (2003)CrossRef

101.

R. Poli, N.F. McPhee, J.E. Rowe, Exact schema theory and Markov chain models for genetic programming and variable-length genetic algorithms with homologous crossover. Genet. Program. Evolvable Mach. 5(1), 31–70 (2004)CrossRef

102.

R. Poli, W.B. Langdon, N.F. McPhee, A Field Guide to Genetic Programming (Lulu, 2008). http://www.gp-field-guide.org.uk

105.

A. Prügel-Bennett, A. Rogers, Modelling genetic algorithms dynamics, in Theoretical Aspects of Evolutionary Computing, ed. by L. Kallel, B. Naudts, A. Rogers (Springer, Berlin, 2001), pp. 59–85

106.

Y. Rabani, Y. Rabinovich, A. Sinclair, A computational view of population genetics. Random Struct. Algorithms 12(4), 313–334 (1998)MathSciNetMATHCrossRef

115.

J.L. Shapiro, Statistical mechanics theory of genetic algorithms, in Theoretical Aspects of Evolutionary Computing, ed. by L. Kallel, B. Naudts, A. Rogers (Springer, Berlin, 2001), pp. 87–108

116.

A. Sinclair, Algorithms for Random Generation and Counting: A Markov Chain Approach (Birkhäuser, Boston, 1996)

117.

C.R. Stephens, H. Waelbroeck, Schemata evolution and building blocks. Evol. Comput. 7(2), 109–125 (1999)CrossRef

124.

P.M.B. Vitanyi, A discipline of evolutionary programming. Theor. Comput. Sci. 241(1–2), 3–23 (2000)MathSciNetMATHCrossRef

125.

M.D. Vose, The Simple Genetic Algorithm: Foundations and Theory (MIT, Cambridge, 1999)MATH

127.

I. Wegener, Theoretical aspects of evolutionary algorithms, in International Colloquium on Automata, Languages and Programming (ICALP 2001), Heraklion (Springer, Berlin, 2001), pp. 64–78

129.

I. Wegener, Towards a theory of randomized search heuristics, in International Symposium on Mathematical Foundations of Computer Science (MFCS 2003), Bratislava (Springer, Berlin, 2003), pp. 125–141

Thomas JansenNatural Computing SeriesAnalyzing Evolutionary Algorithms2013The Computer Science Perspective10.1007/978-3-642-17339-4_4© Springer-Verlag Berlin Heidelberg 2013

4. General Limits in Black-Box Optimization

Thomas Jansen^1

(1)

Department of Computer Science, University College Cork, Cork, Ireland

Abstract

We already observed that evolutionary algorithms are usually thought of as general problem solvers. This implies that they are designed according to a general idea of how search should be implemented. In the case of evolutionary algorithms this idea stems from an understanding of natural evolution. More importantly, they are not designed in a way tailored toward a specific kind of optimization problem. We call this way of doing optimization while being in this sense oblivious to the concrete problem instance at hand black-box optimization. In this chapter we make precise what we mean when talking of black-box optimization. This allows us to recognize general limitations on the performance of any algorithm tackling the problem of black-box optimization. On one hand this helps us to get a clearer picture of what we can and cannot expect from evolutionary algorithms. On the other hand it even yields practically useful lower bounds on the performance of evolutionary algorithms. This is surprising good news since we consider a very general framework without concrete references to evolutionary algorithms that covers an enormous array of optimization algorithms.

We already observed that evolutionary algorithms are usually thought of as general problem solvers. This implies that they are designed according to a general idea of how search should be implemented. In the case of evolutionary algorithms this idea stems from an understanding of natural evolution. More importantly, they are not designed in a way tailored toward a specific kind of optimization problem. We call this way of doing optimization while being in this sense oblivious to the concrete problem instance at hand black-box optimization. In this chapter we make precise what we mean when talking of black-box optimization. This allows us to recognize general limitations on the performance of any algorithm tackling the problem of black-box optimization. On one hand this helps us to get a clearer picture of what we can and cannot expect from evolutionary algorithms. On the other hand it even yields practically useful lower bounds on the performance of evolutionary algorithms. This is surprising good news since we consider a very general framework without concrete references to evolutionary algorithms that covers an enormous array of optimization algorithms.

We consider the task of optimizing an objective function (fitness function in our context) , where S is the search space and R is the value space we map to. We assume without loss of generality that we want to maximize and that there is at least a partial ordering on R. For the purposes of this chapter we can safely assume that holds.

One important assumption that we make is that the search space S is finite. This is a strong assumption—in particular, it makes optimization of f trivial in the sense that simply enumerating the complete search space S is (in theory) a feasible option. On the other hand, the assumption is in a formal sense always fulfilled if we intend to solve the task of optimization by means of a (digital) computer. Since its memory is finite, it cannot really handle infinite search spaces if we insist that there is no restriction with respect to the position of global optima and optimization means locating one exactly. Clearly, the number of memory configurations of any reasonable computer is so enormously large that it cannot be enumerated in any way—even though it is finite in a formal sense it is practically infinite. This holds for the search spaces we will be dealing with, too. They will be way too large so that complete enumeration is absolutely infeasible in practice. We will assume that we can spend at most poly-logarithmic time to find an optimum of the objective function f.

One immediate consequence of our assumption that the search space S is finite is that it is safe to assume that the value space R is also finite. We can restrict ourselves to the image of the objective function f, , and consider the function instead of f without any significant change.

If the search space S and the image set R are both finite and both known, the set of all possible functions R S is also finite and also known. Thus, we can assume that the designer of an optimization algorithm knows that the objective function f is some function f ∈ R S . Since we care about general limits we are interested in finding out what optimal optimization algorithms can achieve. Clearly, we have to assume that the designer of an optimization algorithm does not know the concrete objective function . Knowing the concrete objective function f means knowing everything about this function, in particular knowing the optima. Thus, an optimal algorithm for optimizing f that knows f can simply output an optimal solution in the very first step and then stop. So, in black-box optimization, we have to assume that the concrete objective function f is not known to the designer of an optimization algorithm and thus to the algorithm. Otherwise talking about optimal algorithms becomes pointless.

An extreme alternative would be to assume that the designer of the algorithm (and thus the algorithm) knows nothing about the concrete objective function f beyond the fact that it is a function . We will deal with this extreme case later. But it is very important to note that there is very fruitful middle-ground. We can assume that the algorithm designer knows that the objective function that the algorithm will have to deal with is some function . Clearly, for the design of an optimal optimization algorithm, the algorithm designer will make full use of this knowledge. Anything that is useful when optimizing functions from  can be assumed to be used. This way we can incorporate problem-specific knowledge that one may have and use it in order to tune an evolutionary algorithm to a specific problem class. Depending on this specific class optimal algorithms may be very much faster on than optimal algorithms for arbitrary functions f ∈ R S can possibly be.

So, we consider the following scenario. For known finite sets S and R there is a known class of functions . Optimization algorithms for this class of functions have to be able to find an optimum for any concrete objective function  in finite time. During the course of optimization they can gather additional knowledge about the concrete objective function f by sampling arbitrary s ∈ S and considering the function value f(s). In a formal sense, we assume that there is an oracle that responds to a request s ∈ S by providing f(s) ∈ R. There is no other way of accessing the objective function f for the optimization algorithm. We assume that the task of optimization is completed if an s ∈ S with is sampled. In particular, we do not require that the optimization algorithm knows that it just sampled an optimum of f and acts accordingly. This is very much different from the situation in classical optimization. It matches the common application of evolutionary algorithms where usually a best point in the search space is presented at the end as solution but it is not clear if an optimal solution has actually been found.

Since in this chapter we are interested in general limitations in the black-box optimization scenario that we just described, we do not want to restrict the possible optimization algorithms any further. Thus, we do not specify the exact model of computation and how run time can be measured. Instead we count the number of f-evaluations (or more general function evaluations), the number of search points s ∈ S sampled, as measure of the optimization time. Since optimization means that an optimum is sampled, the optimization time is bounded below by 1. If we only care about optimal optimization algorithms we can assume that no point in S is sampled twice since resampling of points can easily be avoided using a dictionary where the algorithm stores all points sampled together with their function values. Counting only the number of f-evaluations implies that the use of such a dictionary is free and does not add to the optimization time. Thus, the number of function evaluations is bounded above by . In general, however, we do not make this assumption that black-box algorithms avoid resampling completely. In fact, standard evolutionary algorithms resample quite often. In this abstract black-box optimization scenario, however, optimal algorithms do not.

In the following, we will pursue this scenario of black-box optimization in two different ways. First, we will discuss an immediate consequence of considering the extreme case which leads to a result that has become famous under the name ‘no free lunch.’ We discuss this result and put its meaning in appropriate perspective by pointing out severe limitations of its applicability on one hand, and show that nevertheless there is truth in it that can be made precise in an illustrative way on the other hand. Second, we concentrate on lower bounds on the performance of optimal optimization algorithms for concrete sets . We will be able to prove lower bounds that will actually turn out to be useful when assessing the performance of concrete evolutionary algorithms on concrete fitness functions. In order to do that we describe a general way of connecting the performance of evolutionary algorithms on concrete fitness functions to the performance of black-box algorithms on classes of fitness functions in a meaningful way. Before we start we give precise definitions for black-box algorithms and their performance.

Definition 4.1 (Black-Box Algorithm).

Let S and R be finite sets and . An algorithm A is called a black-box algorithm for  if for all the algorithm samples some s ∈ S with within a finite number of function evaluations.

Definition 4.2 (Optimization Time of a Black-Box Algorithm).

Let S and R be finite sets, , A a black-box algorithm for . For we denote by T A, f the number of function evaluations A makes on f until some s ∈ S with is sampled. We call T A, f the optimization time of A on f.

Before we start to investigate what black-box algorithms can and cannot achieve, we equip ourselves with a precise and convenient way of describing such an algorithm. We assume the search space S and the value space R to be fixed. Our starting points are deterministic black-box algorithms since they are easier to handle. Clearly, we are not willing to restrict our attention to deterministic black-box algorithms. Most algorithms of interest are randomized, including evolutionary algorithms. However, we will find a clever way of giving a precise and easy to handle description of randomized black-box algorithms that builds on the description of deterministic black-box algorithms.

Black-box algorithms do not have any inputs. They have to find an optimum of an unknown objective function  that comes from some known subset of all functions, i.e., . Without inputs what a deterministic black-box algorithm does before it samples the first point s 1 ∈ S and learns about its function value f(s 1) is determined by the algorithm. In principle, since we do not impose any restriction on the computation model, a deterministic black-box algorithm may solve any problem and perform any computation before sampling this first point s 1 ∈ S. Since we only count the number of function evaluations, we do not care what it does before sampling s 1. We care about s 1 and can be certain that this point will be the same each time the algorithm is run since the algorithm is deterministic. After sampling s 1 and learning about its function value f(s 1) the algorithm can again perform any computation. This computation, however, may now depend on f(s 1). After the end of these computations the algorithm samples some point s 2 ∈ S. Since the choice of s 2 may depend on f(s 1), it is more accurate to write . Obviously, there can be up to  different choices of s 2, one  for each of the  possible values of f(s 1). Since we assume that we want to maximize, there is no need to sample another point if holds. Thus we can assume that there are at most  different choices of s 2.

We combine all these ingredients in a compact, accessible, and precise way by describing the deterministic black-box algorithm as a tree. For each point s sampled we introduce a node that we label with the point s ∈ S. The root of the tree represents the first point sampled and is labeled with s 1. For each of the non-maximal function values we attach a node as child to the root and label the edge to this child with the function value r. The child is labeled with the second point , where f(s 1) = r holds. If different function values  lead to the same second point s 2 (so holds), we merge the children and label the edge to this one child with . We continue the construction of this tree in the same way. This way we represent a deterministic black-box algorithm in a unique way by such a tree. An example of such a representation for the search space S = { a, b, c} and the value space R = { 0, 1, 2} can be found in Fig. 4.1. From now on we will identify a deterministic black-box algorithm with its tree representation.

Fig. 4.1

Example of a deterministic black-box algorithm for with S = { a, b, c} and R = { 0, 1, 2}

If we restrict our interest to deterministic black-box algorithms that are either optimal or at least aim at being optimal, we can assume that on each path from the root to one of the leaves each point s ∈ S appears at most once as label of a node on this path. Otherwise this point is resampled, and we already discussed that optimal black-box algorithms do not resample points. We observe that under this assumption there is only a finite number of different deterministic black-box algorithms. The depth of the tree is restricted by , the number of children for each node is restricted by , thus the size of the tree is finite. And since the number of different labels is also restricted, for the nodes and for the edges, there is only a finite number of different trees.

General black-box algorithms may in addition make use of random choices. So, differently from deterministic black-box algorithms, for a randomized black-box algorithm there may be no unique point s 1 ∈ S that is always the first point to be sampled. The choice of the first point to be sampled may depend on the outcome of a random experiment. We can imagine that we change any randomized black-box algorithm in the following way without altering the way the randomized black-box algorithm operates on an arbitrary objective function . The algorithm makes any random choice that it may want to make during the course of its run at the beginning and stores the random outcomes in some data structure. After that it operates deterministically. Any time the original randomized algorithm makes a random choice our modified version looks up the outcome of this experiment it did at the beginning in its data structure. From outside, the original randomized algorithm and our modified version cannot be distinguished. What we recognize now is that in our situation any randomized black-box algorithm can be described as making a (possibly very complex) random choice in the beginning and after that operating deterministically. Thus, in a formal sense, a randomized black-box algorithm can be described as a probability distribution over deterministic black-box algorithms. This will turn out to be useful when proving results on randomized black-box algorithms.

So far we have restricted ourselves to exact optimization. One may be concerned that this means that our modeling cannot be applied in cases where one is not interested in exact optimization but is instead content with approximation or some other optimization goal. But this concern can be met. In most cases it is not difficult to change the optimization goal from exact optimization to something else. Here we make this in a more formal way in order to achieve stronger results. To this end we concentrate on deterministic black-box algorithms for a bit longer. We have a representation of such an algorithm as a tree. If we fix an arbitrary objective function  this defines a single path from the root to one leaf in this tree. We call this path  the trace of f in A. Clearly, the length l of this path depends on the black-box algorithm A and the objective function f. Later it will be convenient to assume that all traces have length , the maximal possible length. We can assume this by letting an algorithm that intends to stop sample all points not yet sampled in an arbitrary ordering. If we do not take these additionally sampled points into account when assessing the algorithm’s performance, this change is not essential. So we can consider traces for arbitrary black-box algorithms A and arbitrary objective functions f. Given such a trace T A (f) we consider its projection on function values, . If we consider optimization goals like exact optimization or approximation, we can decide on the number of steps the algorithm A needed on f in order to achieve this goal based on V A (f). We do not need to take T A (f) into account since this decision can be made based on function values without considering the points in the search space the algorithm sampled. We restrict our attention to optimization goals where this is the case. Thus, for us an optimization goal is a mapping  that should be either maximized or minimized. We do not restrict these formal optimization goals in any way, thereby introducing a great deal of flexibility. However, it is not difficult to describe common and simple optimization goals this way. For example, exact optimization could be expressed as

The same formalism can be applied to randomized black-box algorithms. There we obtain one trace (and thus one vector of function values) for each possible outcome of the random choice the algorithm makes. Thus we obtain a probability distribution over traces and vectors of function values and can derive a probability distribution for the performance from that.

4.1 No Free Lunch

In this section we begin our considerations with the extreme case of black-box algorithms for . It may come as a surprise but in this scenario where we aim at designing black-box algorithms that are able to optimize any possible function f ∈ R S it is extremely simple to design an optimal algorithm. This is due to fact that all black-box algorithms perform equally.

Theorem 4.1 (No-Free-Lunch Theorem (Special Case)).

Let S and R be finite sets. On average over all functions f ∈ R S , all black-box algorithms for R S make the same number of different function evaluations in order to achieve an arbitrary optimization goal described by

Using our notation and restricting our attention to deterministic black-box algorithms, Theorem 4.1 can be stated as follows.

Before we prove this statement we make two observations with respect to vectors of function values that will be useful in the proof.

Lemma 4.1.

Let S and R be finite sets, let A be a non-resampling deterministic black-box algorithm for R S , let f,g ∈ R S

Proof.

We consider A, f, and g with . Let T A (f, t) denote the first t items in the trace T A (f). We observe that holds, since A is deterministic. The first point to be sampled s 1 ∈ S depends only on A, not on the objective function. Thus, and . Since holds by assumption we have as claimed.

The second point sampled for f can depend on f(s 1). The second point sampled for g can depend on g(s 1). Since A is deterministic and since holds, A makes the same decision for both functions after the first step. This implies .

Continuing this way we see that implies for all . Thus, we have (by induction). From

we see that f and g yield the same function value for each s ∈ S. Thus, we have f = g.

Lemma 4.2.

Let S and R be finite sets, let A be a non-resampling deterministic black-box algorithm for R S

Proof.

Considering we see that this set cannot contain more vectors of function values than there are functions f ∈ R S . Thus we have .

On the other hand, we have as an immediate consequence of Lemma 4.1. Thus, follows.

Equipped with Lemmata 4.1 and 4.2 it is easy to prove Theorem 4.1 for deterministic black-box algorithms. This will turn out to be sufficient to prove it for randomized black-box algorithms, too.

Proof (of Theorem 4.1).

First, we consider deterministic black-box algorithms only. We want to prove the following equation for arbitrary optimization goals M and arbitrary deterministic black-box algorithms A and A′.

(4.1)

Since we only care about different function evaluations, we can assume that A and A′ both do not resample any point in the search space. From Lemma 4.2 we know that holds. Considering V A (f) for an arbitrary function f ∈ R S we recall that V A (f) is a vector of function values of length . Fixing an arbitrary ordering on S we can use such vectors of function values to define functions . Since there are different functions in R S and different vectors of function values in , we see that the sets actually are the same. Thus, holds. Therefore, the two sums in Eq. (4.1) sum over the same sets, only the ordering of the summation may be different. Thus, Eq. (4.1) obviously holds.

We have Eq. (4.1) for arbitrary deterministic black-box algorithms. We remember that randomized black-box algorithms can be described as probability distributions over deterministic black-box algorithms. Thus, their performance is given as a weighted sum of the performances of deterministic black-box algorithms. Let be the performances of all deterministic black-box algorithms. Then the performance of a randomized black-box algorithm is given by

where holds, since the p i define a probability distribution. We have and get

and hence equal performance for any randomized black-box algorithm, too.

Theorem 4.1 was known as the no free lunch (NFL) theorem and was around in a technical report several years before being properly published. It caused immense discussions since it contradicted the widespread and admittedly in some ways naive belief that evolutionary algorithms are robust problem solvers that perform above average. In a formal sense, it proved that there is no algorithm at all that performs above average. On average over all problems all algorithms perform equally. And there can be no doubt that this means equally bad. We can formulate this result in provocative ways if we want to. On average, no algorithm can beat pure random search. On average, if you aim at maximization, it does not make more sense to do hill-climbing than hill-descending. This last formulation sounds surprising. It highlights the specific way the NFL theorem measures performance. It only cares about the points an algorithm samples, not what the algorithm does with these points. Moreover, it takes into account only the number of distinct points sampled, ignoring resampling completely. Therefore, it is only vaguely related to the real run time of real algorithms. We can argue against the NFL theorem on this ground but that would be inappropriate here. There is some truth in the NFL theorem that needs to be acknowledged. If we know nothing at all about the optimization problem we want to solve, we cannot expect to find an algorithm that performs any better than any other algorithm. For a specific problem at hand, of course, some algorithms are better than others. But if we cannot make any assumption about the concrete problem at hand the only thing that can help us find a good algorithm is luck. And we should try to avoid depending on our good luck alone.

In practical settings, however, we will never deal with ‘all problems.’ Let us assume that we are interested in functions mapping bit strings of length 40 to integers that can be stored in a single byte, so we have S = { 0, 1}40 and . Thus, our set of all possible problems is and we wonder how many such functions there are. This is easy to calculate. One such function can be uniquely described by a table of function values. Such a table has lines and in each line we can write any of the possible function values. Thus, there are

such functions. If we decide uniformly at random which of these objective functions we want to optimize the chances for any specific function to become our concrete objective functions is less than 1 : 102 640 000 000 000. For all practical purposes this is zero. Of course, being exceedingly unlikely does not mean that an event may not occur. Considering any national lottery we observe that each week exceedingly unlikely events happen: each specific outcome of the random experiment that is (hopefully) at the bottom of each national lottery is exceedingly unlikely. Nevertheless, we know that each week one of these exceedingly unlikely events is going to happen with probability 1. Unfortunately, we simply do not know which in advance. Here, when optimizing functions , things are different. After deciding which concrete function we want to optimize by means of an evolutionary algorithm, we need to implement something that realizes the oracle for our evolutionary algorithm. Presented with a point in the search space s ∈ { 0, 1}40, it needs to come up with the function value . Since implementing such functions is boring and tedious we decide to hire a student to do this. Since our funds are limited we tell the student to implement the objective function in Java but only if the source code needed for implementation does not exceed 1 GB in size. Otherwise we are afraid that she may need too long to implement the function and we are unable to pay her for all the time it takes. How many functions does this rule out? Since this is difficult to count we make a very crude estimation that gives an extremely optimistic estimation of the number of functions the student may actually be able to implement given our size bound on the source code. We assume 1 GB to denote 1,024 bytes or 2^33 bits. Clearly, there are at most 2^33 different Java source codes of this length. Thus, the number of objective functions that can be implemented is bounded above by . We see that the percentage of functions that can be implemented is less than

which, again, for all practical purposes equals zero. We realize that it is plainly absurd to even think about all possible objective functions. The vast majority of those will never be of any concern. They cannot even be implemented for evaluation so that optimization is completely infeasible. Luckily, the functions in which we are interested in practice are not taken uniformly at random from the set of all possible functions. They do have some properties and structure that distinguish them from the vast majority of other functions. This is the basis for hope that on relevant functions evolutionary algorithms may excel.

Realizing that the NFL theorem (Theorem 4.1) talks about the set of all functions and we are not interested in the set of all functions is not sufficient to simply neglect it. The NFL theorem tells us that for all black-box algorithms for perform equally. This does not rule out that this may hold for other sets , too. In the worst case it could hold for any set . Luckily, we already observed that this is not the case. For an optimal black-box algorithm optimizes f with a single function evaluation and other black-box algorithms need more function evaluations (provided ) and are thus worse. So, perhaps, the NFL theorem does only hold for ? Unfortunately, this is not the case. We will now consider a stronger formulation of the NFL theorem that explicitly takes into account. In order to do so we need to introduce a property of such sets .

Definition 4.3.

Let S and R be finite sets. Let { Perm}(S) denote the set of all permutations of S.

For a function and a permutation π ∈ { Perm}(S), we define the function by for all s ∈ S, where π − 1 ∈ { Perm}(S) is the inverse of π.

A set of functions is called closed under permutation of the search space if

holds.

It is not so easy to see if a specific class of functions  is closed under permutation of the search space. Clearly, R S itself is since there is no function at all missing. In Sect. 4.2 we will see that there are classes of functions  with that already are closed under permutation of the search space. Moreover, we observe that for each class of functions there is a set with and such that is closed under permutation of the search space. For each function there can be at most permutations π ∈ { Perm}(S) such that πf is missing in . Adding these functions yields a set of functions  that is closed under permutation of the search space. This is due to the fact that adding these missing functions does not introduce new functions where again the permuted functions are missing. Already here we observe that these permuted functions are related to another. We will make this more precise in the proof of the next NFL theorem.

Theorem 4.2.

Let S and R be finite sets, let . If is closed under permutation of the search space, then on average over all functions  , all black-box algorithms for make the same number of different function evaluations in order to achieve an arbitrary optimization goal M (with ).

Proof.

It suffices to prove the statement for deterministic black-box algorithms. It can easily be extended to randomized black-box algorithms analogously to the way this is done in the proof of Theorem 4.1.

We consider two deterministic black-box algorithms A and A′. Following the lines of argumentation in the proof of Theorem 4.1 we prove

This clearly proves the claim since in the equivalent of Eq. (4.1) (see p. 17) in the two sums we sum over the same elements.

Consider some arbitrary function  and how A works on this function f, i.e., we consider

We want to find a function f′ such that the algorithm A′ behaves on f′ the same way the algorithm A behaves on f. Clearly, we cannot achieve that holds since the first points sampled may already differ for A and A′ and we are not free to choose those. This, however, is not important. Since we care about performance as measured by some performance measure it suffices if we find an f′ such that .

Since A′ is a deterministic black-box algorithm it can be represented by its tree representation. Let s 1 ′ be the first point sampled by A′, i.e., the label of its root. We define a permutation π on S step by step and start with . The label of the root in the tree representing A is s 1. This implies , and consequently holds for the function πf.

Let s 2 ′ be the successor of s 1 ′ in A′ via the edge that is labeled with f(s 1). Clearly, the second point sampled by A′ on πf is this point s 2 ′. We define . Now we have , and holds.

We continue this way and observe that we do indeed define a permutation since we can again assume that neither A nor A′ resample any point in the search space. We have , so defining f′ : = πf we do have the function we are looking for. Since the set  is closed under permutation of the search space we know that holds. Since we can easily change the roles of A and A′ we indeed get as needed to complete the proof.

We now have a much clearer picture what is sufficient in order for an NFL theorem to hold. If the class of functions  under consideration is closed under permutation of the search space we know that all algorithms perform equally—given that we concentrate on distinct function evaluations and ignore resampling. To make the picture complete we would need to know about necessary conditions for an NFL theorem to hold. It is probably not too surprising that being closed under permutation of the search space is both necessary and sufficient. Before we are too disappointed by the proof of this result that we will refer to as ‘the’ NFL theorem, we remind ourselves what the negation of ‘for all performance measures all algorithms have equal performance’ actually is. It suffices to find at least one performance measure (no matter how contrived) and two algorithms (no matter how stupid) such that their performance differs.

Theorem 4.3 (NFL Theorem).

Let S and R be finite sets, let . On average over all functions  , all black-box algorithms for make the same number of different function evaluations in order to achieve an arbitrary optimization goal  if and only if the set  is closed under permutation of the search space.

Proof.

The first part of the statement coincides with Theorem 4.2. It therefore suffices to find some set  that is not closed under permutation of the search space, a performance measure M, and two black-box algorithms A and A′ such that

holds.

Since is not closed under permutation of the search space we can find a function  and a permutation  such that holds. We consider this function h, this permutation π and an arbitrary deterministic black-box algorithm A for . We define a performance measure M by

We know from Lemma 4.1 that implies for all functions. Thus, we have

and now need some other black-box algorithm A′ for with different average performance. We define such an algorithm A′. This algorithm will be deterministic and simply enumerate the complete search space S in a specific ordering. To define this ordering we consider

We define the algorithm A′ to enumerate S in the ordering π(s 1), π(s 2), …, where π is the permutation with we fixed above.

By definition of πf we have for all . Thus, follows. Since we see that

holds, so that A and A′ have different performance.

Now we know that exactly for classes of functions that are closed under permutation no search heuristic can outperform any other. This result requires some clarification. Is it typical for a class of functions to be closed under permutation of the search space? Do we have to expect in real applications to be confronted with classes of functions that are closed under permutation of the search space? While it is difficult to answer these questions in general—How can you know what kind of functions you will have to optimize in your life before you actually encounter them?—we are able to give two answers that are similar in spirit though they rely on different assumptions. Both make valid points that question the practical relevance of the NFL theorem. This is good news because it at least allows for the possibility that design and analysis of evolutionary algorithms (and other general problem solvers) is something that may actually be worthwhile. The first answer we give is concerned with the number of classes of functions that are closed under permutation of the search space. Theorem 4.4 states that the vast majority of function classes is not closed under permutation of the search space. The second answer is concerned with the structure of classes that are closed under permutation. This important answer is presented after the proof of Theorem 4.4.

The NFL theorem (Theorem 4.3) makes a statement about average performance. This corresponds to considering objective functions  drawn uniformly at random. We already argued that in practical settings problems are hardly ever generated this way. On the other hand, if this is what the NFL theorem considers it should be allowed to consider classes of functions  that are selected uniformly at random. We do not claim that in practical settings random classes of functions are optimized. We only argue that if the NFL theorem assumes that it is reasonable to consider functions f selected uniformly at random from , it should also be reasonable to consider classes of functions  selected uniformly at random from R S . If we select  uniformly at random from R S the probability that  is closed under permutation of the search space equals the fraction of such sets among all sets . This fraction, however, can easily be recognized as extremely small.

Theorem 4.4.

Let S and R be finite sets. The fraction of sets among all subsets of R S that are closed under permutations of the search space equals

Proof.

We already counted the number of all functions f ∈ R S and know that this equals . Thus, the set of all subsets of such functions equals . Now we need to count the number of these classes that are closed under permutation of the search space.

For a function we define its histogram  by

for all r ∈ R. We define a relation  ∼ on R S by

for any two functions f, g ∈ R S . Note that ∼ is an equivalence relation. Reflexivity, symmetry, and transitivity all follow directly from these properties of = , which also is an equivalence relation. Thus, ∼ defines equivalence classes [f], and we call the equivalence class of f its base class B f .

We observe that f ∼ g holds if and only if there exists a permutation π ∈ { Perm}(S) such that f = πg holds. If f = πg holds we have since πg shifts the function values around in the search space without changing them. If, on the other hand, holds, we can shift the function values around such that we get the same function. This is equivalent to saying that we can find a permutation such that f = πg holds.

Consider some  that is closed under permutation. We claim that

holds. Since we have f ∈ B f and holds, we have . Now assume that there exists some with . Since we have there exists some such that g ∈ B f holds. Since we have g ∈ B f , g ∼ f follows. Thus, there is some π ∈ { Perm}(S) such that g = πf holds. Since is closed under permutation of the search space this implies . We observe the contradiction and conclude that no such g exists. Thus, we have as claimed.

From we conclude that the number of sets  closed under permutation equals 2 b if b is the number of base classes. For the definition of a base class it suffices to distribute the  points of the search space among the  different function values without distinguishing the different points in the search space. This is equivalent to distributing the μ members of a population among the  different individuals in the search space. Thus, we can count the number of base classes in the very same way we counted the number of populations in Sect. 3.1. We conclude that holds.

For typical values of and the fraction of sets of functions closed under permutation of the search space is extremely small, double exponentially small. Thus, it is pretty safe to assume that for sets of functions selected uniformly at random an NFL theorem does not hold. This is a valid way of questioning the practical value of the NFL theorem since the NFL theorem itself considers functions selected uniformly at random. From a practical point of view, however, this line of reasoning is not very interesting. In practice, we do not choose our problems uniformly at random. Practical problems do have some structure, and we would like to know if we should expect those classes of functions to be closed under permutation of the search space.

In order to talk a bit about practical problems, we consider the notion of a neighborhood . For s, s′ ∈ S we say that s and s′ are neighbors if N(s, s′) = 1 holds. Otherwise, if N(s, s′) = 0, we say that s and s′ are not neighbors. In practical applications we almost always have or can define such notion of a neighborhood. We call such a neighborhood nontrivial if not all points are neighbors of each other, i.e., there exist with , and if not all points are not neighbors, i.e., there exist with . Such nontrivial neighborhoods are needed if we want to talk about local optima since the notion of locality is connected to the notion of a neighborhood. The crucial observation is the following. Nontrivial neighborhoods are not preserved under permutation of the search space. There is always a permutation such that two points that are not neighbors are mapped to points that are neighbors and vice versa. We conclude that any set of functions  that has any nontrivial property that is based on a nontrivial neighborhood cannot be closed under permutation of the search space. If, for example, we know that local minima and local maxima are not direct neighbors, if there is any nontrivial bound on the number of local optima, if the functions we are dealing with are smooth in some way that can be defined via a nontrivial neighborhood, we can be absolutely certain that this class of functions cannot be closed under permutation of the search space. This, indeed, is a very strong point against the NFL theorem. Since almost all practical problems have such properties we can say that in practice NFL theorems do not hold.

We just made a very strong point that questions the relevance of the NFL theorem. This should not be surprising. We all know that there are better and worse algorithms. The NFL theorem was originally received with much surprise because it contradicted this practical knowledge. It states that on average all algorithms perform equally worse and it is therefore not at all worthwhile to put effort in the design of algorithms. Computer science exists as a field because of the fact that in practice this is not true. But it would be dangerous and rather short-sighted to ignore the message that is inherent in the NFL theorem completely. If we do not know anything about the classes of functions our heuristic algorithm will be dealing with it is rather presumptuous to assume that it will perform any better than any other heuristic algorithm. Moreover, it would be naive and even wrong to assume that if our heuristic algorithm performs well on some test problems it will perform equally well on other problems, even if the problems are in some sense similar. The converse is true. If our algorithm performs well on some problems, it is bound to perform much worse even on similar problems. We make this point precise in the following almost no free lunch (ANFL) theorem.

We consider the following scenario. For some concrete search space S and some concrete value space R, we have an objective function . We have some black-box algorithm A for some class of functions  that includes f, i.e., . Assume that we are very pleased with the performance of A on f. Does this imply that A is similarly good on similar functions? Unfortunately, we can prove that there are double exponentially many other functions where the black-box algorithm A fails miserably. However, most of these double exponentially many functions are very unlikely to be encountered in practice. What we know about f is that f has in some sense restricted complexity. We already discussed this when we discussed the fraction of functions that we can actually encounter in practice. If we optimize f in the black-box scenario the oracle for f needs to be implemented in some way. It is necessarily feasible to evaluate the function values f(s) for arbitrary s ∈ S, so f has bounded time for evaluation. In special circumstances we may even decide to implement f in special hardware in order to decrease the time needed for f-evaluations. But also in this case, the size of the digital circuit computing f needs to be bounded in a reasonable way. More generally speaking, the complexity of f (as measured by Kolmogoroff complexity, measuring its minimum description length) needs to be bounded in some way. Clearly, we can restrict our interest to functions that are similar to f in the sense that their complexity is not much larger. But even taking these restrictions into account, there is still an exponentially large number of functions where A fails. These statements are proved in a formal way in the proof of the following ANFL theorem. We formulate this theorem in a rather specific way because we want to concentrate on the message it contains and want to avoid to get lost in technical details. It can easily be generalized to other search spaces and value spaces.

Theorem 4.5 (ANFL Theorem).

Let with , let , let S ={ 0,1} n, , let , A a randomized black-box algorithm for some set  with

The number of functions such that A does not find a global optimum of f′ within 2 n∕3 function evaluations with probability at least 1 − 2 −n∕3 is bounded below by . Of these functions at least exponentially many have the additional property that their complexity (measured by time for evaluation, circuit size or Kolmogoroff complexity) exceeds that of f by at most O(n).

Proof.

As we already discussed, we can assume that the black-box algorithm A eventually samples any point in the search space. This can easily be achieved by enumerating the complete search space at the end, and since we do not take points sampled after finding a global optimum into account this does not change anything in a significant way. We consider the situation after A sampled exactly 2 n ∕ 3 points in the search space. For each point s ∈ { 0, 1} n , we denote the event that s has been sampled in these first 2 n ∕ 3 steps by Q(s) and its probability by q(s). We observe that holds: in 2 n ∕ 3 steps at most 2 n ∕ 3 different points can be sampled. Regardless of the concrete probability distribution in each of these steps, the total sum of probabilities over these steps cannot exceed 2 n ∕ 3 since in each single step the sum of all probabilities to be sampled in this step equals 1.

Points s ∈ { 0, 1} n are bit strings of length n. Now we consider bit strings of length 2n ∕ 3, b ∈ { 0, 1}2n ∕ 3. We can use such a bit string b as a prefix of a schema b ∗  n ∕ 3 that represents 2 n ∕ 3 points in {0, 1} n (using the notation of Sect. 3.2). We collect all the bit strings matching this schema in the set S b , i.e., . Analogously to q(s), we denote the probability that there exists some x ∈ S b that has been sampled in the first 2 n ∕ 3 steps by q′(b). Obviously, since we have and can simply make use of the union bound.

For any we have . Thus,

holds. We observe that there exists some  with . Otherwise we would have

in contradiction to .

We now make use of this when defining the functions f′. Each function is defined such that f′(x) = f(x) holds for all . For the 2 n ∕ 3 points from the function value can be defined in an arbitrary way. We only need to have f′(s′) = N for some . We see that there are more than different concrete definitions of such functions f′.

We know that algorithm A samples no point from within the first 2 n ∕ 3 steps when operating on f with probability at least 1 − 2 − n ∕ 3. Since f and f′ agree in all points not in this holds on f′, too. Since all global optima of f′ have function value N and are in , with probability at least 1 − 2 − n ∕ 3 the algorithm A does not optimize f′ within the first 2 n ∕ 3 function evaluations. This completes the proof of the first statement.

For the second part it suffices that there are exponentially many different ways of completing the definition of f′ that are not complex to realize. It suffices to store the global optimum  and set f′(x) = c for all and some arbitrary .

We summarize what we have learned about principal limits in black-box optimization from the no free lunch theorem. The most important lesson is that we cannot hope to achieve results without making any assumptions at all. There is no good heuristic for all problems. This is the valid part of the NFL theorem. But it is important to keep in mind that even for classes of functions where the NFL theorem holds (classes of functions closed under permutation of the search space), the NFL theorem refers to distinct function evaluations. It is only very loosely connected to the real optimization time of optimization heuristics. On the other hand, even more importantly, for the vast majority of classes of functions, and among them all practically relevant classes of functions, the NFL theorem does not apply. So, in practice, it does make sense to look for good optimizers and it is worthwhile to think about design and analysis of optimization algorithms. We have to be careful and realistic about what we hope to achieve, though. We know from the ANFL theorem (Theorem 4.5) that good performance on some problems implies bad performance on many others. The best we can hope for is to tune the search heuristic we work on in a way that it performs well on the problems we happen to be interested in.

4.2 Black-Box Complexity

The NFL perspective that we discussed in the last section is strange in several ways. Concentrating on distinct function evaluations and inherently not taking resampling into account detaches the analysis from the real run time of the algorithms we use. We know that resampling can be avoided using a dictionary. But this is usually not done in practice. Most search heuristics do resample points. An appropriate theory for black-box optimization should take this into account. Second, the NFL perspective focuses on the average performance of black-box algorithms. In complexity theory as well as in the analysis of algorithms it is much more common practice to concentrate on the worst-case performance. Adopting this more pessimistic perspective guarantees good performance in practice in a stronger sense. Here we will address both issues by developing a complexity theory tailored toward black-box optimization.

We recall our definition of the optimization time of an black-box algorithm A for on an objective function  (Definition 4.2, see p. 47), where x t denotes the t{ th} point the black-box algorithm A samples. This notion of optimization time is realistic if we can assume that the actual computational effort of A is proportional to the number of function evaluations. It is at least more realistic than the number of distinct function evaluations since it takes resampling points into account. Equipped with this more realistic notion of optimization time, we can now give a formal definition of the worst-case perspective we will adopt.

Definition 4.4.

Let S and R be sets, , and let A be a black-box algorithm for . For let T A, f denote the optimization time of A on f.

We call the worst-case expected optimization time of A on .

We call the black-box complexity of .

The definition of black-box complexity is completely compliant with the usual notion of complexity in complexity theory. Since we are dealing with randomized algorithms we consider the expectation of the random optimization time, . We adopt the usual worst-case perspective and consider the maximum expected optimization time over all functions , the expected optimization time for a most difficult objective function. Using this measure of the worst-case expected optimization time, we want to find out what an optimal algorithm can achieve. Thus, we define the complexity of a class of functions  as the smallest worst-case expected optimization time any black-box algorithm for can achieve.

In some sense, our definition of black-box complexity implies a strong influence of subset relations between classes of functions and their black-box complexity. We make this simple observation precise in the following lemma.

Lemma 4.3.

Let S and R be sets,

Proof.

Since , in the definition of the worst-case expected optimization time for the supremum is taken over a subset of those functions for which it is taken over for . This cannot increase the supremum. Since this holds for any black-box algorithm A, it also holds for the black-box complexity.

Black-box complexity is very different from more classical measures of complexity because it does not take the actual computational effort into account. This is justified by the fact that usually randomized search heuristics like evolutionary algorithms do not perform very costly computations in addition to the function evaluations. However, since there is nothing in their definition that hinders black-box algorithms from performing very costly operations in addition to the function evaluations, one could take this to an extreme. The clear differences from classical complexity theoretical results should not come as a surprise. This is no serious objection to black-box complexity. We should simply be aware of the fact that it really only cares about function evaluations. This simplicity allows us to actually prove very strong and absolute lower bounds on the black-box complexity. This is different from classical complexity theory, where almost all bounds are relative only and rely on some plausible yet unproven assumption like P ≠ NP. In order to make the difference between black-box complexity and classical complexity clear, we consider an illustrative example.

We define the class of polynomials of degree at most 2 as the set of functions

that we want to maximize and investigate . Consider the following algorithm A, where we use the notation used to define schemata, so 0 i denotes the concatenation of i 0-bits. If i = 0, 0 i is the empty string.

With respect to the number of function evaluations we make the following observations. Algorithm A makes exactly one function evaluation in line 1, exactly n function evaluations in line 2, exactly {n}{2} = n(n + 1) ∕ 2 function evaluations in line 4, and exactly one function evaluation in line 9. No other functions evaluations are made, so the total number of function evaluations equals . We only remark that the computational effort of algorithm A is since the loop in lines 6–8 iterates over 2 n different x.

We want to prove that A actually is a black-box algorithm for . The crucial observation is that holds. This implies for all . This, in turn implies for all i and j with 1 ≤ i < j ≤ n. This implies that at the end m = o holds and y is a global maximum. Thus, algorithm A is in fact a deterministic black-box algorithm for . Since any upper bound on the number of function evaluations for any black-box algorithm for yields an upper bound on the black-box complexity of , we have just proved that holds.

This is remarkable if compared to the computational complexity of optimizing functions from . We remember that is the class of all polynomials over of degree at most 2. Maximization of this class contains as a subproblem the well-known problem Max-2-Sat, the problem of finding an assignment for a set of clauses that satisfies a maximal number of clauses simultaneously. It is well known that Max-2-Sat is NP-hard. So we have an NP-hard problem with only polynomial black-box complexity. Note that this is no contradiction. In fact, our black-box algorithm A that established the polynomial upper bound requires exponential computational effort. But it demonstrates that we need to be careful when dealing with black-box algorithms that make very costly computations in addition to the function evaluation. For these algorithms the black-box complexity may be an inappropriate measure. However, for the algorithms we are dealing with here it turns out to be very useful and powerful.

In Lemma 4.3 we observed an influence of subset relations on the black-box complexity of the involved sets of functions. Clearly, this is not directly related to the size of the sets. There is, however, an easy-to-observe connection between the size of a set of functions  and its black-box complexity .

Lemma 4.4.

Let S and R be sets, a finite set of functions.

If in addition S is finite, holds.

Proof.

Let . Each function  has at least one point in the search space where the maximal function value  is assumed. Let s i  ∈ S be such a point for f i . Consider the deterministic black-box algorithm A that samples s i as i{ th} point in the search space. A is an black-box algorithm for , since it finds an optimum for each . Moreover, since it optimizes each within at most function evaluations it proves . We can improve on this by using a randomized black-box algorithm that samples the at most different optima in an order that is determined uniformly at random. Since there are at most different optima, we see that for each one of its optima is sampled as t{ th} search point with probability . This yields

as improved upper bound.

For the bound we consider pure random search equipped with a dictionary to avoid resampling points. Thus, we enumerate the search space in a random ordering. This is feasible since S is finite. We observe that for any fixed s ∈ S and any we have that s is the t{ th} point sampled with probability . Any objective function  has at least one optimum s  ∗  ∈ S. Thus, the number of points sampled before some optimum is sampled is bounded above by the number of points sampled before s  ∗  is sampled. This yields

as upper bound on the expected optimization time of pure random search with dictionary for any , and proves .

Since both upper bounds, and , are valid, we have the minimum of the two as upper bound.

When one analyzes the expected optimization time of an evolutionary algorithm, this is typically done for a very specific evolutionary algorithm A operating on a very specific fitness function f. While this yields a possibly valuable result on , it is not too helpful from the perspective of black-box complexity. As long as we consider a single objective function f we are dealing with a very specific class —and we already know that in this case holds. Each black-box algorithm needs at least a single function evaluation for any function so holds for any nonempty set of functions . On the other hand, follows from and Lemma 4.4. But, luckily, this is no reason to despair. Since we are dealing with evolutionary algorithms we are able to make use of invariance properties of evolutionary algorithms that help us to generalize very specific results on for a specific evolutionary A and a specific fitness function f to results for this algorithm A and a class of fitness functions  with and . Based on we can hope to prove nontrivial lower bounds on and compare with this lower bound on in order to assess the performance of A on f. We do so in three steps of generalizing objective functions. We define all three steps right here and discuss the concrete relations to different evolutionary algorithms afterwards. We use the notation for bit strings x, a ∈ { 0, 1} n to denote the bitwise exclusive or of x and a, i.e., for all we have if and only if x[i] + a[i] = 1 holds.

Definition 4.5.

Let be some function.

For a ∈ { 0, 1} n we define by .

For we define by .

We define .

We define .

We define .

For i, j, k ∈ { b, p, v} we define and .

We discuss this somewhat lengthy definition step by step. For a ∈ { 0, 1} n , f a is identical to f if we change to roles of 0-bits and 1-bits at exactly those positions i where a[i] = 1 holds. In particular, is identical to f, and is identical to f if we replace x ∈ { 0, 1} n by its bitwise complement . We see that f (b) is a set that contains all functions with the following properties. Any two functions from f (b) can be turned into one another by exchanging the roles of 0-bits and 1-bits at some subset of positions. Moreover, f (b) contains all functions that can be constructed that way. We observe that holds and that these bounds are tight. For f(x) = c ( some arbitrary constant value) we have f (b) = { f}, since regardless of x the function value is always c. Thus, changing the roles of 0-bits and 1-bits does not change anything. On the other hand, for we have since we have g a (x) = 0 for all and g a (x) = 1 for . Thus, for we have , and we see that there are as many different functions g a as there are different choices of a.

We motivated Definition 4.5 by invariance properties of evolutionary algorithms. We observe that any evolutionary algorithm that is symmetric with respect to 0-bits and 1-bits behaves on any f′ ∈ f (b) as it does on f. Thus, any bound on implicitly and simultaneously proves the very same bound on for all f′ ∈ f (b). We realize that it makes sense to compare the performance of algorithm A on f with the black-box complexity . An upper bound that is close to  indicates very good performance. We observe that all evolutionary algorithms constructed by means of the standard modules described in Chap. 2 are symmetric with respect to 0-bits and 1-bits and thus we can always consider the generalization from f to f (b).

For , f π is identical to f if we rearrange the bits in x with respect to π − 1. We observe that holds, and it is again easy to see that these bounds are tight. As for f (b), we have for any constant function f. If we consider a function g that assigns 2 n  different function values to the 2 n  points in the search space {0, 1} n , we have . As far as the performance of evolutionary algorithms, we see that evolutionary algorithms are oblivious with respect to this generalization if they are symmetric with respect to bit positions. This holds for evolutionary algorithms that are purely mutation-based or that make (additionally) use of uniform crossover. It does not hold, for example, for evolutionary algorithms that make use of k-point crossover.

Finally, f (v) contains all functions where the function values are transformed in any way possible such that the ordering of the function values is unaltered. So, we have

for any function . Since there are any infinitely many strictly increasing functions , we have that f (v) is an infinite set even for constant functions f. Evolutionary algorithms have equal performance on f and any f′ ∈ f (v) if they are oblivious to the concrete function values and are sensitive only with respect to the ordering of the function values. At least when we only make use of our standard modules this depends on selection, only. For example, for uniform selection, plus-selection, comma-selection, and tournament selection this is the case. It is not the case for fitness-proportional selection and selection types derived from it, like Boltzmann selection.

We already noticed that evolutionary algorithms may be oblivious to more than just one of these generalizations. For those evolutionary algorithms, Definition 4.5 allows for combinations of the generalizations in order to come to classes of functions with larger black-box complexity. This will become clearer when considering concrete functions, generalizations of those functions, and deriving results on the black-box complexity of these classes of functions.

In the ideal case, we would like to determine the black-box complexity  for some class of functions  exactly. This, however, will turn out to be too difficult in many cases. Therefore, we will often be content with upper and lower bounds on . As usual in complexity theory, it is much more difficult to prove lower bounds. For an upper bound on , it suffices to prove some upper bound on  for a concrete black-box algorithm A for . For a lower bound on , we need to prove that no black-box algorithm for can beat this bound. Clearly, statements for all possible black-box algorithms are much harder to prove. Before we will be able to do so, we introduce a very valuable tool for proving lower bounds, namely Yao’s minimax principle.

We start with a very brief detour in game theory. We consider games between two players where the loss of the first player is what the second player wins and vice versa. Such games are called two-player zero-sum games in game theory. You may think of complex games like chess or simple games like rock-paper-scissors. In order to keep things simple we will concentrate on rock-paper-scissors and agree that the winning player will pay the losing player 1. In case of a drawn, nothing is payed. We can exactly express what is paid under what circumstances using a matrix, the so-called payoff matrix. In the payoff matrix for rock-paper-scissors (see Table 4.1) we observe that rock beats scissors (because it can make them blunt), scissors beat paper (because they can cut it), and paper beats rocks (because it can wrap it).

Table 4.1

Payoff matrix defining the game of rock-paper-scissors

Rock

Paper

Scissors

Rock

0

− 1

1

Paper

1

0

− 1

Scissors

− 1

1

0

One round of rock-paper-scissors consists of both players deciding on their strategy, i.e., deciding if they want to play rock, paper, or scissors. Once this decision is made by both players they can use the payoff matrix to find out who wins. Given such a payoff matrix, we can associate one player with the rows of the matrix and the other player with the columns. In rock-paper-scissors there is an equal number of rows and columns and, moreover, the rows and columns have equal labels. This, however, is a special case and may be different in other two-player zero-sum games. In general, the payoff matrix is an n ×m-matrix  with n rows and m columns. If the row player decides to play i and the column player j, the outcome of the game is defined by m i, j . The row player receives m i, j from the column player, thus the column player has to pay m i, j to the row player. Receiving negative amounts actually means paying and paying negative amounts actually means receiving. We see that the row player aims at maximization while the column player aims at minimization. Assuming that the players both know the game, i.e., know the payoff matrix, and assuming that they are rational, the row player will choose a row i that maximizes the payoff given the best choice the other player can make, thus receiving . In the same way, the column player will choose a column j that minimizes the payoff given the best choice the other player can make, thus receiving . In case we have we call the game solved since there are optimal strategies for both players. For rock-paper-scissors, this is not the case since we have V r  =  − 1 and V c  = 1.

Readers who are familiar with the game of rock-paper-scissors may wish to raise objections. Typically, you do not decide on a strategy and after that stick to it. In order to play successful rock-paper-scissors you will probably make some random decision about the strategy you choose to play. In the language of game theory this means that the row player chooses a probability distribution p over the rows. Accordingly, the column player chooses a probability distribution q over the columns. Now the actual payoff becomes a random variable where the concrete value depends on the random experiments determining the actual strategies played. The expected payoff can be computed as

Since the row player still wants to maximize the (expected) payoff and the column player still wants to minimize the (expected) payoff we now get and . In this situation, every finite game has a solution. This result is very well-known as von Neumann’s minimax theorem. In our notation we can simply write it as . The simple example of rock-paper-scissors proves that randomization is necessary. The game does not have a solution using pure strategies only. Using randomization, allowing for so-called mixed strategies, it has. In the case of rock-paper-scissors, playing each of the three possibilities with equal probability 1 ∕ 3 is optimal. Randomization, however, is not necessary on both sides. If I know the mixed strategy of my opponent there is a pure strategy that is an optimal choice for me. This result is known as Loomis’ theorem.

While playing games is always fun it is unclear why we took this detour into game theory. But there is more than a few rounds of rock-paper-scissors for us to gain. The key observation is that we can consider the design of efficient algorithms as a two-player zero-sum game. We restrict our attention to problems with a finite number of possible inputs each of finite size. In this case there is only a finite number of deterministic algorithms. We have already seen such a situation when we discussed deterministic black-box algorithms in the context of the NFL theorem in Sect. 4.1. We remember that in this situation we can describe randomized algorithms as probability distribution over deterministic algorithm. For a concrete finite problem the game consists of choosing, on one hand, a deterministic algorithm A for the problem and, on the other, choosing a concrete input I. We define the payoff to be the run time of algorithm A on input I, so the payoff is T(A, I). Thus, the algorithm designer corresponds to the column player who aims at minimizing this run time. The opponent or adversary aims at maximizing the run time and thus to select a very difficult input. If we allow for randomized algorithms, we allow the algorithm designer to decide upon a probability distribution p over all deterministic algorithms.

Using this perspective on algorithm design we get von Neumann’s minimax theorem as

and Loomis’ theorem as

(4.2)

where denotes the set of all deterministic algorithm for the problem under consideration and the set of all possible inputs for this problem. Considering Loomis’ theorem in this context we can make two very simple modifications. If, on the left-hand side of Eq. (4.2), we omit the over the probability distributions over the inputs and fix an arbitrary probability distribution p over the inputs this cannot increase the left-hand side but it may decrease it. If, on the right-hand side of Eq. (4.2), we omit the min over the probability distributions over all deterministic algorithms and fix some arbitrary randomized algorithm A q , this cannot decrease the right-hand side but it may increase it. Thus, we can make both modifications and replace the ‘ = ’ by a ‘ ≤ ’ and this inequality will obviously be valid. It is a simple conclusion from Loomis’ theorem. It is known as Yao’s minimax principle and will turn out to be tremendously useful in proving nontrivial lower bounds on the black-box complexity of interesting classes of functions.

Theorem 4.6 (Yao’s Minimax Principle).

Consider a problem with finite set of possible inputs each of finite size allowing for a finite number of different deterministic algorithms.

For all probability distributions p over the set of inputs  and all randomized algorithms A q given by a probability distribution q over the set of deterministic algorithms

holds.

We rephrase what Yao’s minimax principle delivers. If we want to prove a lower bound on the worst-case optimization time of a randomized algorithm, it suffices to prove a lower bound on the expected optimization time of an optimal deterministic algorithm, where the expectation is taken over an distribution over the inputs that we are free to choose. There are two advantages. First, we are free to fix any probability distribution over the inputs that we consider to be difficult for algorithms but easy to handle in a proof. Second, and more important, it is often much easier to deal with deterministic algorithms than with randomized algorithms. This advantage, however, comes at the price that we need to consider an optimal deterministic algorithm. We will see in several examples that in spite of this difficulty Yao’s minimax principle is extremely helpful in proving lower bounds.

Using Yao’s minimax principle we will prove lower bounds on the black-box complexity of some classes of objective functions and begin with simple examples. These examples are derived from simple single example functions using the generalizations from Definition 4.5 (see p. 66). Since such example functions will be important for us not only here but also in Chap. 5 when we develop methods for the analysis of evolutionary algorithms, it is worthwhile to comment on the role of example functions here. Example functions are simply structured fitness functions that some call ‘toy problems’ as if they were something despicable. It is true that example functions are artificially constructed ‘problems’ that are in no way problems ‘from the real world’ that can be expected to be seen in practical applications. So why should anyone care?

Our goal is to achieve a firm and solid understanding of how evolutionary algorithms work, what they can do and what they cannot do. Admitting that this is our goal implies that we admit that currently we do not really understand how evolutionary algorithms work. If we confront an algorithm that we do not really understand with a complicated problem that we do not understand either, it is hard to imagine that much understanding can be gained from that. It is much more plausible to consider the algorithm on a simply structured and well-understood problem—hence, a ‘toy problem.’ Due to their clear structure, example problems facilitate the analysis of evolutionary algorithms. This way they help us to develop methods for the analysis of such algorithms that prove to be very useful way beyond the problems they where developed for. The clear structure of our example functions reflects our understanding of typical properties of ‘real’ objective functions. They exhibit such properties in a very precise, distinct, and paradigmatic way. Therefore, they are ideally suited to make important aspects of evolutionary algorithms clearly visible. We hope to and often can discover general properties of evolutionary algorithms this way. Once we have developed a useful collection of example functions and have a solid understanding of how evolutionary algorithms function on them, we can use them as a test bed for new kinds of randomized search heuristics and new variants of evolutionary algorithms. This yields valuable insights about new algorithms on a solid basis. Moreover, they can help us to develop new ideas for the design of new variants of evolutionary algorithms and other search heuristics. Sometimes our example functions can serve as counterexamples for some wrong intuition or widespread wrong belief. In these cases we allow for more ‘contrived’ example functions tailored toward their role as counterexample. Finally, example functions have an important place in teaching. They allow us to present important facts and ideas in an easily accessible yet precise and formal way. They are very concrete and simply structured so that they are easy to understand. And yet they are defined in a formal way and allow for the derivation of theorems and proofs allowing us to develop our knowledge about evolutionary algorithms in a sound scientific way.

We begin with a first and very simple example function that does not seem to be particularly interesting at first sight. We use it here for a first (but, strictly speaking, unnecessary) application of Yao’s minimax principle. It will become more interesting when analyzing variants of evolutionary algorithms.

Definition 4.6.

The fitness function Needle is defined by Needle .

We observe that Needle yields the function value 0 everywhere except for the unique global optimum 1 n , where the function value is 1. If a black-box algorithm does not know about the location of this global optimum it cannot gather any information from the function values other than that the global optimum has not yet been found. We speculate that this makes the function difficult to optimize. We can confirm this intuition by considering Needle (b). For unknown the location of the unique global optimum actually is unknown.

Theorem 4.7.

Proof.

An upper bound follows from Lemma 4.4 since 2 n is the size of search space. For the lower bound we apply Yao’s minimax principle and use the uniform distribution on Needle (b) as probability distribution on the inputs. Now we need to derive a lower bound on the number of function evaluations an optimal deterministic black-box algorithm needs on average to locate an optimum for any .

A deterministic black-box algorithm for Needle (b) is a tree that contains at least 2 n  nodes. This is the case since Needle (b) contains 2 n  functions with 2 n  different optima. If a black-box algorithm contains less then 2 n  nodes there exists at least one function in Needle (b) that is not optimized by this algorithm. Thus, this algorithm cannot be a black-box algorithm for Needle (b).

We consider an optimal deterministic black-box algorithm for Needle (b) given by its tree and know that it contains at least 2 n  nodes. Since each f ∈ NEEDLE(b) maps to {0, 1}, each node on this tree can only have one outgoing edge, the edge labeled with 0. If the point s ∈ { 0, 1} n with function value 1 is found the optimization is complete and no further points are sampled. Thus, the tree is actually a list of exactly 2 n nodes. We realize that any optimal black-box algorithm for Needle (b) enumerates the search space in some order. Now we consider the average depth of the global optimum where the average is taken over all functions Needle (b). For any fixed ordering of points, we have exactly one function f ∈ NEEDLE(b) where the global optimum has depth i in this tree (for each ). Thus, the expected performance of an optimal deterministic black-box algorithm for Needle (b) for the uniform distribution over Needle (b) equals

The proof of Theorem 4.7 is a nice example for the application of Yao’s minimax principle because it is so simple. The single steps of the application are thus easy to follow. It is, however, strictly speaking, an unnecessary application of Yao’s minimax principle. We could have know the result before. The key observation is that Needle (b) is closed under permutation of the search space. This is easy to see for this specific class of functions. All a permutation of the search space can change is the position of the unique global optimum. Since we include all functions for all 2 n  different positions of the local optimum in Needle (b) this class is obviously closed under permutation of the search space. This implies that we can apply the NFL theorem (Theorem 4.3, see p. 56) and know that all algorithms that do not resample make the same number of function evaluations on average. We can count this average number by considering any algorithm, say one that simply enumerates the search space in a fixed ordering. This is what we did in the proof of Theorem 4.7. The result is 2 n − 1 + 1 ∕ 2. Since optimal black-box algorithms obviously do not resample the result follows. This, however, is a rare case. Usually classes of functions are not closed under permutation of the search space and we do not learn much from the NFL theorem.

We continue with a second application of Yao’s minimax principle and introduce a second example function. This example function, known as OneMax, yields as function value the number of 1-bits in the bit string.

Definition 4.7.

The fitness function OneMax is defined by OneMax .

OneMax may well be the best investigated fitness function in the context of evolutionary computation. It gives very strong hints toward its unique global optimum 1 n . Not only do the fitness values increase if the Hamming distance to the unique global optimum decreases. It is even the case that the fitness values depend on this Hamming distance in the most direct way, we have OneMax . The fitness value equals the Hamming distance to the bitwise complement 0 n of the unique global optimum 1 n . We can easily generalize this observation to OneMax a . The fitness value of OneMax a (x) equals the Hamming distance to the bitwise complement a of the unique global optimum , i.e., OneMax . Since the two unique points with minimum function value 0 and maximal function value n have maximal Hamming distance n, we see that OneMax (b) cannot be closed under permutation of the search space. Thus the NFL theorem cannot tell us much useful about this class of functions. This is no surprise. Due to its clear structure and due to the clear hints toward the global optimum we expect that problem-specific black-box algorithms for OneMax (b) are clearly superior to other algorithms like pure random search with dictionary. For this algorithm, just like for any deterministic algorithm A that simply enumerates the search space, we have for all a ∈ { 0, 1} n that holds since the number of global optima for OneMax equals 1. We observe that for these algorithms the calculations from the proof of Theorem 4.7 apply for all functions with unique global optimum.

Theorem 4.8.

Proof.

We apply Yao’s minimax principle and use the uniform distribution on OneMax (b). As in the proof of Theorem 4.7, we observe that any deterministic black-box algorithm for OneMax (b) contains at least 2 n  nodes. Here, however, the degree of each node may be as large as n since there are n + 1 different function values and only for the optimum value n no other points need to be sampled. The average number of function evaluations of an optimal black-box algorithm therefore equals the average depth of such an n-ary tree with at least 2 n  nodes. It is well known that this average depth is bounded below by .

We see that the lower bound is very much smaller than the trivial upper bound of 2 n − 1 + 1 ∕ 2 for enumerating the search space. This does not come as a surprise. Since we are not interested in developing problem-specific algorithms, we will not find out if the bound may actually be tight for  (which it is, in case you wonder; see the remarks at the end of the chapter). We will, however, prove some upper bound on  by proving an upper bound on the expected optimization time of a simple evolutionary algorithm in Sect. 5.1.

We introduce a third (and for this section the final) fitness function. It will highlight the reasons for not being content with f (b) in an interesting way.

Definition 4.8.

The fitness function BinVal is defined by BinVal .

Just like OneMax, BinVal is a linear function. The function value for x can be computed as a weighted sum of the values of the separate bits without any interaction between the bits. In some sense, it is an extreme linear function. While for OneMax all bits are equally important, for BinVal the importance of bits decreases very strictly from left to right. For any , it is the leftmost bits where x and x′ differ that decides if x or x′ has the larger function value under BinVal. If and only if x has a 1-bit at this position, then BinVal(x) > { BinVal}(x′) holds. The name BinVal is motivated by the fact that the function value BinVal(x) equals the non-negative integer that is represented by x in standard binary encoding. Since BinVal is similar to OneMax as both functions are linear, one may speculate that is similar to . This, however, is not at all the case.

Theorem 4.9.

Proof.

The lower bound is a very simple application of Yao’s minimax principle. We choose the uniform distribution on BinVal (b). Regardless of the first point a deterministic black-box algorithm samples, it finds the unique global optimum of f ∈ BINVAL(b) with probability 2 − n . The best an optimal black-box algorithm for BinVal (b) can achieve is finding the optimum with the second point sampled. In this case the average performance equals

Since this is the best a deterministic black-box algorithm for BinVal (b) can achieve, it is a lower bound on .

For an upper bound we make the following observations and describe a randomized black-box algorithm. We know that we have to optimize some f ∈ BINVAL(b). This implies that there exists some a ∈ { 0, 1} n , such that f = BINVAL a holds. The unique global optimum of BINVAL a is , the bitwise complement of a. The key observation is that given some known x ∈ { 0, 1} n , knowing f(x) enables us to compute a. We have

and knowing f(x) implies that can easily be calculated. It suffices to calculate the standard binary encoding of f(x). Knowing and x[i] implies that we can calculate a[i] since holds. We restate these observations in form of a randomized black-box algorithm A for .

We observe that A optimizes f with probability 2 − n in line 2. If this is not the case (with probability 1 − 2 − n ) we use f(x) to calculate a (in lines 3–4). Finally, in line 5, the unique global optimum of f is sampled. We see that the average number of function evaluations of algorithm A on any f ∈ BINVAL(b) equals 2 − 2 − n .

The reason for the extremely small black-box complexity of BINVAL(b) is that there is way too much information in the function value. The exact location of the unique global optimum can be calculated by considering a single function value f(x) for a know point x in the search space. In this situation it pays to hide the information that is otherwise given in the concrete function values. We do so by considering BINVAL(b, v).

Theorem 4.10.

Proof.

We observe that we cannot apply Yao’s minimax principle since BINVAL(b, v) is not finite. Therefore, we consider a slightly different scenario, first. We change a black-box algorithm for BINVAL(b, v) such that for the t{ th} point sampled it does not get its function value but its position in the sorted sequence of the function values of the t − 1 points sampled so far. In this scenario we can easily derive the bound in the following way. We apply Yao’s minimax principle and again use the uniform distribution. Again we have at least 2 n nodes in each deterministic black-box algorithm. Since until we sample the n{ th} point the range of possible positions is at most we have degree at most n up to depth n in the tree. This yields average depth .

The difference from the real situation is that a black-box algorithm receives some concrete function value. Due to an arbitrary function h transforming the function values there is no additional information in the function value. Thus, follows.

So far we have seen how generalizations (Definition 4.5, see p. 66) can be used to generate nontrivial classes of functions and how Yao’s minimax principle can be applied to derive lower bounds on the black-box complexity. All examples considered so far have been restricted to classes of functions based on single example functions. Neither black-box complexity nor Yao’s minimax principle are restricted to such classes of functions. To demonstrate this we are now going to introduce a practically relevant class of functions and derive a nontrivial lower bound on its black-box complexity. This will yield an interesting and practically relevant insight concerning the difficulty of this function class.

For functions over the domain  we all know what a local maximum is. We have a picture of a gradient in mind that leads us to such a local maximum. For functions over the domain {0, 1} n things are less obvious. We consider the Hamming distance to be the most natural metric for {0, 1} n and thus Hamming neighborhood as the most natural neighborhood in this space. One could define that a point is called a local maximum if all its Hamming neighbors have strictly smaller function value. This definition, however, makes Needle a function with a unique local maximum 1 n , so it should be called unimodal. Clearly, for Needle there are no hints in direction of the local maximum at all. We have seen that Needle (b) is among the most difficult classes of functions over the domain {0, 1} n by finding out that holds. Both aspects lead us to not call this function unimodal. This is the reason that we define a local maximum in a different way.

Definition 4.9.

For a function we call x ∈ { 0, 1} n a local maximum, if and only if

holds. The function f is called unimodal if and only if it has exactly one local maximum. The function f is calledweakly unimodal if and only if all local maxima have the same function value.

This definition of unimodal has the immediate consequence that local search (with Hamming neighborhood) finds a global maximum on an objective function f regardless of the starting point if and only if f is weakly unimodal. For such functions a point is either a local (and thus also global) maximum or it has a Hamming neighbor with strictly larger function value. So, from any point s ∈ { 0, 1} n , there exists a path of Hamming neighbors with strictly increasing function values leading to some global maximum. There are no local maxima that are not global ones, so the local search cannot get stuck and evolutionary algorithms do not need large mutations or other ‘lucky events’ in order to leave a local maximum. These properties may lead to the belief that (weakly) unimodal functions are in some way easy to optimize. If easy to optimize is supposed to mean that a global maximum can be found rather quickly this belief, however, is completely wrong. We will prove this making use of the notion of black-box complexity. We derive an exponential lower bound on the black-box complexity of unimodal functions.

Theorem 4.11.

The proof of Theorem 4.11 is neither short nor simple. Since the result is useful to correct our wrong intuition about unimodal functions being easy to optimize and since it says something about unimodal functions, a practically relevant function class, it is worthwhile investing some effort.

We remember Lemma 4.3 (see p. 63). We can fix an arbitrary subset  and prove for any constant δ < 1. Then Theorem 4.11 is a direct consequence. We define a class of unimodal functions that implements the idea of a path leading to a global optimum in a very direct way. We call a sequence of points a path of length l if and only if for all . We restrict our attention to such paths that start in 1 n , i.e., holds. For such paths we define a path function by

We observe that f P is unimodal. For points not on the path the function value increases strictly with the number of 1-bits. Thus, there is a path leading to 1 n , the first point on the path. For points on the path, the function value is given by adding n to the position of the point on the path. Note that we do not require the path to be a simple path. Thus some points in {0, 1} n may appear on the path multiple times. For such points we define the function value by adding n to the maximal position on the path. Clearly, the last point of the path p l is the unique global optimum of f P .

We fix some arbitrary constant with , where δ is the constant from Theorem 4.11. Now we restrict our attention to such paths where holds. We assume that and spare us the inconvenience of writing if this is not the case. Now, for this choice of l(n) we define the class of path functions . Obviously, holds.

Theorem 4.12.

Let δ < 1 be an arbitrary constant. Let with be an arbitrary constant. Define

This is the result we needed for the proof of Theorem 4.11. We note this here before proving Theorem 4.12.

Proof (of Theorem 4.11).

We make use of Theorem 4.12 where δ < 1 is the constant from Theorem 4.11. Since holds we have as an immediate consequence of Theorem 4.12 and Lemma 4.3.

The proof of Theorem 4.12 will be based on one central lemma that we will state and prove first. We prepare ourselves for this by describing the most natural random construction of the kind of paths we consider. All paths we consider have in common that they start in , they have length l(n), and each point on the path (except for the last point p l(n)) has a successor on the path with Hamming distance 1. The most simple way of constructing such a path randomly is starting in  and choosing the next point on the path uniformly at random among all Hamming neighbors.

Consider such a randomly constructed path P. If you start in some point p i and follow the path for a not too small number of steps, how likely is it that you are still close to p i (closeness measured by means of Hamming distance)? This probability is really small since the space {0, 1} n is so huge. We make this precise in the following lemma that will be central for the proof of Theorem 4.12 and thus Theorem 4.11.

Lemma 4.5.

Consider a randomly constructed path

Proof.

Since the random path construction is oblivious to the path position i it suffices to prove the lemma for i = 1 and some fixed j ≥ βn. We assume that i + j = 1 + j ≤ l(n) holds, otherwise there is nothing to prove. We define random variables  for all . We need to prove for some constant α that we are free to choose depending on β.

We make the following crucial observations concerning the random construction of the path P. We have since the next point on the path is chosen among all Hamming neighbors of the current point. This can be imagined as selecting one of the bits in the current path point uniformly at random and flipping it. Thus, we have , since in order to decrease the Hamming distance by 1 we need to flip one of the H t out of n bits that differ from p 1 in p 1 + t . We can conclude that holds.

We define . Since we have β > 0, α > 0 follows and this is a valid choice of α. Moreover, we define γ : = min{1 ∕ 10, j ∕ n}. Note that γ is not necessarily a constant since j may not be constant. However, due to its definition we have 5α ≤ γ ≤ 1 ∕ 10. We consider the last γn steps of the random path construction until p j is constructed. It is important to note that γ ≤ j ∕ n holds. This guarantees that these last γn steps of the random path construction until p j is constructed actually do exist.

We make a case distinction with respect to the value of H T , where T is the first of these last γn steps we consider. First, consider the case where H T  ≥ 2γn holds. We consider γn steps and in each step the value of H t can decrease by at most one. Thus, at the end we have H t  ≥ 2γn − γn = γn with probability 1. We see that if we are sufficiently far away at the beginning of those last steps we cannot be too close at the end.

Now we consider the other case, where H t  < 2γn holds. Again, since we consider only γn steps and in each step the value of H t can increase by at most one, we have H t  < 2γn + γn = 3γn all the time. This implies that . We see that we have a strong tendency to increase the Hamming distance during those last γn steps in this case.

We consider a series of γn random experiments with success probability at least 7 ∕ 10 in each experiment. We are interested in the number of successes M′. Now consider another random process where we consider a series of γn independent random experiments each with success probability exactly 7 ∕ 10. Let M denote the number of successes in this second random process. For any , we have . Thus M stochastically dominates M′, and we may analyze this second random process instead. Due to the independence of the random experiments, we can apply Chernoff bounds (see Appendix A.2). We have EM = (7 ∕ 10)γn and get

With probability 1 − 2 − Ωn we have at least (3 ∕ 5)γn steps, where H t is increased among those last γn steps. Thus there can be at most (2 ∕ 5)γn steps where H t is decreased. Since H t is non-negative, we have

with this probability .

Considering the two cases together, we have with probability  a Hamming distance of at least (1 ∕ 5)γn ≥ αn between p 1 and p 1 + j .

Equipped with this knowledge about properties of random paths we can now come to the proof of Theorem 4.12.

Proof (of Theorem 4.12).

We want to apply Yao’s minimax principle to prove the lower bound. Clearly, each path  defines one path function f P . We remember the random path construction and observe that this way each such path has a certain probability to be created. We associate this probability of such a path P to be created with its corresponding path function f P . This defines a probability distribution on . We use this probability distribution in our proof.

We would like to prove that any black-box algorithm cannot make much progress on f P with probability close to 1. Unfortunately, this is not really true for all f P . Since the beginning of the path  is known, black-box algorithms can start on the path P. Clearly, it is simple to make progress on the path P (and thus on f P ) by looking for a Hamming neighbor with larger function value. This requires at most n function evaluations. If we are lucky (from the perspective of proving a lower bound) this increases the function value by only 1. But if the algorithm is lucky it may constitute a huge step on the path, skipping many points. Remember that a path is not necessarily simple and thus shortcuts may exist. We deal with these shortcuts by considering the path after removing all shortcuts. This shortens the path and reduces its length. We call the number of remaining path points the true length of the path P.

We realize that it is highly likely that a random path P is not a simple path. For example, we have a (very short) cycle of length 2 if holds. This happens with probability 1 ∕ n, since from the arbitrary neighbor p i + 1 we need to pick exactly the correct Hamming neighbor p i as new path point p i + 2. While 1 ∕ n is a small probability converging to 0, it is highly likely to see many such events in  steps of the path construction. The probability of not having such a circle at all is  and thus is extremely small.

Obviously, we cannot ignore the existence of circles. Lemma 4.5, however, provides us with a means of proving that with probability close to 1 there are no large cycles. We claim that having cycles of length  ≥ n is highly unlikely. We apply Lemma 4.5 with β = 1. The probability to return to the same path point after n steps is . The probability that this happens in the  steps of path construction is bounded by due to the simple union bound. The key observation is that holds. We have a path length that is weakly exponential but a probability that is strictly exponentially small. Thus multiplying the two still yields an exponentially small probability. We pessimistically assume that the path is full of cycles of length  < n. This yields .

Now we need to prove a lower bound on the performance of an optimal deterministic black-box algorithm. In order to do so we consider a simplified scenario where we equip the black-box algorithm with additional knowledge and ask for less than optimization. The advantage is that this way we come to a scenario that is much easier to handle.

Before the first point is sampled a black-box algorithm for knows that the path starts in . In addition to that, we equip it with the knowledge of all path points p i with and with . This corresponds to removing all circles. If there is a circle coming back to the first point p 1 we let the algorithm know everything about this circle. We observe that this can only improve the algorithm’s performance.

At any point, i.e., after any number of points sampled, we can describe the knowledge the algorithm has about its objective function f P in the following way. Among all path points it knows there is some path point p j with maximal function value . We assume that the algorithm also knows about all path points p i with . If this is not the case we additionally equip the algorithm with this knowledge. This covers our practice of removing cycles. Moreover, there is a set of points known to not be on the path. Every x ∈ N has been sampled by the algorithm and could easily be identified as not being a path point by its function value f P (x) < n.

In the beginning we have j = 1 and . In each round the algorithm has to decide which point x ∈ { 0, 1} n to sample next. Clearly, since we are considering an optimal black-box algorithm this will be some point not sampled before. We distinguish three different cases with respect to this newly sampled point x.

1. Case:

In this case we first update N to . Moreover, we update j to j + n and additionally equip the algorithm with the knowledge about all path points p i with for the new value of j.

We equip the algorithm with additional knowledge very generously. Obviously this cannot decrease its performance.

2. Case: x  ∈  P and

In this case we again update j to j + n and additionally equip the algorithm with the knowledge about all path points p i with for the new value of j. This is similar to the first case.

3. Case: x  ∈  P and

In this case we update j to l(n) so that the optimum is found. The algorithm stops successfully in this case.

The description of our modified and simplified scenario is complete. What we really ask from the black-box algorithm is only to make an advance on the path of at least n in a single function evaluation. If this succeeds, we stop the process and pretend that the optimum was found. We observe that this way we can at best prove a lower bound of : We reduce the path length from l(n) to l(n) ∕ n by omitting short circles. We cannot consider the last points on the path since we need to have at least n more path points (leading to (l(n) ∕ n) − n). Finally, since we advance the algorithm by n in almost all steps we further reduce the effective path length to . Since we only need to prove a lower bound of for some this is sufficient.

Using our scenario we consider the situation when the very first point x to be sampled is decided. Since this is the first point the algorithm samples we have and j = 1. We again apply Lemma 4.5 with β = 1 and use the constant α(1) from this lemma. We distinguish two cases with respect to .

If the optimal black-box algorithm decides to sample some point x with  ≤ α(1)n (relatively close to the best known path point 1 n ) we have

due to Lemma 4.5.

The other option for an optimal black-box algorithm is to sample some x with (relatively far away from the best known path point 1 n ). Now we consider the random process of path construction as if it had not yet taken place completely. We have some first part of the path constructed (up to 1 n ) and ask ourselves what is the probability that we hit exactly some x with when starting in 1 n ? The situation is analogous to the situation considered in Lemma 4.5, and we see that again this probability is bounded by .

We realize that in the first step an optimal black-box algorithm for succeeds only with probability . What is different in later steps? Later we have j > 1, , and the algorithm knows about all path points p i with . Clearly, an optimal black-box algorithm will avoid to resample these points. We consider steps of the algorithm, in each step the algorithm gathers additional knowledge about at least n new points. Thus the number of points the algorithm knows about can be enormous, . Can this change a lot in favor of the black-box algorithm in comparison to the situation when the first point is sampled?

We collect all points known to the algorithm in a set K, i.e., . We partition this set K of known points into the set K { close} of points that are more or less close to the best known path point p j and a set K { far} for the other points that are farer away. We define

where α(1 ∕ 2) is the constant from Lemma 4.5 for β = 1 ∕ 2. Now we make a case inspection with respect to K { close}.

If holds, we claim that not much is changed in comparison to the situation in the very first step. We consider the next point x the algorithm samples and the path construction starting in the best known path point p j . Let A denote the event that the random path construction hits x. Let B denote the event that the random path construction avoids to hit any point in K = K { far}. Clearly, we are interested in (the next sampled point x is a path point given that the path is constructed respecting the knowledge the algorithm has). We have

and observe that holds. We already noted that we can conclude from the proof of Lemma 4.5 that a single point that has Hamming distance  from the current path point is hit in the process of random path construction only with probability . Again making use of the simple union bound we get that any of the points in K { far} is hit is also bounded by since again we are dealing only with a weakly exponential number of points. Thus, we have

and see that knowledge about points that are far away does not help much.

Knowledge about points far away does not help much because ‘far away’ is so huge. What the algorithm knows there covers only a tiny fraction of the whole space and is thus not too helpful. For points in K { close} things are different. What is ‘close by’ is much less and therefore knowing a lot about things nearby can considerably change the probability of finding something useful. We solve this problem by a relatively simple trick. Remember that we ask the algorithm to make progress of at least n. If holds, we ignore the next n ∕ 2 steps of random path construction and advance the algorithm by this on the path. After this we again have with probability  and can argue as above.

We have that in each step the algorithm succeeds with probability . Another application of the union bound yields that the algorithm succeeds in steps with probability at most . Thus, we have as claimed.

Here, we end our considerations of general limitations that are inherent to the black-box scenario. We will make use of the lower bounds we derived when trying to assess the quality of upper bounds on the expected optimization time of specific evolutionary algorithms on specific problems in the next chapter.

4.3 Remarks

In the evolutionary computation community the no free lunch discussion was initiated by Wolpert and Macready, who proved Theorem 4.1 in a technical report [136]. It caused immense discussion way before being properly published in the very first issue of the IEEE Transactions on Evolutionary Computation [137]. The sharpened NFL theorem that we presented as Theorem 4.3 came considerably later [113]. Even sharper results taking probability distributions into account are known [53].

The important observations about the fraction of subsets that are closed under permutation of the search space as well as the observation that neighborhoods are not preserved under permutation of the search space are due to Igel and Toussaint [52]. A tiny concrete example that the introduction of a bound on the complexity of the functions under considerations leads to function classes where an NFL does hold is due to Droste et al. [32]. The same authors presented the ANFL (Theorem 4.5), making the point that search heuristics need to fail on many problems [33].

What we presented with respect to the NFL is by no means the end of the story. We did not cover NFL theorems for continuous spaces [7] and also did not discuss relevant set-theoretic versions of it [36, 109]. We also did not cover attempts to concentrate on actually useful performance measures and to gain advantages from the insights, see work by Valsecchi et al. for an example [123].

Black-box complexity was introduced by Droste et al. [34, 35]. It has since then found its way into a textbook on computational complexity [130]. It has seen rapid developments and dramatic improvements lately. Lehre and Witt [77, 79] introduced a model for a restricted class of algorithms where the population size and the variation operators are restricted. Using the concept of unbiased variation operators and restricting the population size one can show that simple evolutionary algorithms like the (1 + 1) EA or random local search are actually optimal for OneMax. Another approach to obtain stronger results by sharpening and restricting the black-box complexity model is to restrict the arity of the variation operator: Mutation operators randomly change single individuals so that they have arity 1. Typical crossover operators operate on two parents, i.e., have arity 2. Variation operators with larger arity are conceivable, gene pool crossover has arity μ in an evolutionary algorithm with population size μ. Doerr et al. [26] introduced this concept and proved that variation operators with higher arity can actually imply lower black-box complexity. Note that both concepts, unbiased variation and considering the arity of variation operators, can be combined [22].

Results on black-box complexity are by no means restricted to classes of problems obtained by starting with single example functions (as outlined in Definition 4.5). Results on the black-box complexity of combinatorial optimization problems are known, too [25].

It has been pointed out that the notion of black-box complexity is closely related to the game of Mastermind [21]. Results from this area imply that the -bound for the black-box complexity of ONEMAX(b) are actually tight, something that was independently proven by Anil and Wiegand [4].

References

4.

G. Anil, R.P. Wiegand, Black-box search by elimination of fitness functions, in 10th ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA 2009), Orlando, ed. by I. Garibay, T. Jansen, R.P. Wiegand, A.S. Wu (ACM, New York, 2009), pp. 67–78

7.

A. Auger, O. Teytaud, Continuous lunches are free! in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007), London (ACM, New York, 2007), pp. 916–922

21.

B. Doerr, C. Winzen, Playing Mastermind with constant-size memory, in 29th International Symposium on Theoretical Aspects of Computer Science (STACS 2012), Paris, ed. by C. Dürr, T. Wilke. Leibniz International Proceedings in Informatics, vol. 14 (Dagstuhl Publishing, Saarbrücken, 2012), pp. 441–452

22.

B. Doerr, C. Winzen, Reducing the arity in unbiased black-box complexity. in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2012), Philadelphia (ACM, New York, 2012), pp. 1309–1316

25.

B. Doerr, J. Lengler, T. Kötzing, C. Winzen, Black-box complexity of combinatorial problems, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2011), Dublin (ACM, New York, 2011), pp. 981–988

26.

B. Doerr, D. Johannsen, T. Kötzing, P.K. Lehre, M. Wagner, C. Winzen, Faster black-box algorithms through higher arity operators, in 11th ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA 2011), Schwarzenberg, ed. by H.-G. Beyer, W.B. Langdon (ACM, New York, 2011), pp. 163–172

32.

S. Droste, T. Jansen, I. Wegener, Perhaps not a free lunch but at least a free appetizer, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 1999), Orlando (Springer, Berlin, 1999), pp. 833–839

33.

S. Droste, T. Jansen, I. Wegener, Optimization with randomized search heuristics – the (A)NFL theorem, realistic scenarios, and difficult functions. Theor. Comput. Sci. 287(1), 131–144 (2002)MathSciNetMATHCrossRef

34.

S. Droste, T. Jansen, K. Tinnefeld, I. Wegener, A new framework for the valuation of algorithms for black-box optimization, in Foundations of Genetic Algorithms 7 (FOGA), Torremolinos, ed. by K.A. De Jong, R. Poli, J.E. Rowe (Morgan Kaufmann, San Francisco, 2003), pp. 253–270

35.

S. Droste, T. Jansen, I. Wegener, Upper and lower bounds for randomized search heuristics in black-box optimization. Theory Comput. Syst. 39(4), 525–544 (2006)MathSciNetMATHCrossRef

36.

E.A. Duéñez-Guzán, M.D. Vose, No free lunch and benchmarks. Evol. Comput. (2013). doi:10.1162/EVCO_a_00077

52.

C. Igel, M. Toussaint, On classes of functions for which no free lunch results hold. Inf. Process. Lett. 86(6), 317–321 (2003)MathSciNetMATHCrossRef

53.

C. Igel, M. Toussaint, A no-free-lunch theorem for non-uniform distributions of target functions. J. Math. Model. Algorithms 3(4), 313–322 (2004)MathSciNetMATHCrossRef

77.

W.B. Langdon, R. Poli, Foundations of Genetic Programming (Springer, Berlin, 2002)MATHCrossRef

79.

P.K. Lehre, C. Witt, Black box search by unbiased variation, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2010), Portland (ACM, New York, 2010), pp. 1441–1448

109.

J.E. Rowe, M.D. Vose, A.H. Wright, Reinterpreting the no free lunch. Evol. Comput. 17(1), 117–129 (2009)CrossRef

113.

C. Schumacher, M.D. Vose, L.D. Whitley, The no free lunch and problem description length, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), San Francisco (Morgan Kaufmann, San Francisco, 2001), pp. 565–570

123.

A. Valsecchi, L. Vanneschi, G. Mauri, Optimisation speed and fair sets of functions, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2010), Portland (ACM, New York, 2010), pp. 1475–1476

130.

I. Wegener, Complexity Theory: Exploring the Limits of Efficient Algorithms (Springer, Berlin, 2005)MATH

136.

D.H. Wolpert, W.G. Macready, No free lunch theorems for search. Technical report SFI-TR-9502-010, Santa Fe Institute, 1995

137.

D.H. Wolpert, W.G. Macready, No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)CrossRef

Thomas JansenNatural Computing SeriesAnalyzing Evolutionary Algorithms2013The Computer Science Perspective10.1007/978-3-642-17339-4_5© Springer-Verlag Berlin Heidelberg 2013

5. Methods for the Analysis of Evolutionary Algorithms

Thomas Jansen^1

(1)

Department of Computer Science, University College Cork, Cork, Ireland

Abstract

Analyzing evolutionary algorithms is often surprisingly difficult. It can be challenging even for very simple evolutionary algorithms on very simple functions. This is due to the fact that evolutionary algorithms are a class of randomized search heuristics mimicking natural evolution that have been designed to search effectively for good solutions in a vast search space without any thought about analysis. The directed random sampling of the search space they perform gives rise to complex and hard to analyze random processes that can be described as complex Markov chains as we have discussed in Sect. 3.1. This motivates us to start our analysis with evolutionary algorithms that are as simple as possible and consider fitness functions that are as simple as possible. We do not claim that such evolutionary algorithms are actually applied in practice or that example problems bear much resemblance to real optimization problems.

Analyzing evolutionary algorithms is often surprisingly difficult. It can be challenging even for very simple evolutionary algorithms on very simple functions. This is due to the fact that evolutionary algorithms are a class of randomized search heuristics mimicking natural evolution that have been designed to search effectively for good solutions in a vast search space without any thought about analysis. The directed random sampling of the search space they perform gives rise to complex and hard to analyze random processes that can be described as complex Markov chains as we have discussed in Sect. 3.1. This motivates us to start our analysis with evolutionary algorithms that are as simple as possible and consider fitness functions that are as simple as possible. We do not claim that such evolutionary algorithms are actually applied in practice or that example problems bear much resemblance to real optimization problems. One must be careful not to hastily assume that such results are neither interesting nor important. It is not the specific results that are of lasting interest but the methods developed and applied to obtain them. While all methods are developed for analytical situations that are very much simplified, they prove to be applicable and useful way beyond such simple circumstances. We concentrate on the methods here, and describe their basis and their application using simple examples devoting one section to one method. We use instructive examples and consider the same example using different methods where possible, demonstrating the specific strengths and weaknesses of the different methods this way. Readers are urged to apply these methods to examples of similar simplicity themselves. It is only in the practical application of these analytical tools that competence in the analysis of evolutionary algorithms can be gained.

Clearly, we analyze theoptimization time T A, f here for different simple evolutionary algorithms A and different simple example functions f. Since T A, f is a random variable there are many different aspects we can concentrate on. Except for trivial cases, it is not feasible to describe its precise distribution completely. Moreover, such results confront us with a degree of precision that is not necessary to gain an understanding of the functioning of evolutionary algorithm A on example function f. In most cases we will be satisfied to derive results on the expected optimization time . Even there we do not strive for an exact calculation but are content with upper and lower bounds. We make use of the well-known Landau notation (see Appendix A.1) for growing dimension of the search space n to describe these bounds. While this asymptotic notion is not informative with respect to concrete, and in particular not with respect to small problem sizes n, it describes the way the expected optimization time scales with the size of the search space. This is much more informative than concrete bounds and even precise values for a specific problem size.

The derivation of upper and lower bounds on the expected optimization time  takes place in two separate steps making use of quite different methods. In the ideal case the lower and the upper bound match, and we successfully determined the expected optimization time asymptotically exact. When discussing a very simple lower bound technique in Sect. 5.3 we will realize that in some cases the expected optimization time can be a very misleading measure. It may be the case that while the expected optimization time  is really huge, the probability to observe a huge optimization time in a particular run may be rather small. In such cases it makes more sense to consider the distribution of the optimization time in some more detail. We consider the probability for rather short runs and rather long runs and derive upper or lower bounds for these probabilities.

5.1 Fitness-Based Partitions

We start with a very simple example and consider the (1+1) EA (Algorithm 2) on the examplefunction ONEMAX (Definition 4.7). We want to derive an upper bound on the expected optimization time . While ONEMAX is sufficiently simple to grasp its important properties without any support, we seize the opportunity to introduce to different graphical representations for example functions that are helpful to get an understanding of important properties of more complex example functions (see Fig. 5.1).

Fig. 5.1

Graphical representation of ONEMAX using hypercube representation (left) and function values over the number of 1-bits (right)

One way of representing a fitness function is based on a graphical representation of the Boolean hypercube. We draw the Boolean hypercube in an elliptic form, placing the 2 n  bit strings in n + 1 layers, where the i{ th} layer contains all  bit strings with exactly i 1-bits. Within each layer we sort the bit strings from left to right according to the positions of their 1-bits. The more 1-bits that are to the left of the bit string, the farther to the left it is placed. This makes the leftmost and the rightmost bit strings of the i{ th} layer. Clearly, we do not actually draw the 2 n  bit strings but know about their positions within this picture of the Boolean hypercube. Within this picture we use arrows to indicate directions of increasing fitness. For ONEMAX this yields the graph on the left-hand side in Fig. 5.1. Note that the levels in the middle represent an exponential number of bit strings whereas the upper and lower levels represent only a small number of bit strings. Thus, the proportions of the graphical representation are necessarily misleading. In spite of this unavoidable inaccuracy, this graphical representation is often helpful in developing an intuitive understanding of an example function’s properties.

The example function ONEMAX has the property that the function value { ONEMAX}(x) only depends on the number of 1-bits in x and not on the actual bit string x itself. Functions with this property are calledfunctions of unitation. For such functions it makes sense to plot the function value over the number of 1-bits. For ONEMAX, such a graph is given on the right-hand side in Fig. 5.1. It is important to remember that the middle of this graph represents exponentially many points in the search space whereas the left and right margin represent only a small number of bit strings.

The (1+1) EA employs strictplus-selection. Thus, the fitness value of the current population f(x t ) is increasing monotonically with t. Thus, we know that we get an upper bound on the expected optimization time when we bound the expected time needed to increase the function value to . We may do so be considering each x ∈ { 0, 1} n and derive an upper bound on the expected waiting time for a direct mutation leading from x to the unique global optimum 1 n . The largest of these expected optimization times is an upper bound on since, regardless of the actual path the (1+1) EA takes in the search space, for no population can the expected time needed to reach the global optimum exceed this maximal value. Clearly, this approach is much too coarse to allow for meaningful results. But it contains in its a core a useful idea. We can partition the search space into layers of search points with similar fitness. Clearly, each of these layers needs to be left at most once. Assume that we have an upper bound on the time needed to leave each layer. If the top layer contains exactly all global optima then the sum of these upper bounds is an upper bound on the expected optimization time . We capture this idea in the following definition and the subsequent theorem.

Definition 5.1.

Let be a fitness function and . A collection of sets is called an f-based partition if the following holds.

1.

form a partition of the search space {0, 1} n . and .

2.

The fitness values increase with increasing index. .

3.

L k is the set of all global maxima. .

The sets are called fitness layers. For each and some p m  ∈ (0, 1) we call

the probability for an improvement.

The term for the probability for an improvement

may look a bit intimidating at first sight. However, it is not difficult to understand if read from the inside out. For any two bit strings x, y we observe that equals the probability that using standard bit mutations with mutation probability p m we create y as mutant of x. When we consider some x ∈ L i and sum these probabilities for all y ∈ L j , we obtain the probability to mutate x into any bit string in L j . If we sum up these probabilities for all L j with j > i we obtain the probability to leave x in a single mutation and reach some point in some fitness layer above L i . Due to the definition of f-based partitions this implies that function value under f increases. Up to this point the calculations are exact. When considering the (1+1) EA on f and an f-based partition we will keep track of the fitness layer the current x t belongs to. Clearly, knowing the current fitness layer does not imply that we know the current x t exactly. We overcome this difficulty by taking the minimum probability for an improvement over all x ∈ L i . Thus, s i is a lower bound for the probability that the next offspring belongs to a better fitness layer given that holds for the current x t . Using this lower bound on the actual probability it is easy to obtain an upper bound on the expected optimization time.

Theorem 5.1.

Let be a fitness function, an f-based partition (for some ), and be the corresponding probabilities for an improvement.

Proof.

Consider for some . The time until L i is left, i.e., until x t is replaced by some with j > i and t′ > t, is stochastically dominated by the number of Bernoulli trials needed to get one success with success probability s i . This is due to the fact that s i is a lower bound on the probability to mutate x into some for all x ∈ L i . Thus, the expected waiting time for leaving L i is bounded above by 1 ∕ s i . Clearly, before L k is reached each fitness layer L i is left at most once. Moreover, once L k is reached a global optimum of f is found due to the definition of L k . Thus, we get an upper bound on by summing up the upper bounds on the expected waiting times 1 ∕ s i for all .

While the idea and proof of the method of f-based partitions is extremely simple, it yields good upper bounds in surprisingly many cases. This is true even though this upper bound does not take into account at least three obvious reasons why the (1+1) EA may actually be quite a bit faster than this simple upper bound. First, it may be very likely that many of the fitness layers are never encountered since the fitness values are way below the expected initial fitness . A second source for not encountering many fitness layers can be mutations from the current layer L i to a layer L j with j ≫ i. Finally, the probability s i may be way too pessimistic. While there is at least one x ∈ L i where s i is the actual probability of mutating x in some for many other x′ ∈ L i the probability may be considerably larger. When applying this method for the first time, however, we will not only live with these three sources of inaccuracy but even introduce a fourth. Instead of computing the probability for an improvement s i exactly we will be contented with a lower bound on s i .

Theorem 5.2.

Proof.

We apply the method of f-based partitions (Theorem 5.1) and need to define fitness layers. We define the fitness layers in the most trivial way, defining . This defines one separate fitness layer for each function value. Computing the probability for an improvement s i exactly for these fitness layers is quite difficult. We decide to consider a simple to prove lower bound instead. Consider some x ∈ L i for an . Since holds, we know that x contains exactly n − i 0-bits. A mutation increases the function value and thus causes the algorithm to leave L i if exactly one of these 0-bits is mutated and all other bits do not change. We can calculate the probability of such a mutation easily and see that

holds. This yields

as claimed.

Even though the upper bound is not large, we cannot know if it is tight. One may be disappointed that we have not been able to prove an upper bound of order since it obviously suffices to flip each bit once to determine its correct value. Note, however, that the (1+1) EA makes no assumption at all about ONEMAX and thus cannot make use of such structural insights. Moreover, we do not yet know if is asymptotically tight and thus really reflects the actual expected run time accurately. But we do know that the upper bound cannot be too far off. The result on the black-box complexity (Theorem 4.8) yields that the upper bound can be off by a factor of at most. We determine the deviation from the true asymptotic expected optimization time when discussing a lower bound in Sect. 5.2.

The extremely simple definition of fitness layers used in the proof of Theorem 5.2 proves to be useful not only for ONEMAX but in many other cases, too. We call an f-based partition with this property thetrivial f-based partition. The property can be formally defined as follows. .

We consider as second example. Remember that the function value (Definition 4.8) equals the non-negative integer that is represented by x in standard binary encoding. Thus for x ≠ x′ we have , and follows.

Using the trivialfitness layers is a bad idea for this function. We do not even need to consider the probabilities for an improvement s i for any i to realize that only upper bounds ≥ 2 n  − 1 can be proved this way. Clearly, we have s i  ≤ 1 and thus follows. For BINVAL with trivial fitness layers we have k = 2 n  − 1 since there are 2 n different function values. We observe that for the method of f-based partitions the number of fitness layers must not be too large. Using a less trivial definition of fitness layers, we can prove a reasonable upper bound for BINVAL.

Theorem 5.3.

Proof.

We apply the method of f-based partitions (Theorem 5.1) and define fitness layers

for . We need to be sure that this is indeed a BINVAL-based partition. The number is represented as in standard binary encoding. Thus, for all , L i contains all bit strings where the i leftmost bits are set to 1 and the next bit is a 0-bit. Thus, L 0 contains exactly all bit strings with a leading 0-bit and we have { BINVAL}(x) < 2 n − 1 for all x ∈ L 0. For we have . In particular, we have and see that the L i are in fact a BINVAL-based partition.

Similar to the proof of Theorem 5.2, we do not compute the probability for an improvement s i exactly but only prove a lower bound. A mutation of x ∈ L i leads to some y ∈ L j with j > i if the leftmost 0-bit (the bit x[i + 1]) is mutated and all other bits are unchanged. Such a mutation has probability and we have

for all . Thus

follows.

Again, we do not know how far this upper bound deviates from the actual asymptotic expected optimization time. The result on the black-box complexity, (Theorem 4.10), yields that this upper bound may be off by a factor of .

Our third example is concerned with . It demonstrates limitations of the method of f-based partitions.

Theorem 5.4.

Proof.

Since the function NEEDLE yields only two different function values, namely , there is only one way to define a NEEDLE-based partition. So we define and . The probability to mutate x into y decreases with increasing Hamming distance of x and y. It becomes minimal for . We see that we have since , , and hold. This yields .

In comparison with the black-box complexity, (Theorem 4.7), this upper bound may be off by a factor of . One may be even tempted to believe that the actual expected optimization time of the (1+1) EA on NEEDLE needs to be since { NEEDLE}(b) is closed under permutation of the search space and thus the NFL theorem (Theorem 4.3) applies. But we need to be careful here. The NFL theorem is concerned with the number of distinct function evaluations and does not take resampling into account. On the other hand, takes into account the number of all function evaluations and thus may be very different. This is another example for the very limited usefulness of the NFL theorem.

Do we believe that the upper bound n n is actually tight? It would be very surprising if it was. For all the probability for a direct mutation to the global optimum 1 n is at least . As long as 1 n is not found, the (1+1) EA performs a random walk in the search space, accepting any new bit string as new population x t . Thus, it appears to be unlikely that the expected waiting time is dominated by the waiting time of the very worst bit string. We will reconsider this when we have other analytical methods that we can use (see Sect. 5.5).

The next example we consider involves a new example function called LEADINGONES. It yields as function value the length of the uninterrupted sequence of 1-bits starting at x[1] as function value. We see that LEADINGONES is a unimodal function.

Definition 5.2.

The fitness function is defined by .

Theorem 5.5.

Proof.

We consider the trivial fitness layers . Note that these fitness layers coincide with the layer defined for BINVAL. Clearly, we can leave L i exactly the same way we do there. Thus, we have exactly the same upper bound.

Comparing the upper bound with the exponential lower bound on the black-box complexity of unimodal functions (Theorem 4.11), we see that LEADINGONES is a particularly simple unimodal function that is easy to optimize. Of course, the same holds for all linear functions (for example, ONEMAX and BINVAL). If we want to find a concrete example where at least the (1+1) EA needs considerably longer it pays to take note of a structural property of unimodal functions that translates directly into an upper bound on the expected optimization time of the (1+1) EA.

Theorem 5.6.

Let be aweakly unimodal function, and let denote the number of different function values.

Proof.

We consider the trivial fitness layers . Since f is weakly unimodal for each and each x ∈ L i , there exists at least one y ∈ { 0, 1} n with and f(y) > f(x). This yields , and we have

as an immediate consequence.

We see that the (1+1) EA is necessarily fast on LEADINGONES since there is only a linear number of different function values. If we want to see an exponential expected optimization time on a unimodal function we need to consider functions with an exponential number of different function values. Note that this is a necessary condition, but not a sufficient one. We have an upper bound even though BINVAL is a unimodal function with 2 n different function values.

Our next example is another unimodal function with an exponential number of fitness values. We divide its definition into two parts. First, we define a family of paths in {0, 1} n . In a second step we embed such a path into a unimodal fitness function.

Definition 5.3.

Let and with be given. , the long k-path of dimension n, is a sequence of points ∈ { 0, 1} n and is defined recursively. We have , the empty bit string. Let be the long k-path of dimension n − k. The long k-path of dimension n is defined as

where we call the points , , …, the front part, the points , , …, the back part, and the points , , …, the bridge. The length of is denoted by and equals the number of points in .

Before we make use of long k-paths of dimension n to define a specific class of unimodal functions we take note of important properties of these long k-paths. It is important to realize that these long k-paths are folded into the Boolean hypercube in a very special and useful way.

Lemma 5.1.

Let and with be given. The long k-path of dimension n has length . All path points are pairwise different. For all and all there is exactly one successor of p i in with Hamming distance d, namely p i+d

Proof.

We begin with the statement on the length  and prove it via induction over n. For n = 0 we have and holds. For n we know that the front and back part of have both length and can assume that holds. We observe that the bridge contains exactly k − 1 points. Together this yields

holds as claimed.

To see that all path points are pairwise different we consider and for each point  the first k bits . All points in the bridge are pairwise different since they differ in these bits. For the same reason they are all different from all points in the front and back parts. Since these first k bits are 0 k in the front part and 1 k in the back part no point in the front part can equal any point in the back part. Within the front and back parts we consider the rest of bit strings . These bits are all points from . It follows by induction over n that they are all pairwise different.

Now consider and such that . We prove the last statement via induction over n and thus can assume that it holds for . For n = 0 there is nothing to prove since the path is so short that there are no successors. If p i and p i + d are both either in the front part or in the back part then the statements holds via induction since it holds in .

We consider the bridge. By definition the j{ th} point in the bridge differs from the last point in the front part in exactly j bits. If differs from the first point in the back part in exactly k − j bits. The Hamming distances to all other points x in those parts can be computed as sum of this number and the Hamming distance between x and either the last point in the front part or the first point in the back part. Thus, the statement holds if p i is in the front part and p i + d is in the bridge as well as in the case where p i is in the bridge and p i + d is in the back part. Since we have d < k we cannot have the case that p i is in the front part and p i + d is in the back part. In the final case, where p i and p i + d are both points in the bridge the statement is obviously true due to the construction of the bridge.

We want to embed the long k-path  into a function such that the function is unimodal and the function values increase along the path. We define the function values in a way that all points lead toward the first path point 0 n .

Definition 5.4.

Let and with be given. The fitness function is defined by

For points not on the path Definition 5.4 puts strong emphasis on the first k bits. Thus, steps off the path can only lead in the direction of the front part. This way it is highly likely that evolutionary algorithms are faced with at east half of the long k-path when first entering the path. When proving lower bounds this will be important (see Theorem 5.18 in Sect. 5.4). In this section we only consider upper bounds.

Theorem 5.7.

Let and with be given.

Proof.

The upper bound is a direct consequence of the general upper bound for unimodal functions (Theorem 5.6). We only need to prove that is an upper bound, too.

We use the method of f-based partitions but only give implicit definitions of the fitness layers. While not on the path the fitness value can be increased by mutating the leftmost 1-bit into a 0-bit. Such a mutation has probability . After at most n such mutations some point on the path is reached. This yields an upper bound of for the expected time spent off the path.

When on the path we distinguish two cases. In the first case we are not yet on the back part of the path. In this case there is a mutation of at most k specific bits leading to the back part of the path. We flip exactly k bits when in the front path and less than k bits when in the bridge. Thus, the probability to reach the back part in a single mutation is always bounded below by . When in the back part we can ignore the first k bits. Then we are in the same situation as if we were on . We repeat the argumentation. Clearly, after at most n ∕ k such steps the end of the path is reached. Thus we have an upper bound of for the expected time spent on the path. Together this yields as upper bound.

The upper bound on (Theorem 5.7) is a result on a class of fitness functions since it holds for all valid choices of k. Such results are more general and thus more desirable than results on specific example functions. We introduce another class of example functions called and obtain a second results on a class of fitness functions by means of the method of f-based partitions.

Definition 5.5.

Let and be given. The fitness function is defined by

A graphical representation of { JUMP} k is given in Fig. 5.2. Since { JUMP} k is a function ofunitation it can be drawn by plotting the function value { JUMP} k (x) over the number of 1-bits in x.

Fig. 5.2

Graphical representation of . The representation describes the influence of the parameter k. It is the actual graph for n = 18 and k = 5

The search space {0, 1} n can be divided into three parts in a natural way. For all x ∈ { 0, 1} n with { ONEMAX}(x) ≤ n − k the function value equals k + { ONEMAX}(x). Since the additive constant k is of little importance the function can be optimized like ONEMAX in this part of the search space. The search is directed into the direction of the all ones bit string 1 n , the unique global optimum. This stops, however, at bit strings with exactly k 0-bits. This part of the search space is denoted as A in Fig. 5.2. For bit strings with more than n − k but less than n 1-bits the function value is given by n − { ONEMAX}(x). Thus, function values vary between 1 and k − 1 there. The function value can be optimized like n − { ONEMAX} in this part of the search space. The search is directed away from the all ones bit string 1 n . This part of the search space is denoted as B in Fig. 5.2. Note that all bit strings in part B have smaller function values than all bit strings not in this part. Finally, the unique global optimum, 1 n , forms the third and last part of the search space. This part is denoted as C in Fig. 5.2.

Theorem 5.8.

Let and be given.

Proof.

We make use of the trivial fitness layers that are defined by for , and . For i ≤ k − 2 we have exactly i + 1 0-bits in any x ∈ L i . It suffices to mutate one of the n − i − 1 1-bits into a 0-bit and not to change any of the other bits to leave L i . This yields

for . For we have exactly i − k + 1 1-bits in any x ∈ L i . It suffices to mutate one of the n − i + k − 1 0-bits into a 1-bit and not to change any of the other bits to leave L i . This yields

for . All x ∈ L n − 1 have exactly k 0-bits and second largest function value. It is necessary to mutate exactly all 0-bits into 1-bits in order to generate a bit string with larger function value. Thus, we have . Making use of these lower bounds we obtain

as upper bound on the expected optimization time.

We discuss one more example of an upper bound for a class of fitness functions. Unlike { LONGPATH} k and { JUMP} k we now consider not a family of example functions but a ‘natural’ class of functions: linear functions. Recall that a function is called linear if there are weights so that the function value equals for all x ∈ { 0, 1} n .

Theorem 5.9.

Let be a linear function.

Proof.

Let be the weights such that we have for all x ∈ { 0, 1} n . Since the (1+1) EA employsplus-selection that does not depend on the specific function values but only on the order of function values we can assume without loss of generality that w 0 = 0 holds. Since the (1+1) EA is completely symmetric with respect to 0-bits and 1-bits, we can assume without loss of generality that w i  ≥ 0 holds for all . If we have w i  < 0 we may replace w i by − w i and exchange the roles of 0 and 1 at the i{ th} position. Since the (1+1) EA is completely symmetric with respect to bit positions we can assume without loss of generality that holds.

In order to simplify the definition of an f-based partition we would like to assume that w i  ≠ 0 holds for all . In general, this cannot be done without loss of generality. Since we have to prove an upper bound on the expected optimization time, however, we may do so. If w i  = 0 holds the i{ th} bit has not influence on the function value. Setting w i  ≠ 0 implies that now there is a unique value for x[i] in the global optimum. The need to find this value cannot decrease the expected optimization time.

According to our assumptions, we now have . We define fitness layers

for and . These fitness layers are very similar to the fitness layers we used for BINVAL (see proof of Theorem 5.3, page 14). For any x ∈ L i it suffices to mutate exactly the leftmost 0-bit in order to increase the fitness value such that a higher fitness layer is reached. Since such a mutation has probability we see that

holds.

All the results obtained with the method of f-based partitions are for the (1+1) EA. This is due to the definition of the probability for an improvement (Definition 5.1) and the way we connect these probabilities to the expected optimization time (Theorem 5.1). It is, however, not difficult to change the definitions slightly to make method applicable to other randomized search heuristics.

When considering random local search we need to change the definition of the probability for an improvement. When we define

(where k denotes the total number of fitness layers as in Definition 5.1) with

when 1-bit Hamming neighborhood is used we can apply the method without any other changes. Most of the results in this section transfer to random local search directly. This is not true for the upper bounds for NEEDLE (Theorem 5.4), { LONGPATH} k (Theorem 5.7), and { JUMP} k (Theorem 5.8). In the proofs of these theorems mutations of multiple bits played a vital role.

More interesting than adapting the method to accommodate random local search is to change it in a way that makes it suitable for use with the (1 + λ) EA, i.e., the (μ + λ) EA (Algorithm 1) with population size μ = 1. Here, two changes are necessary. The (1 + λ) EA creates λ offspring and replaces its current population x t by one of them with maximal function value if not all are worse than x t . Therefore it makes more sense to define the probability for an improvement as the probability to have an improvement in λ independent mutations. This leads us to

as probability for an improvement. Like for the (1+1) EA, the term p equals the probability to mutate from x to some y in a higher fitness level. Thus, 1 − p is the probability not to have such a mutation and (1 − p)λ is the probability not to have such a mutation at all in λ independent mutations. Thus, 1 − (1 − p)λ is the probability to have at least one out of λ offspring leaving L i . The second change that needs to be made is a direct consequence of the change in the definition of the probability for an improvement s i . Since we consider a completegeneration, we need to take into account that λ function evaluations are made in each generation. Therefore we get

as upper bound (where k denotes the total number of fitness levels). We apply the method to LEADINGONES so that we see an example.

Theorem 5.10.

Let and be given with

Proof.

We consider the trivial fitness layers as we already did for the (1+1) EA in the proof of Theorem 5.5. We remember that the mutation of the leftmost 0-bit increases the fitness value and has probability at least . Thus we have

as lower bound for the probability for an improvement. This yields

as upper bound and we deal with 1 − e  − λ ∕ (v) in order to derive a more handy formulation. We make a case distinction with respect to the offspring population size λ. For λ ≥ en we have λ ∕ (en) ≥ 1, so − λ ∕ (en) ≤ − 1 follows and we have e  − λ ∕ (en) ≤ e  − 1 as an immediate consequence. Thus, we have − e  − λ ∕ (en) ≥ − e  − 1, and follows. We see that follows and we have

for λ ≥ en. For λ < en we make use of the fact that for all t ∈ [0, 1] we have 1 − e  − t  ≥ t ∕ 2 holds (see Fig. 5.3).

Fig. 5.3

Graph of 1 − e  − t and t ∕ 2 for t ∈ [0, 1]

Since we have λ < en we know that λ ∕ (en) < 1 holds and we have 1 − e  − λ ∕ (en) ≥ λ ∕ (2en) as an immediate consequence. Thus, we have

for λ < en. By adding up both upper bounds we have

in all cases.

The upper bound can be used to extract useful hints in practical circumstances. Assume that you want to implement an (1+λ) EA on a parallel machine with λ processors. Clearly, in each generation the λ offspring and function evaluations can be carried out in parallel. While the computational effort remains the time needed for optimization can be reduced. It is reduced to the number of generations . We see that for we have a speedup while increasing the offspring population size and number of processors to  does not lead to even faster optimization. Thus the result yields insight in the number of processors that can be utilized in a parallel computing environment. Note, however, that this reasoning would only be fully valid if the bound on was asymptotically tight. Since we have not yet discussed any lower bound methods we cannot tell, yet. Thus, we are now motivated to start working on analytical methods for proving lower bounds.

5.2 A General Lower Bound for Mutation-Based Algorithms

So far the only way of proving lower bounds on is proving a lower bound on the black-boxcomplexity of a suitable generalization of f. These bounds are systematically weak since they do not take the specific algorithm A into account but hold for all algorithms. While such general results are helpful for understanding problems, they do not provide us with any insights with respect to specific algorithms. Therefore we start to consider a more specific aspect of a class of algorithms in this sections. Namely we prove a general lower bound on the expected optimization for evolutionary algorithms with standard bit mutations with mutation probability p m  = 1 ∕ n as only variation operator for a wide class of fitness functions. Clearly, this bound is still general since it applies not only to a specific algorithm but to a wide class of algorithms.

Theorem 5.11.

Let be a function with a unique global optimum  . Let A be an evolutionary algorithm with uniform initialization and standard bit mutations with mutation probability p m = 1∕n as only variation operator.

Before we prove Theorem 5.11 we make one simple observation on expected values. It proves to be helpful in the establishment of lower bounds on the expected optimization time.

Lemma 5.2.

Let T be a non-negative random variable, B some event.

Proof.

The law of total probability states that for any partition of the probability space and any random variable T we have

Clearly, B and are a partition of the probability space. Thus,

holds. Since T is non-negative we have and follows.

Proof (Proof of Theorem 5.11).

If the population size μ is rather large, , the evolutionary algorithm A spends function evaluations in initialization. The probability to find the unique global optimum x  ∗  within the first function evaluations is bounded by as application of the union bound shows. Thus, with probability the algorithm needs function evaluations, and

follows by application of Lemma 5.2.

The more interesting case is concerned with smaller population sizes, i.e., . We consider the initial population P 0 and some x ∈ P 0. Application of Chernoff bounds yields . We apply the union bound and see that we have

Thus, in each individual of the initial population P 0 there are at least n ∕ 3 bits that all need to be mutated at least once before the unique global optimum x  ∗  is to be reached. It is elementary that the probability that this happens in (n − 1)lnn mutations is not too large. We argue as follows.

The probability that a specific bit is mutated equals p m  = 1 ∕ n. Thus, the probability that this bit is not mutated equals 1 − 1 ∕ n. We consider this bit in t subsequent mutations and see that is never flipped in this t mutations with probability . We define t : = (n − 1)lnn and have that a specific bit is never flipped in (n − 1)lnn mutations with probability at least . Thus, with probability at most 1 − 1 ∕ n this bit is mutated at least once within (n − 1)lnn mutations.

Now we consider not one but n ∕ 3 bits that all need to be mutated at least once. The probability that they all are mutated at least once in (n − 1)lnn mutations is bounded above by . Thus, with probability at least 1 − e  − 1 ∕ 3 there is at least one bit among the n ∕ 3 bits that all need to flip that is never mutated within (n − 1)lnn mutations. Thus, with this probability after (n − 1)lnn mutations the unique global optimum x  ∗  is not found. We summarize what we have for using Lemma 5.2.

Since we have as lower bound for population sizes as well as for population sizes , this bound holds for all population sizes.

It is not difficult to generalize Theorem 5.11 to fitness functions with more than just a single global optimum. The only thing we need is that in an initial population of size each member of the population has Hamming distance from each global optimum with probability . This is easy to show to be true for any polynomial number of global optima and even for an exponential number of global optima if these are sufficiently far away from the bit strings with n ∕ 2 1-bits.

Corollary 5.1.

Proof.

The upper bound is given in Theorem 5.2. We observe that ONEMAX has the unique global optimum 1 n . The (1+1) EA initializes its population of size 1 uniformly at random at uses standard bit mutations with mutation probability 1 ∕ n as only variation operator. Thus, the lower bound follows from Theorem 5.11.

It is surprising to see that two rather simple and seemingly weak methods for proving bounds turn out to be able to yield asymptotically exact results. The upper bound derived by means of f-based partitions turns out to be tight. And for this algorithm and fitness function the general lower bound is tight, too.

The general lower bound for all mutation-based evolutionary algorithms bears strong resemblance with a well-known problem from probability theory called the coupon collector’s problem. We can use known results from this problem to derive stronger results here. We do so to see a practical example how known results from probability theory can be transferred and how differences between the problem in probability theorem and the evolutionary algorithm can be overcome. While interesting, the following results in this section are not central for the rest of the book and may be skipped by readers who want to concentrate on the core concepts.

The coupon collector’s problem is about someone who aims at collecting all of n different coupons. The coupons are handed out one at a time, and each of the n types is handed out with equal probability 1 ∕ n. We are interested in the random number of coupons T that have to be collected until the collection of n different coupons is complete. The following lemma not only describes the average number of collected coupons  but also makes very precise statements about deviating from this expected value.

Lemma 5.3.

Let n be the number of coupons in the coupon collector’s problem, and T the number of coupons obtained when the collection becomes complete.

1.

2.

3.

Proof.

We consider the random process of collecting coupons and divide it into disjoint phases. The i{ th} phase encompasses all coupons that are collected while the current collection consists of exactly i different coupons. The random number of coupons obtained in this phase is called T i .

By definition of T i we have . Thus, holds. In the i{ th} phase the collector posseses exactly i different coupons. This implies that the probability to obtain a new coupon equals (n − i) ∕ n in this phase. Therefore, holds and we have

With

the first statement follows.

Let the n different types of coupons be . For the proof of the second statement we define random events S i, t for and , where S i, t denotes the event that no coupon if type i was among the first t coupons. Clearly, we have

It is easy to connect the number of coupons obtained at the end with these random events. As long as at least one type of coupon has not yet been obtained the collection is incomplete. This translates to

and we want to estimate . The simplest way we can do this is making use of the union bound. This yields

and we can set t : = cnlnn. This way we obtain

and prove the second statement. Interestingly, for the proof of the third statement no new ideas are needed. We only replace our application of the union bound by much more careful calculations.

We know about the principle of inclusion and exclusion. Applying this, we have for an arbitrary collection of sets

and obtain the Bonferroni inequalities.

We apply these inequalities to derive a better bound for . In order to simplify notation a bit we define

Using this notation and setting t : = nln(n) + cn we have

and, more useful,

with such that n ∈ { 2k, 2k + 1} holds. Obviously, we need to determine q i . Remember that we have

and observe that

holds for symmetry reasons since the probabilities are equal for all types of coupons. The notation describes the event that among the first t coupons none of the coupons h 1, h 2, …, h i has been obtained. Thus we have . We set t : = nln(n) + cn and have

Before we plug this in and continue, we make a short calculation that will help us when determining the limit value we are looking for. We have

with for each fixed i and see that therefore

holds. We keep this in mind when considering . We have

and see that

holds. In we recognize the Taylor series of and follows.

Lemma 5.3 shows that in the coupon collector’s problem the probability distribution is extremely concentrated around its mean value. Such results are referred to as sharp threshold results. Figure 5.4 shows the graph of to give a more vivid impression for the third statement .

Fig. 5.4

Graph of the function illustrating the sharp threshold result for the coupon collector’s problem

We want to apply Lemma 5.3 to obtain a sharper result for the optimization time of the (1+1) EA. In order to do so we need to connect the coupon collector’s scenario to the random process the (1+1) EA optimizing some function  corresponds to. We will consider ONEMAX as a concrete example. It is not immediately obvious how this connection can be made. Let us for a moment consider random local search (RLS) instead of the (1+1) EA. Let us assume in addition that the initial bit string is . Now the connection is pretty obvious. The n different types of coupons correspond to the n bits. In each round of the algorithm exactly one bit is selected uniformly at random and flipped. This corresponds to obtaining a coupon of this type. The selection ensures that bits in x t are only changed from 0 to 1 and are never changed back. Thus, a bit has value one if and only if ‘we have obtained a coupon of this type.’ Optimization ends if all bits are set to 1; this corresponds to having obtained at least one coupon of each type. We recognize a direct correspondence between this scenario and the coupon collector’s problem. With this observation in mind, we want to transfer results from the coupon collector’s problem to the (1+1) EA on ONEMAX.

One difference is that the (1+1) EA does not start with x 0 = 0 n but with a chosen uniformly at random. Thus we have a random number of M = { ONEMAX}(x 0) 1-bits in the beginning. This corresponds to M different coupons that the collector receives before actually starting to collect. This changes the number of coupons needed from n to n − M—but each different type of coupon is still obtained with equal probability 1 ∕ n. This makes this a rather minor change that changes the expected number of coupons needed from to . With probability we have M < (2 ∕ 3)n (Chernoff bounds). For a lower bound we can replace ln(n − M) by ln(n − (2 ∕ 3)n) = ln(n) + ln(1 ∕ 3). We see that in the bound nln(n) + cn we need not change the nln(n) and can adjust the statement by replacing c with an appropriate c′ : = c + ln(1 ∕ 3). Since the third statement in Lemma 5.3 holds for arbitrary values of c this is no problem.

There is another difference when considering the (1+1) EA. While RLS flips exactly one bit in each mutation, the (1+1) EA flips a random number  of bits. We do not have a problem with mutations flipping no bits at all since they can only increase the number of steps needed. Since we want to prove a lower bound we are free to simply ignore this effect. Neither are mutations flipping exactly one single bit a problem. They correspond exactly to the coupon collector’s problem. But mutations where B > 1 bits mutate are a difficulty for us. We can think of such mutations flipping exactly B bits as B consecutive single bit mutations. The expected number of mutating bits in each generation equals 1. Thus the expected number of mutations does not change if we replace such B bit mutations with B consecutive mutations of single bits. We can apply Chernoff bounds to have this with probability in the generations we consider. We compensate for the slight multiplicative increase inferred by this method by taking into account mutations that do not flip a single bit. Since these mutations occur with probability this is no problem. The real difficulty is that B consecutive mutations of single bits differ from a B bit mutation in an important way. In a B bit mutation we are guaranteed to see B different bits flipped. In B consecutive mutations of single bits the very same bit may flip more than once. Since we look for the first point of time when each bit is flipped at least once this difference ‘goes in the wrong direction.’ We overestimate the time if we do not take this into account. When we consider B consecutive mutations in each mutation each bit is flipped with probability 1 ∕ n. If we consider a B bit mutation and imagine that the B bits that flip are selected randomly one after the other then the probabilities are different. In the first step each bit is selected with probability 1 ∕ n. In the second step each bit is selected with slightly larger probability 1 ∕ (n − 1) since we can be certain that the first will not be selected again. This probability increases with each bit reaching 1 ∕ (n − B + 1) for the B{ th} bit. Now we decide on a bound b and consider only mutations of at most b bits. If the probability not to have any mutation flipping B > b bits in the generations we consider is sufficiently close to 1 we have no problem. Since we do not want the number of steps changed significantly from the coupon collector’s problem we need (since then ). The probability for a mutation flipping more than b bits is bounded above by

Setting we see that we do not have any problem. Making use of the third statement of Lemma 5.3 this way we proved the following theorem.

Theorem 5.12.

Let be a function with a unique global optimum

Proof.

The line of reasoning above was only concerned with ONEMAX. When we replace ONEMAX by an arbitrary function with a unique global optimum we do not make a significant change. The proof for ONEMAX only relies on the fact that with large probability there are bits that all need to flip at least once. This holds for such functions f, too.

5.3 Typical Events

When discussing the general lower bound (Theorem 5.11) we introduced a rather small technical lemma. In some way, this lemma is the first method for proving lower bounds on the expected optimization time we discussed. It precedes the general lower bound and is applied in its proof in a typical way. For the sake of convenience we repeat the lemma here.

Lemma 5.2.

Let T be a non-negative random variable, B some event. .

We apply Lemma 5.2 in the proof of Theorem 5.11 in two places. For large population sizes μ we consider as B the event that the first search points that are generated uniformly at random are all different from the unique global optimum x  ∗ . For small population size μ we consider as B the event that we have at least n ∕ 3 bits different from the unique global optimum x  ∗  in all members of the initial population and that after (n − 1)lnn steps in each member of the population there is at least one of these bits that never flipped. Both events are in some sense typical. In a ‘normal’ run we expect to see these events. This is usually the way Lemma 5.2 is applied to prove lower bounds on the expected optimization time. Since this method is fairly simple to apply and often useful we give it a name, method of typical events, and consider a few examples in this section.

We already considered the function { JUMP} k and proved an upper bound of by means of fitness layers. Here we can prove that this bound is asymptotically tight by the method of typical events.

Theorem 5.13.

Let and be given.

Proof.

The upper bound is given by Theorem 5.8. The lower bound follows from Theorem 5.11 since { JUMP} k has the unique global optimum 1 n . Thus, it suffices to prove . We do this using the method of typical events. As typical event B we consider the following event.

We consider the event that at some point of time the current search point x t is left of the gap. If this is the case we consider the very last mutation leading to the global optimum 1 n . In this mutation exactly all 0-bits in the current search point need to be mutated simultaneously. Each x ∈ { 0, 1} n with has smaller function value than this x t . Thus, after the t{ th} generation such search points can never become the current search point of the (1+1) EA. We conclude that in the very last mutation there are at least k 0-bits that all need to be mutated simultaneously and no other bit may flip. Such a mutation has probability at most . We conclude that holds. Now it suffices to prove that holds. We do this by making a case distinction with respect to the value of k.

For k ≤ n ∕ 3 we have

by application of Chernoff bounds. For k > n ∕ 3 we need to say a bit more. We still have . We see that the probability to mutate to the unique global optimum 1 n is bounded above by 1 ∕ n n ∕ 3 and can only decrease with time since the fitness increases with increasing Hamming distance to the all ones bit string in the part of the search space. We know from Theorem 5.2 that holds. Thus, there is a constant c > 0 such that . Applying Markov’s inequality we have . We can apply the method of fitness layers using the trivial -based partition like in the proof of Theorem 5.2 to prove that the same bound cnlogn holds for the expected number of steps the (1+1) EA needs to reach some x t with { ONEMAX}(x t ) ≤ n − k given that it does not reach the unique global optimum 1 n before. We apply the union bound and see that the probability to reach the unique global optimum in 2cnlogn steps is bounded above by (2cnlogn) ∕ n n ∕ 3 in this situation. Thus, we have

in this case. Thus we have in all cases, and

follows.

We consider one other example function, called RIDGE. It is similar to ONEMAX and LEADINGONES and has some interesting properties that we discuss in the next section. Here we only give its definition and concentrate on upper and lower bounds for the expected optimization time of the (1+1) EA.

Definition 5.6.

The fitness function is defined by

A graphical representation of RIDGE is given in Fig. 5.5. The name of the function is motivated by the ridge of increasing function values 1 i 0 n − i .

Fig. 5.5

Graphical representation of

Theorem 5.14.

Proof.

For the upper bound we consider the trivial RIDGE-based partition. For all layers L i with i < n the situation is essentially equivalent to ONEMAX with swapped roles of 0-bits and 1-bits. After this the situation is essentially equivalent to LEADINGONES. We conclude that we have an upper bound of on the expected optimization time.

For the lower bound we use the method of typical events and consider the following event B. Let T  ∗  denote the first random point of time when the ridge is found, i.e., . Let B 1 denote the event that holds. Let B 2 denote the event that after T  ∗  no mutation that mutates more than 3 bits generates an offspring y that replaces its parent x t (with t ≥ T  ∗ ). Let B denote the event that both, B 1 and B 2, happen.

We consider B 1. We have by application of Chernoff bounds. Since the function value decreases with an increasing number of 1-bits we have that { ONEMAX}(x t ) ≤ { ONEMAX}(x 0) holds for all x t with t < T  ∗ . Now we consider the final mutation leading to some point 1 i 0 n − i on the ridge. The probability that in this mutation at least n ∕ 12 bits are mutated is bounded above by

Thus, with probability the number of 1-bits in this first point on the path is bounded above by (2 ∕ 3)n + (1 ∕ 12)n = (3 ∕ 4)n. This prove .

We consider B 2. A mutation of x t to y such that y replaces x t and holds can only be a mutation that flips a sequence of the leftmost 0-bits to 1-bits. The probability for flipping i such bits equals . We see that

holds.

We are interested in . The event B happens if neither nor happen. Making use of the union bound and have .

With it suffices to prove that holds. We have some with some i ≤ (3 ∕ 4)n and t ≥ T  ∗ . We need to prove that the expected number of steps needed to reach the unique global optimum 1 n is . Due to B 2 (and thus due to B) we can assume that in each mutation the number of leading 1-bits is increased by at most 3. Thus at least n ∕ 12 mutations increasing the number of leading 1-bits are needed. The probability for such a mutation is bounded above by 1 ∕ n since the leftmost 0-bit needs to flip. We consider n 2 ∕ 25 steps and see that we expect to see at most n ∕ 25 such mutations. Making use of Chernoff bounds, we see that with probability we have at most 2(n ∕ 25) < n ∕ 12 such mutations. Thus,

holds.

In the part that deals with the proof of the lower bound for the (1+1) EA on RIDGE the proof for was a bit involved. In the next section we discuss another method for proving lower bounds that will allow for a much more elegant proof.

5.4 Drift Analysis for Lower Bounds

The function RIDGE is in some sense similar to LEADINGONES. After reaching the first point on the ridge 1 i 0 n − i with the function value can only be increased in mutations that flip the leftmost 0-bit into a 1-bit. Since such a mutation has probability at least (1 ∕ n)(1 − 1 ∕ n) n − 1 ≥ 1 ∕ (en) and since at most n such mutations are sufficient to reach the unique global optimum an upper bound of en 2 for the (1+1) EA follows. This holds for LEADINGONES, too. For the lower bound we showed that with probability the number of 0-bits when the ridge is first encountered is . Moreover, it is easy to see that it is highly unlikely to make large progress in a single mutation since the probability to increase the function value by k is bounded above by 1 ∕ n k . This leads to the bound in Theorem 5.14. We would like to have a similar bound for the (1+1) EA on LEADINGONES. There, however, things are less clear. For RIDGE it is clear by the definition of the function that all n − i bits following the i leading 1-bits are 0-bits. For LEADINGONES, this is only clear for x[i + 1]. All other bits may have any value. In the extreme case we have 1 i 01 n − i − 1, and we can increase the function value by n − i and reach the global optimum with a mutation of a single bit having probability . These n − i − 1 trailing bits are a source of random progress that we cannot control with the methods we discussed so far. In order to be able to prove a tight lower bound on LEADINGONES we now consider a new proof technique that is very flexible and powerful and will prove itself very useful in many situations.

The idea is to introduce a measure of progress and bound the average progress in a single step. This way the average optimization time can be bounded from below. Consider some evolutionary algorithm A. Let Z denote the set of all populations. Remember that holds if the evolutionary algorithm A has a population size of μ and uses a fitness function (see Sect. 3.1). In order to apply drift analysis one needs to define some distance measure  such that d(P) = 0 holds if and only if the population P ∈ Z contains a global optimum. Thus, we have if P t denotes the population after exactly t function evaluations have been made. We are interested in the decrease of the distance after a single function evaluation, i.e., . The expected value of this decrease  is called drift. Since we want to prove a lower bound on T A, f , we assume a point of view that tends to overestimate the progress and consider the maximum drift . This maximum drift can be described as the maximum expected speed the evolutionary algorithm A has when optimizing the fitness function f. If the initial distance happened to be M, one could speculate the expected optimization time is bounded below by M ∕ Δ. The following drift theorem proves that this is indeed the case.

Theorem 5.15.

Let A be some evolutionary algorithm, a fitness function, Z the set of populations, P t the current population after exactly t function evaluations, P 0 the initial population, a distance measure with d(P) = 0 if and only if P contains a global optimum, ,

Proof.

Consider . Note that while we assume that is positive, a specific decrease in distance D t may still be negative indicating an actual increase in distance. We have

since all but the very first and the very last term cancel each other out in this telescoping sum and holds. Thus

holds. We can make use of the law of total probability to obtain

where the last equality follows from the linearity of the expectation. Since we consider the conditional expectation given that T A, f  = t holds, we can write

For each value of i the term appears exactly once for each value of t. We can thus change the ordering of the summation and obtain

Now we make use of the law of total probability again (this time applied to conditional probabilities) and have

where the last equality holds since is independent of t. The event T A, f  = t is a subset of the event T A, f  ≥ i for all i ≤ t, and the intersection of the two events equals T A, f  = t. Thus, we can replace by and obtain

For all t < i we have , thus we can have the inner sum start at t = 1 without changing anything, i.e.,

Applying the law of total probability, we see that we can simplify this to obtain

By definition we have so that

holds. Since T A, f can only assume integer values we have

and realize that

holds by definition of the expectation. Plugging this in we obtain

so that for Δ > 0 we have

as claimed.

While the proof of the drift theorem (Theorem 5.15) was lengthy, it could be carried out by of sequence of steps where each step of the proof was quite elementary. We see that the statement made in the drift theorem is very intuitive and there is nothing really difficult in it. Now we convince ourselves that it is also easy to apply. We start with a simple example that highlights an inherent weakness of drift analysis.

In the drift theorem (Theorem 5.15) a very pessimistic estimate of the drift is used. One considers the maximal expected decrease in distance in a single generation where the maximum is taken over all possible populations. Thus, if the expected decrease in distance is not sufficiently uniform across populations quite weak bounds may be the result. To see this more clearly we consider a simple random process that is interesting in its own right. In order to be able to judge the quality of the bound that we obtain using drift analysis we perform an exact analysis first. This exact analysis makes use of martingales and the optional stopping theorem (Theorem A.3; see Appendix A.3).

Consider the following random process with for all . The process starts with with . It can be described as the random walk of a particle moving in integers starting at i. In each step the particle is moved with equal probability one step left or right. Thus, we have and for all . We are interested in the first point of time when either − n or n are reached, i.e., . We refer to this random process as the fair random walk on the integers.

Lemma 5.4.

Consider the fair random walk on the integers started in X 0 = i with . For the stopping time holds.

Proof.

We define the random process on by for all . We stop the process when the random process is stopped. Clearly, T is a stopping time for , too. We claim that is a martingale. The following simple calculation

proves that this indeed the case.

We have

and follows.

It is easy to see that holds. There is always a sequence of at most n random movements that causes the process to stop. It suffices to move the particle to the close ‘end’ to achieve this. Such a sequence occurs with probability at least (1 ∕ 2) n and we have as immediate consequence. Since we have

follows and we can apply the optional stopping theorem (Theorem A.3) to . This yields

and we have as claimed.

Let us consider another fair random walk that operates on the non-negative integers. We define Z 0 = i with , for all and again have the particle with equal probability move either left or right if it is not yet at the end 0. In this case it is moved to the right and reaches 1 with probability 1. Formally, we have for Z t  > 0, and otherwise. We are interested in the first point of time when n is reached, i.e., . We refer to this random process as the fair random walk on the non-negative integers.

Lemma 5.5.

Consider the fair random walk on the non-negative integers started in Z 0 = i with . For this random walk consider the stopping time

Proof.

We consider the fair random walk on the integers . We define the random process . Clearly, and have identical stopping times. We observe that coincides with .

We see that the fair random walks on the integers and the non-negative integers are essentially the same random process. This can be illustrated and thus becomes intuitively clear (see Fig. 5.6). The transformation from to in the proof of Lemma 5.5 corresponds to ‘folding’ the negative numbers onto the positive ones.

Fig. 5.6

Graphical representation of the fair random walk on (left), on (right), and a figure illustrating their equivalence (middle)

Now we consider the fair random walk on the integers started in X 0 : = 0 and want a bound on by means of drift analysis. Remember that holds. We use a very simple distances measure d that directly measures the distance to one of the ends and define . Clearly, 0 ≤ d(X) ≤ n holds and we have d(X) = 0 if and only if the process stopped. Thus, d really is a distance measure. For almost all we have

and we may be tempted to believe that we have Δ = 0 and we do not get a bound at all. This, however, is not true. The crucial equation does only hold for almost all values of i. The exception is i = 0. For X t − 1 = 0 we have . For the proof of a lower bound on we can thus use Δ = 1 and have . Clearly this is a correct but rather weak bound. It coincides with the trivial lower bound n. We cannot decrease the initial distance of n in less than n steps since the step size equals 1.

The example of the fair random walk on the integers shows that we cannot expect drift analysis to work in arbitrary situations. At least we need to come up with an appropriate distance measure. The simplest distance measure yields the distance in function value to the global optimum. We can expect this distance measure to yield good bounds if the probability for an increase in fitness is sufficiently uniform across all populations. For the (1 + λ) EA on LEADINGONES this is the case since the probability for an increase in function value is bounded above by 1 ∕ n and below by , so it is for all populations. Since currently we only have an upper bound, we aim at proving a lower bound using drift analysis. We start with a lower bound for the (1+1) EA even though it is only a special case. This way we can practice the application of the drift theorem (Theorem 5.15) at a simpler example first.

Theorem 5.16.

Proof.

The upper bound follows from Theorem 5.10. For the lower bound we apply the drift theorem (Theorem 5.15) and define the distance measure . Since we have 0 ≤ d(x) ≤ n and d(x) = 0 for x = 1 n this d really is a distance measure. We need to estimate the initial distance and the maximum drift .

For the expected initial distance we observe that we have a distance of exactly n − i if and x 0[i + 1] = 0 holds. This is the case with probability 1 ∕ 2 i + 1. Thus, we have

The probability for a decrease in distance is bounded above by 1 ∕ n since it is a necessary condition that the leftmost 0-bit is flipped. In the case the distance is decreased by 1 due to this bit and in addition by k if exactly k bits to the immediate right of the leftmost 0-bit are 1-bits in the offspring. We need an upper bound in the decrease in distance. In the worst case the distance may be decreased by n. We cannot afford to be this pessimistic since this yields only a lower bound of . We see that we need to find an upper bound for the expected decrease in distance.

We consider the bits that are to right of the leftmost 0-bit. These bits never had any influence on selection since selection depends on fitness only and the bits that are to the right of the leftmost 0-bit do neither contribute to the fitness nor did they ever contribute to the fitness. We claim that this implies that they are distributed uniformly at random. Clearly, this is the case after random initialization. So we claim that standard bit mutations alone do not change anything about this uniform distribution. More formally, our claim is the following. Consider an arbitrary x ∈ { 0, 1} n . Let { mut}(x) denote the outcome of standard bit mutations applied to x. Consider y ∈ { 0, 1} n with y : = { mut}(x). We claim that

holds. For our y = { mut}(x) and an arbitrary z ∈ { 0, 1} n we have

The key observation is that for arbitrary x′, z ∈ { 0, 1} n . Note that this holds for standard bit mutations and does to need to be the case for arbitrary mutation operators. Using this we have

as claimed.

This implies that the expected decrease in distance in a single mutation is bounded above by

by multiple application of the geometric sum. With and Δ < 4 ∕ n we have a lower bound of .

The proof of the lower for the (1+1) EA on LEADINGONES can be generalized to the (1 + λ) EA on LEADINGONES. While the calculations are slightly more tedious no new idea is needed. It is, however, remarkable that the size of the offspring population has a noticeable impact on the expected optimization time.

Theorem 5.17.

Let and with be given.

Proof.

We make the following change to the method of drift analysis since it simplifies our notation. Instead of having x t denote the current population after t function evaluations we denote by x t the current population after tgenerations. We make up for this change by multiplying the result we obtain by λ, since in each generation exactly λ function evaluations are made.

Since we consider the (1 + λ) EA we have Z = {0, 1} n as the set of all populations. We use d(x) : = n − { LEADINGONES}(x) as distance measure like we did in the proof of Theorem 5.16. We know from this proof that { E}d(x 0) > n − 1 holds. So what we need is a lower bound on the drift in one generation. We make a case distinction with respect to the offspring population size λ and distinguish small offspring population sizes λ ≤ n from large offspring population sizes λ > n.

We begin with considering a small offspring population size λ ≤ n. We need to find an upper bound for the expected decrease in distance in one generation. This equals the expected increase in the number of leading ones in one generation. Clearly, the increase in the number of leading ones is an integer value, so we can compute its expectation by . We can start the summation at 1 since due to plus-selection the number of leading ones cannot decrease. We can end the summation at n since the increase in the number of leading ones is trivially bounded above by n. We know from the proof of Theorem 5.16 that for a single mutation the probability to increase the number of 1-bits by at least i is bounded by 1 ∕ (n2 i − 1). We apply the union and obtain λ ∕ (n2 i − 1) as upper bound on the probability to have an increase by at least i in one generation. Thus, we have

as upper bound. Together with this yields

for small offspring population sizes λ ≤ n.

For large offspring population sizes λ > n we also use

but split the sum into two sums. For i − 1 < 2log(λ ∕ n) we simply use 1 as trivial upper bound on the probability. For larger values of i we note that holds. Together this yields

(2)

where the last inequality holds since we have λ > n. Together with the result on the expected initial distance, , this yields

for large offspring population sizes λ > n. Note that for λ > n we have . Thus, we can combine the two bounds and have

for all offspring population sizes.

As last example for the application of drift analysis to derive lower bounds on the expected optimization time we come back to the (1+1) EA and consider . Recall that (with n = hi) is a class of functions that contain a long k-path P k n of length and are defined like ONEMAX for all points not on the path so that the first path point is easy to find. We have an upper bound of on the expected optimization time for the (1+1) EA (Theorem 5.7). The term in the upper bound refers to the (1+1) EA optimizing { LONGPATH} k by following the path. The term n k + 1 ∕ k in the upper bounds refers to the (1+1) EA optimizing { LONGPATH} k by taking shortcuts requiring at least k specific bits to be mutated simultaneously. It depends on the value of the parameter k which route of optimization is faster. For following the path becomes faster due to the path length being decreased with increasing values of k. We concentrate on this case here.

Theorem 5.18.

Let and with and be given.

Proof.

The upper bound follows from Theorem 5.7 so that we only prove the lower bound here. We do this with help of the drift theorem (Theorem 5.15). We first consider the situation assuming that the search is not started with x 0 ∈ { 0, 1} n chosen uniformly at random but with x 0 = 1 n , the first point on the long k-path P k n .

We measure the distance by considering the distance to the unique global optimum in function value, thus we use as distance measure. Clearly, d is actually a distance measure.

Since we consider the (1+1) EA and measure the distance in function value, the distance cannot increase. It can be decreased in two different ways. If in a mutations the number of mutated bits equals b and we have b < k the decrease in distance is bounded above by b. Moreover, the probability to make distance b is since there is exactly one mutation leading to a bit string with larger function and Hamming distance b (Lemma 5.1). If, on the other hand, for the number of mutated bits b ≥ k holds, a shortcut may be used. We use a very weak estimate here and bound the decrease in distance from above by . This way we get as upper bound on the drift

where the last equality holds since we have , and thus

and therefore

follows. Assuming that the (1+1) EA starts in the first path point 1 n , we have as initial distance and application of the drift theorem (Theorem 5.15) yields as lower bound on the expected optimization time. Obviously, this bound still holds if the (1+1) EA is started in an arbitrary point on the long k-path P k n that belongs to the first half, but it does not necessarily need to be the very first one.

In order to derive a lower bound on the we apply the method of typical events. The typical event B we consider is that the first point the (1+1) EA encounters on the long k-path P k n belongs to the first half. We have already seen that holds. Thus, it suffices to prove that holds.

Let H 1 denote the first half of the path points. Due to symmetry we have . Due to the definition of { LONGPATH} k (Definition 5.4) the Hamming distance to points in the first half cannot be increased while the Hamming distance to points in the second is reduced. This is due to the fact that the first k bits of path point determine if the point belongs to the first or second half of the path. Definition 5.4 rewards these bits being equal to points in the first half. Thus, holds, and follows.

5.5 Drift Analysis for Upper Bounds

With Theorem 5.15 we have a flexible and powerful method for proving lower bounds on the expected optimization time . It is, of course, desirable to have something similar for upper bounds, too. Reconsidering the proof of Theorem 5.15 it is not difficult to see that drift analysis can easily be extended so that it becomes a method for proving upper bounds, too.

Theorem 5.19.

Let A be some evolutionary algorithm, a fitness function, Z be the set of populations, P t the current population after exactly t function evaluations, P 0 the initial population, a distance measure with d(P) = 0 if and only if P contains a global optimum, ,

Proof.

We reconsider the proof of Theorem 5.15. We recognize two differences in the theorem itself. First, Δ is defined to be the minimum drift instead of the maximal drift. Second, we have an upper bound on the expected optimization time, not a lower bound. We conclude that as long as we have exact equalities and as long as Δ is not involved, nothing changes. Thus, we have

here, too. Using we obtain

and have

like in the proof of Theorem 5.15. Together this yields for Δ > 0 as claimed.

Theorem 5.10 states that holds. We proved this by means of LEADINGONES-based partitions. For λ = n we have as a special case of Theorem 5.10. Since the method of fitness layers depends on the evolutionary algorithm employingplus-selection this result cannot be transferred to the (1, n) EA using the same method of proof. But we can prove the result using drift analysis without any difficulties.

Theorem 5.20.

Proof.

As we did in the proof of Theorem 5.17 we consider the drift in one generation and multiply the result by the offspring population size λ to make up for this. We also make use of the same distance measure, namely d(x) = n − { LEADINGONES}(x). Clearly, holds. Now we need a lower bound on . The condition T (1,  n) { EA}, { LEADINGONES} > t translates to the x ≠ 1 n for the current population x. Under this condition we need a lower bound on the expected increase in fitness in one generation. We make the following very rough estimation. The probability to increase the function value by exactly one in a single mutation equals (1 ∕ n)(1 − 1 ∕ n) n − 1 ≥ 1 ∕ (en). Thus, with probability 1 − (1 − 1 ∕ (en)) n  ≥ 1 − e  − 1 ∕ e this happens in a single generation. The fitness of the population cannot decrease in a single generation if among the n offspring there is at least one copy of the current population. Such a copy is produced with probability (1 − 1 ∕ n) n  ≥ 1 ∕ (2e). Thus, the probability to decrease the fitness of the population in a generation is bounded above by (1 − 1 ∕ (2e)) n . If the fitness decreases this decrease cannot be larger than n − 1 since we have { LEADINGONES}(x t ) < n for the current population x t and { LEADINGONES}(x) ≥ 0 for all x ∈ { 0, 1} n . Putting this together, we have

(0.2)

for sufficiently large values of n (in this case for all n > 10, but this is not important). We thus have Δ > 0. 2 and obtain n ∕ 0. 2 = 5n as upper bound on the number of generations. This implies that

holds.

The proof of (Theorem 5.10) uses trivial fitness layers. This implies that only steps increasing the fitness by 1 are taken into account. With large offspring population size λ, however, it is conceivable that the fitness is often considerably increased in larger steps, reducing the expected optimization time. Making use of drift analysis we can easily prove that this is indeed the case. This way we can close the gap between the lower bound proven by means of drift analysis (Theorem 5.17) and the weak upper bound proven by means of fitness layers (Theorem 5.10). This demonstrates a strength of drift analysis in comparison to the method of f-based partitions.

Theorem 5.21.

Let and with be given.

Proof.

The lower bound follows directly from Theorem 5.17. For the proof of the upper bound we reconsider this proof of the lower bound. The key observation is that we bounded the probability for a decrease in distance by exactly i in one single mutation from above by (1 ∕ n)(1 ∕ 2) i − 1. This is an upper bound because it takes into account two necessary conditions for such an event, namely that the leftmost 0-bit needs to be flipped (with probability 1 ∕ n) and that i − 1 bits need to set to 1 (independently, each with probability 1 ∕ 2). For a lower bound on this probability we can additionally take into account that the leading 1-bits must not flip (independently, each with probability 1 − 1 ∕ n, at most n − i − 1 many, so with probability (1 − 1 ∕ n) n − i − 1 ≥ 1 ∕ e) and that there may be another bit following these i − 1 bits that needs to be a 0-bit so that the distance is decreased by no more than i (with probability 1 ∕ 2). Thus, we have an upper bound of e  − 1(1 ∕ n)(1 ∕ 2) i and see that this deviates from the lower bound only by a constant factor. It is easy to see that this constant factor does not change the order of growth of the drift. Thus, the upper bound follows in the very same way as the lower bound.

Our next application of the drift theorem (Theorem 5.19) will be for the expected optimization time on a set of functions, namely we consider the class of linear functions. Before we do this, we state a lemma that reduces the technical burden that we have to carry in the proof a bit.

Lemma 5.6.

Let A be some evolutionary algorithm, a fitness function, Z be the set of populations, P t the current population after exactly t function evaluations, P 0 the initial population, a distance measure with d(P) = 0 if and only if P contains a global optimum, the m{ th} Harmonic number.

Proof.

Using the distance measure d we define another distance measure d′(P) : = H d(P). Since H d(P) = 0 holds if and only if d(P) = 0 holds we see that d′ actually is a distance measure. For any k ≥ l we have

For any k < l we have

so that holds for any k and l.

Using this and the definition of d′ we have

since d(P t − 1) is not random given that we consider the conditional mean using P t − 1 as condition. Now we make use of the assumption made in the lemma on , and see that

holds. Applying the drift theorem (Theorem 5.19) to d′ with Δ ≥ 1 ∕ δ we obtain

as claimed. Note that δ does not need to be a constant, and in particular may depend on n.

Remember that we have for any linear function (Theorem 5.9). If we consider linear functions with w i  ≠ 0 for all i > 0 we have a lower bound on the expected optimization time of due to Theorem 5.11 since such linear functions have a unique global optimum. Thus, the best we can hope for with respect to an upper bound is , something that we proved to be true for the very simple linear functionONEMAX. The next theorem shows that the same upper bound holds for all linear functions.

Theorem 5.22.

Let be a linear function.

Proof.

Like in the proof of Theorem 5.9, we consider without loss of generality a linear function where all weights are positive and sorted in decreasing order with and w 0 = 0.

In order to simplify notation a bit we assume that n is even. Extending the proof to odd values of n is simple. We are interested in the bits of x t − 1, x t , and y and introduce the following notation. Let be the set of indices of the n bits. For I′ ⊆ I and b ∈ I let . Using this notation we define the indices of the bits in the left half L : = I n ∕ 2 and the indices of the bits right half R : = I ∖ L. For a bit string a ∈ { x t − 1, x t , y} we define the set of its 0-bits and the set of its 1-bits A (1) : = I ∖ A (0).

Using this notation we define the distances measure by

We have 0 ≤ d(a) ≤ (3 ∕ 2)n for any a ∈ { 0, 1} n and d(a) = 0 if and only if a = 1 n holds. Due to our assumptions on f we see that 1 n is the unique global optimum of f and d indeed is a distance measure.

We need to bound from below. We can apply Lemma 5.6 and see that it suffices to prove that holds in order to have .

In all cases where x t − 1 = x t holds, the contribution toward the drift is 0. Thus, it suffices to consider the cases where x t − 1 ≠ x t holds. Clearly, a sufficient condition for to hold is the offspring y is produced by flipping exactly one 0-bit in x t − 1. Thus, we have

Each 0-bit in x t − 1 contributes to d(x t − 1) either one (if its position is in R) or two (if its position is in L). Thus, we have and see that

holds. This implies that it suffices to prove that the conditional decrease in distance is bounded below by some positive constant c.

We consider some x t − 1 ≠ 1 n since otherwise optimization is already complete. We consider some . Note that f(x t ) ≥ f(x t − 1) holds since the (1+1) EA employsplus-selection. Thus, there is at least one bit changing its value from 0 to 1, i.e., . We make a case distinction with respect to the position of the leftmost of these bits.

First we consider the case . The leftmost bit flipping from 0 to 1 contributes 2 to the decrease in distance in this case. Let be the number of 1-bits with positions in L that may flip while we still have x t − 1 ≠ x t . Let be the number of 1-bits with positions in R that may flip while we still have . The expected contribution of these l + r bits is bounded above by − (2l + r) ∕ n since each bit flips with probability 1 ∕ n. Note that it is only an upper bound since it is not clear that all l + r bits (or arbitrary subsets of these) may flip simultaneously while still allowing for x t − 1 ≠ x t . Since we have l < n ∕ 2 and r ≤ n ∕ 2 the total contribution is more than 2 − (n + n ∕ 2) ∕ n = 1 ∕ 2. As discussed above this is sufficient.

Now we consider the case A t − 1 (0) ∩ A t (1) ∩ L = ∅. Since we only consider cases where x t − 1 ≠ x t holds, we have . We cannot make use of the simple estimation used for the other case. Since the leftmost bit flipping from 0 to 1 now contributes only 1, the simple bound now yields 1 − (n + n ∕ 2) ∕ n =  − 1 ∕ 2, which is insufficient. We need to be more careful and make better use of the fact that we only have nonzero contributions in the cases where holds. To do that we make another case distinction and consider the number g of bits flipping from 0 to 1, i.e., . Remember that holds in the case that we currently consider. Let l and r be defined like in the first case. Note that l and r depend on g. The key observation is that l ≤ g holds: If more bits in the left half flip from 1 to 0 than bits in the right half flip from 0 to 1 we cannot have since the weights are sorted. Thus, the expected contribution is bounded by

(at least for n > 4). Since we have a contribution that is bounded below by 0. 1 in all cases, the result follows by application of Lemma 5.6.

Theorem 5.22 proves the general lower bound (Theorem 5.11) to be tight in surprisingly many cases. We state this consequence of combining Theorem 5.22 with Theorem 5.11 as a corollary.

Corollary 5.2.

Let be a linear function with f(x) = w 0 with and

The only observation one needs to make to prove Corollary 5.2 is that such linear functions have a unique global optimum. It is not difficult to extend Corollary 5.2 to linear functions with nonzero weights.

5.6 Typical Runs

Sometimes when considering an evolutionary algorithm on an example problem it seems to be pretty clear how the evolutionary algorithm will function on this problem. And yet, it may be the case that none of the methods discussed so far are sufficient to actually prove this. While this is annoying in general in some cases there is an escape from this dilemma. A clear understanding how an evolutionary algorithm functions can sometimes be translated into an actual proof quite directly. We discuss this in an abstract way for a proof of an upper bound first. After that we consider a concrete example.

The starting point is a clear picture in our minds how an evolutionary algorithm A performs on an example problem f from the beginning to the end when some global maximum is found. We formalize this understanding by describing phases the evolutionary algorithm goes through. The random length of the i{ th} phase is denoted by . Since the phases are described in a way that they cover the complete run holds. Now we make our understanding of the algorithm’s behavior on f more precise by defining upper bounds T i on the length  of the i{ th} phase for each of the phases. For each phase, there is a probability  that this upper bound does not hold, so we have . What we actually prove for each phase is an upper bound for this probability of a failure. Summing up these bounds we get some result on the optimization time:

Obviously, we need to hold. In some cases the algorithm ‘cannot get stuck’ when operating on f in the following sense. Whenever the algorithm deviates from the behavior that we expect things are still not worse than if the algorithm was stopped and restarted. If we have this, that any failure is no worse than a restart, then

holds if we have ∑_{i}p i  < 1: We have as lower bound on the probability that a run is successfully finished within steps. If a run is not successful in this way we may assume that the algorithm is restarted. The length of each failed run (before such a hypothetical restart) is also bounded by , obviously. Since a run is successful with probability at least , we have that the expected number of runs we observe before we have the first successful run is bounded above by . This yields the upper bound of on the expected optimization time . We make these abstract considerations concrete by considering an example.

We consider the (1+1) EA on an example function that contains a long but not overly large plateau. A plateau is a set of points P where for each pair of points a, b ∈ P there is a path of Hamming neighbors connecting a and b that is completely in P and where holds, i.e., all points on the plateau have equal function value. We consider the following example function PLATEAU that shares some properties with RIDGE.

Definition 5.7.

The fitness function is defined by

A graphical representation of PLATEAU is given in Fig. 5.7. The n points form the plateau. While the number of points in the plateau is rather small, the Hamming distance between the first and the last of point of the plateau equals n − 1 and is really large.

Fig. 5.7

Graphical representation of

We want to consider the (1+1) EA on PLATEAU and prove an upper bound on its expected optimization time . For an example of the method of typical runs we need as a starting point a clear understanding of how the (1+1) EA operates on PLATEAU. In general, an intuitive understanding serves as a starting point in these situations. Intuition, however, is a difficult thing that is likely to vary very much from person to person. Since we want to follow the example we take a route that is quite a bit more formal than something that would be usually done. We have seen in many examples thatrandom local search performs similar to the (1+1) EA when simultaneous mutations of several bits are not important. We speculate that this may be the case for PLATEAU. Note that PLATEAU is not unimodal since all points 1 i 0 n − i with are local optima but only 1 n is a global optimum. But all local optimal of PLATEAU that are not global optima can be escaped by single bit mutations since they are all neighbors of at least one other local optimum. Moreover, it is incorrect to assume that for unimodal functions simultaneous mutations of several bits cannot be crucial. The function { LONGPATH}3 is a clear counterexample (see Theorem 5.7). So we believe that RLS and the (1+1) EA perform very similarly on PLATEAU. We thus will develop an upper bound on as basis for the following application of the method of typical runs.

Theorem 5.23.

Proof.

We consider RLS on PLATEAU. Let T 1 denote the number of steps with . Let T 2 denote the number of steps with . Clearly, holds. Note that any run can be partitioned into two phases, where all steps of the first phase are counted by T 1 and all other steps are counted by T 2. The first phase is very similar to optimizing n − { ONEMAX}. The only difference is that the run may stop early, which has no relevance for a proof of an upper bound. Using a PLATEAU-based partition, it is easy to prove an upper bound .

In the second phase random local search is on the plateau 1 i 0 n − i with . The only way to leave this plateau is to reach the unique global optimum 1 n . We consider some on the plateau. For i = 0, i.e., , the only step that leads to some x t + 1 with is the step to 10 n − i . This step is taken with probability 1 ∕ n. For all other values of i there are two steps that imply . These are the steps to and to . The probability to take one of these two steps equals 2 ∕ n; each step is taken with probability 1 ∕ n. We now consider the probabilities under the condition that x t + 1 ≠ x t holds. The conditional probability for the move from 0 n to 10 n − 1 equals 1. The conditional probability for the move from 1 i 0 n − i to equals 1 ∕ 2. The conditional probability for the move from 1 i 0 n − i to equals 1 ∕ 2, too. The process stops when the unique global optimum 1 n is reached. We can characterize the situation simply by giving the current number of 1-bits i. Now this process is identical to the fair random walk on the non-negative integers. We know from Lemma 5.5 that the expected duration equals if the process is started in i. Remember that we considered conditional probabilities and that the random event that we used as condition has either probability 1 ∕ n (for 0 n ) or 2 ∕ n (for all other x t ). Thus, if the expected duration equals we have and thus . It is easy to see that we have . This is true since initially the number of 0-bits is bounded below by n ∕ 2 with probability 1 ∕ 2 and this number can only increase until the PLATEAU is reached. This implies , and follows.

We believe that the (1+1) EA is not very much different from random local search on PLATEAU. We make this more concrete and consider the same two phases as above, again denoting their lengths as T 1 and T 2. For the first phase we already know that holds for the (1+1) EA, too, since n − { ONEMAX} is a linear function (see Theorem 5.22). For T 2 we have to take into account that the (1+1) EA may make larger steps. Steps of size k have probability , and since we expect to have to consider steps we cannot ignore such steps completely. In steps at least steps of sizes up to 3 are quite likely to happen. On the other hand, steps of size are extremely unlikely, and thus we assume that the larger steps only have an influence that is negligible in Landau notation. Considering things a bit more closely, however, may raise some doubts. Consider some with i > n ∕ 2. The probability of moving away from the global optimum equals

whereas the probability of moving toward the global optimum equals

and is thus by

smaller. Thus we have a small drift away from the global optimum there. This drift is not present for random local search and is introduced by the possibility to have larger mutations. In particular, for i = n − 1 this drift is of size and it is not clear that it has no large impact. Clearly, this calls for a closer inspection and a formal proof. This is in line with our reasoning that knowing something is always better than merely believing it and that only formal proofs can provide us with guarantees. Clearly, we make use of the method of typical runs.

Theorem 5.24.

Proof.

We have using the notation from the proof of Theorem 5.23. Clearly, holds and it suffices to prove . We begin with a proof of the upper bound.

We consider only T 2 and thus start our considerations at the first point of them when holds for some . We consider cn 3 steps of the (1+1) EA, where c > 0 is a sufficiently large constant. We will be able to fix a concrete value for c later but, clearly, for proving as upper bound this is not important. Since we are proving an upper bound it is good to reach the unique global optimum 1 n early. Therefore, it does no harm at all to include x t  = 1 n when describing a ‘success.’

We consider these cn 3 steps on the plateau and call a mutation relevant if the algorithm moves on the plateau, i.e., , or holds. We now formulate five conditions on these cn 3 steps on plateau. By definition, on these conditions it will be clear that the optimum is reached if all conditions are satisfied. For each of these five conditions we prove an upper bound on the probability p i that is is not satisfied. Application of the union bound yields that all five conditions are satisfied with probability at least . In the case one of the conditions is not satisfied we are in a situation that is not worse if we had just reached the plateau. This is due to the fact that we make no assumptions about the first point on the plateau. This yields as upper bound on . With we have as claimed.

The first condition is that we have at least c′n 2 relevant mutations within the considered cn 3 steps, where c′ > 0 is a constant that we fix later. The probability for a mutation to be relevant is always bounded below by (1 ∕ n)(1 − 1 ∕ n) n − 1 ≥ 1 ∕ (en). Thus, the expected number of relevant mutations is bounded below by (c ∕ e)n 2. Chernoff bounds yield that with probability the number of relevant mutations is bounded below by c ∕ (2e)n 2. We define c′ : = c ∕ (2e) and see that we can have c′ arbitrarily large (but constant) by choosing c sufficiently large. Thus, we have .

The second condition is that there is no relevant mutation changing more than three bits, i.e., . The probability for such a mutation is bounded above by

where the last inequality holds for n ≥ 3, which is of course sufficient here. Making use again of the simple union bound, we have cn 3 ⋅3 ∕ n 4 = 3c ∕ n as upper bound for at least one such mutation in our cn 3 steps. Thus, holds.

The third condition is that we have at least c′n 2 relevant mutations that flip exactly one bit within the considered cn 3 steps, where c′ > 0 is a constant that we fix later. We remember our reasoning about the first condition. There we only took relevant mutations into account. Thus, we can set c′ : = c′ = c ∕ (2e) and have p 3 = 0 since this is not an additional condition—the failure probability is already included in p 1.

The fourth condition is that we have within the cn 3 considered steps. Note that due to the first three conditions we have at least c′n 2 relevant mutations of single bits in this phase and no relevant steps flipping more than three bits simultaneously. We consider a situation with with i < n − 3 since otherwise the fourth condition is already satisfied. Since no mutation we consider flips more than three bits, we have for every considered mutation that decreases the number of 1-bits an analogous mutation that increases it. Thus we have in this case. In fact, we have

for 3 < { ONEMAX}(x t ) < n − 3 and

otherwise (since we only have to deal with the case { ONEMAX}(x t ) < n − 3 here). We see that most of the time the process is equivalent to a fair random walk. During the steps it differs it is biased toward our target region {1 i 0 n − i ∣i ≥ n − 3}. Thus, the expected number of considered mutations until our target region is hit is bounded above n 2 (due to Lemma 5.5). Applying Markov’s inequality, we see that the probability not to hit the target region in the at least c′n 2 = (c ∕ 2e)n 2 steps that we have is bounded above by 2e ∕ c. We have p 4 ≤ 2e ∕ c and see that we can make p 4 arbitrarily small by increasing c, the constant in the number of steps cn 3 that we consider.

The fifth (and final) condition is that we have x t  = 1 n within the cn 3 considered steps. We assume that after the first four conditions are all satisfied there are at least three relevant mutations of single bits left. If this is not the case we can easily achieve this (with probability ) by increasing c by some constant factor. We consider the situation when we have with i ≥ n − 3. Clearly, at most three consecutive relevant single bit mutations suffice to reach the global optimum 1 n . The conditional probability for such a mutation (under the condition that we have a relevant single bit mutation) equals . Thus, the probability to reach the optimum in the next three relevant single bit mutations is bounded below by (1 ∕ 2)3 = 1 ∕ 8 and we have p 5 ≤ 7 ∕ 8.

We sum up the failure probabilities and have

for c and n sufficiently large. This concludes the proof of the upper bound.

For the lower bound we see that the first point on the plateau is with i ≤ n ∕ 2 with probability at least 1 ∕ 2 as it is for RLS. We consider steps with a sufficiently small constant. With probability there is no mutation with x t  ≠ x t + 1 that changes more than three bits simultaneously as discussed for the second condition in the proof of the upper bound. Now, the situation is analogous to a fair random walk on at least points, and we know that the expected number of steps to reach the end is . Since the probability for a relevant step is bounded above by , we have as an immediate consequence.

5.7 Delay Sequences

The method of delay sequences comes from the analysis of communication in networks. It is a method that allows for the proof of upper bounds. The main idea is that if something takes long there has to be a reason for this delay. Proving that such delays occur only with small probability can lead to the proof of an upper bound for the expected time. We consider this method here for three reasons. First, it will lead to an example where we are able to prove particularly tight upper bounds, much more precisely than we did before. Second, it is a nice example where consideration of simpler structured and thus easier to analyze random processes leads to proven upper and lower bounds for the actual evolutionary algorithm we consider. Finally, it is an example where crossover is involved.

We begin our example by defining another fitness function, called { H-IFF}. The name is an abbreviation for hierarchical if and only if. While its formal definition looks a bit complicated the main idea is rather simple.

Definition 5.8.

For let n = 2 k . For such n the fitness function is defined by

We consider a bit string x = x[1]x[2]⋯x[n], where n is a power of 2, i.e., n = 2 k for some . We look at x at k + 1 levels simultaneously. On the i{ th} level () we think of x as consisting of 2 k − i consecutive blocks of length 2 i . A block on the i{ th} level contributes 2 i to the function value if it is either an all ones block 1 i or an all zeros block 0 i . The function value { H-IFF}(x) is computed as sum of all contributions of all blocks on all levels. We have { H-IFF}(x) ≥ n since on level 0 each bit contributes 1. Moreover, { H-IFF}(x) = n holds exactly for x = 1010⋯10 = (10) n ∕ 2 and x = 0101⋯01 = (01) n ∕ 2. The only global optima of { H-IFF} are 0 n and 1 n with . Second best points are and with . We observe that holds for any x ∈ { 0, 1} n and its bitwise complement . We visualize this perspective on { H-IFF} in Fig. 5.8. On the i{ th} level we have 2 k − i nodes v i, 0, v i, 1, …, . On the 0{ th} level the node v 0, j is labeled with x[j]. On all other levels a node v i, j is labeled with b ∈ { 0, 1} if its two children v i − 1, 2j and are both labeled with b. Otherwise, v i, j is not labeled. If and only if node v i, j is labeled does it contribute 2 i to the total function value. The function value { H-IFF}(x) is obtained as sum of all contributions of all nodes.

Fig. 5.8

Graphical representation of { H-IFF} for n = 16, x = 0000100011111100. All nodes v 0, j contribute 1, all nodes v 1, j with j ≠ 2 contribute 2, nodes v 2, 0 and v 2, 2 contribute 4, all other nodes contribute 0 to the function value { H-IFF}(x) = 16 ⋅1 + 7 ⋅2 + 2 ⋅4 = 38

We do not perform a formal analysis of mutation-based algorithms for { H-IFF}. It is, however, rather obvious that their chances of finding one of the two global optima are rather slim. It is highly likely that consecutive blocks of both kinds, 0-blocks and 1-blocks, evolve. Since it is very difficult to change a 1-block to a 0-block and vice versa by means of mutation we should not expect to find a global optimum on average in polynomial time. An evolutionary algorithm with k-point crossover (k small) and a sufficiently diverse population can combine two different individuals, leading to a better search point with longer blocks. We make this idea concrete by introducing a very extreme crossover-based algorithm and performing a formal analysis of its expected optimization time on { H-IFF}. We denote the algorithm as (1+1) GA since it is almost identical to the (1+1) EA, only the mutation is replaced by crossover. Since crossover requires us to have a population of size at least 2 we decide to use the current search point x t and its bitwise complement  for crossover. Note that we do not consider this (1+1) GA to be a reasonable evolutionary algorithm. It is merely used as a vehicle for the analysis of { H-IFF}, as an extreme realization of mechanisms that aim at maintaining diversity in an evolutionary algorithm’s population.

Theorem 5.25.

Before we prove Theorem 5.25 we develop a better understanding of the random process that the (1+1) GA on { H-IFF} defines. In addition, we develop a bit of notation to deal with this process.

We consider a single 1-point crossover and use the notation from Fig. 5.8. The crossover point can be identified with exactly one inner node v i, j (i > 0). The offspring y consists of the concatenation of all bits left of v i, j in x t and all bits right of v i, j in . Consider the unique path from the root v k, 0 to this crossover node v i, j . We denote the set of all nodes on this path as V p . The set of all nodes left of this path is denoted as V l , and the set of the remaining nodes (all to the right of this path) are denoted as V r . We consider these sets in the trees for x t , , and the offspring y. All nodes in V l have the same labeling in x t and y. In the same way, all nodes in V r have the same labeling in and y. Remember that holds. We see that the fitness contribution of V l and V r are equal in x t and y. Differences can only be caused by the nodes in V p .

If v i, j is labeled in x t then we know that its children v i − 1, 2j and v i − 1, 2j + 1 have equal labels in x t . This implies that v i − 1, 2j and v i − 1, 2j + 1 have different labels in y and thus v i, j is no longer labeled in y. Clearly, all nodes in V p above v i, j are also not labeled. Thus, { H-IFF}(y) < { H-IFF}(x t ) holds, and the offspring y does not replace its parent x t .

If v i, j is not labeled in x t we distinguish two different cases. First, consider the case where at least one of the children v i − 1, 2j and v i − 1, 2j + 1 is also not labeled. In this case v i, j is still not labeled in y and { H-IFF}(y) = { H-IFF}(x) holds. Due to the usual selection mechanism we have x t + 1 = y ≠ x t in this case, but this is not important.

In the remaining case, in x t the crossover node v i, j is not labeled but both children v i − 1, 2j and v i − 1, 2j + 1 are labeled, and we know that the labels of v i − 1, 2j and v i − 1, 2j + 1 are complimentary. If they were equal v i, j would share their label. Clearly, in the offspring y the two children v i − 1, 2j and v i − 1, 2j + 1 are equally labeled and thus v i, j is labeled in y. We conclude that { H-IFF}(y) > { H-IFF}(x t ) holds. This is the important case. Note that in x t all nodes in V p are unlabeled, whereas in y it may be the case that in addition to v i, j other nodes in V p are now labeled, too.

Using the description as tree (Fig. 5.8) and the notion developed here we can formulate the (1+1) GA on { H-IFF} in a different but equivalent way.

Note that Algorithm 4 is identical to the (1+1) GA in the sense that at all points of time the probability distribution governing the labels of the leaves in the tree in Algorithm 4 and x t in the (1+1) GA are equal. Algorithm 4 stops when the (1+1) GA reaches a global optimum of { H-IFF}. Let T be the random variable describing the number of rounds when this happens. Clearly, T = T { (1+1) GA}, { H-IFF} holds.

For the proof of an upper bound on it is useful to consider an algorithm that is obtained by modifying Algorithm 4. This algorithm works on the same kind of tree but each node now either has some label or is unlabeled. We do not have different kinds of labels anymore.

Algorithm 5 is in some sense a pessimistic version of Algorithm 4. In Algorithm 4 the tree can be labeled bottom up since for the leaves v 0, j nothing needs to be done, and any other node v i, j (with i > 0) can be labeled by picking v i, j as crossover node and having labeled its children v i − 1, 2j and v i − 1, 2j + 1 before. In Algorithm 5 this still works, but it is the only way to label nodes. We do not have the ‘lucky cases’ where nodes are labeled due to ‘lucky initialization’ or ‘lucky equal labelings’ in different parts of the tree. Let T′ denote the number of rounds until Algorithm 5 stops. Clearly, T′ ≥ T holds and any upper bound on is also an upper bound on . Equipped with these insights we can now prove Theorem 5.25.

Proof (of Theorem 5.25).

For the proof of the upper bound we consider Algorithm 5 and make use of the method of delay sequences. Let δ > 1 be an appropriately chosen constant. We will chose an appropriate value for δ later. We consider a sequence of t = δ(n − 1)logn steps (where logn = log_2 n). For node v 1, j consider the unique path from the root v k, 0 to v 1, j . We call this path t-slow if after the t steps we consider the root v k, 0 is not labeled. Clearly, after these t steps the algorithm is still running if and only if there is some node v 1, j such that the corresponding path is t-slow. If there is a t-slow path we consider another sequence of t = δ(n − 1)logn steps. Clearly, the probability of having a t-slow path in this second phase of length t = δ(n − 1)logn is bounded above by the probability to have a t-slow path in the first phase. If we have a t-slow path in the second phase we continue with a third phase of equal length, and so on. For an upper bound on it suffices to derive an upper bound on the probability to have a t-slow path for each of the phases. Thus, it suffices to consider only the first phase, i.e., the first sequence of t = δ(n − 1)logn steps.

In each round of the algorithm and for each node v i, j with i > 0 we have that v i, j is selected as crossover node with probability 1 ∕ (n − 1). Only the crossover node can get a label in the current round. Moreover, on each path there is at most one node that can get labeled in the current round. Thus, the probability that a node on the path is labeled in one step is bounded above by 1 ∕ (n − 1). Let L be the random number of times a node on the path is labeled in the t = δ(n − 1)logn steps we consider. The expected number of times the path is labeled in these steps is bounded above by { E}L ≤ δlogn = δk. In the first phase all nodes are not labeled. Thus, the root is not labeled and the path therefore is t-slow if it is labeled less than logn times. Applying Chernoff bounds, we have that the probability not to label the path logn times is bounded above by

Since we have n ∕ 2 nodes v 1, j we get

as upper bound on the probability that there exists a t-slow path by application of the union bound. If we choose δ such that holds we have 1 − (δ − 1)2 ∕ (2δln2) > 0 and therefore as upper bound on the probability that the root is labeled (for a constant ). We define δ : = 3. 06. This yields that in each phase the probability not to label the root is bounded above by n  − 0. 0003. This yields

as claimed.

For the lower bound we start with a lower bound on . Note that such a lower bound l with does not imply a lower bound on since holds. The reason for starting with a lower bound on is that T′ is easier to analyze than T and that it will be relatively simple to obtain a lower bound on based on a lower bound on . We consider a run of Algorithm 5 and partition it into two disjoint phases. The first phase encompasses all steps until all nodes v 1, j are labeled and has length T′ 1. The remaining steps are the second phase with length T′ 2. Obviously, T′ = T′ 1 + T′ 2 holds.

In the first phase we have n ∕ 2 out of n − 1 crossover nodes that lead to a labeling of one of the v 1, j nodes. Thus, with probability (n ∕ 2) ∕ (n − 1) we have a step that changes something in a relevant way. For each node v 1, j the conditional probability that it is selected as crossover node given that the crossover node is some node v 1, j′ equals

If we only consider steps with crossover nodes v 1, j for some j we are exactly in the scenario of the coupon collector’s problem with n ∕ 2 cards. Taking into account the probability (n ∕ 2) ∕ (n − 1) for a relevant step, we have

as lower bound on .

For a lower bound on we consider the last node that is labeled in the first phase. While already during the first phase some nodes v i, j with i > 1 may have been labeled this cannot be the case for all nodes on the path from the root v k, 0 to . Clearly, these nodes can be labeled one by one bottom-up. For each of these k − 1 nodes the probability to be labeled in a specific step equals 1 ∕ (n − 1). Thus, the expected time to label these k − 1 nodes is bounded below by . This yields , and we have

as lower bound on In order to derive a lower bound on we partition the run of Algorithm 4 into two disjoint phases in the same way as we did for Algorithm 5. Let T 1 and T 2 denote the random lengths of these phases, respectively. We need to take into account the differences between Algorithms 4 and 5. In the first phase in Algorithm 4 we have nodes that are already labeled without being subject to crossover. This happens to a node v 1, j if both children v 0, 2j and v 0, 2j + 1 have equal labels and thus with probability 2 ⋅(1 ∕ 2)2 = 1 ∕ 2. The expected number of labeled nodes equals (1 ∕ 2) ⋅(n ∕ 2) = n ∕ 4, and using Chernoff bounds we have at most labeled nodes with probability . Thus, we are in the situation of the coupon collector’s problem with cards with this probability. Note, however, that if we have L labeled nodes in the beginning the probability to not hit one of those is also reduced to ((n ∕ 2) − L) ∕ (n − 1). This yields

so that we have no difference in the lower bounds for and .

In the second phase we again consider the path from the root v k, 0 to , where is the last node to be labeled in the first phase. On this path a node becomes labeled if the other child (the one not on the path) happens to have the same label as the one on the path. For each node this happens with probability at most 1 ∕ 2. This yields

and we obtain

as claimed.

5.8 Random Family Trees

All results that we discussed in this chapter so far are concerned with evolutionary algorithms that utilize a population of size 1. A larger population was only discussed in form of an offspring population of size λ > 1 (see the results for the (1 + λ) EA on LEADINGONES (Theorems 5.10, 5.17, and 5.21)). Increasing the offspring population size λ is in some sense very different from increasing the population size μ. Remember the description of evolutionary algorithms as Markov chains (see Sect. 3.1). Increasing the size of the offspring population λ changes the transition probabilities that describe the probabilities to move from one population to the next. The state space, the set of all populations, is not changed. This is different when the size of the population μ is increased. Since the size of the state space equals it grows quite dramatically with the population size μ. Since the size of the state space is one factor in the complexity of a Markov chain, we see that larger population sizes have at least the potential to complicate things much more severely than larger offspring population sizes. This difficulty is no reason to abandon evolutionary algorithms with population sizes μ > 1. In some sense, having a population of search points is one of the main ingredients of evolutionary algorithms that makes them different from other randomized search heuristics. It is therefore important to understand the role the population and its size play. We do this by analyzing a rather simple evolutionary algorithm with a population that we call (μ + 1) ∗  EA.

There is a slight difference between this (μ + 1) ∗  EA and the (μ + 1) EA as it can be obtained as a special case of the (μ + λ) EA (Algorithm 1). While the (μ + λ) EA favors offspring in selection in the case of equal fitness, the (μ + 1) ∗  EA does not do that. This difference becomes most obvious in the special case μ = 1. If we have P t  = { x} and an offspring y with f(x) = f(y) then in the (μ + λ) EA (that actually is the (1+1) EA since we have μ = λ = 1) the offspring y replaces its parent x. In the (μ + 1) ∗  EA, however, we have P t + 1 = { x, y} after the first line in the selection for replacement (line 4). In the subsequent step either x or y is removed, both with equal probability 1 ∕ 2.

This change in the way each offspring is handled in comparison to the (μ + 1) EA facilitates analysis. It creates a symmetry in the handling of the population and the offspring. It can be expected that, at least for not very small population sizes μ, the difference between the (μ + 1) EA and the (μ + 1) ∗  EA is hardly visible. We stick to the analysis of this variant for the sake of simplicity.

For many (but not all) fitness functions it is not difficult to see that if we have an upper bound , an upper bound of can be proved. This holds at least in all cases where the upper bound for the (1+1) EA is proved via fitness-based partitions. In addition to the probability s i to leave the current fitness level via mutation, we get the factor 1 ∕ μ for picking a current best individual in our population for reproduction. Having this additional factor for each fitness level yields an additional factor μ in the upper bound for the total expected optimization time. But it is not clear that such an upper bound is actually tight. While we expect to pay a price for a population, at least for easy to optimize fitness functions, we do not expect this price to be a factor that equals the size of the population in many cases. We will see in this section that actually the price that we pay for the unnecessary use of a population is smaller than this. In order to prove that, we need a method that enables us to deal with populations. This method is based on random family trees.

We consider one run of the (μ + 1) ∗  EA and define one random family tree T t (x) for each individual x in the initial population P 0. The index t denotes the current step. The set of nodes V t contains only non-negative integers, i.e., . Each node v ∈ V t corresponds to the generation when it was generated, thus the node was generated in the v{ th} generation. Considering the (μ + 1) ∗  EA implies that there is exactly one new node in each generation, and thus for each there is always exactly one random family tree that contains a node i (if the number of generations is at least i). We want to be able to recognize the nodes in a family tree. Thus, we define a labeling of nodes that maps a node i ∈ V t to the that the node i represents. The edges E t in a family tree represent offspring relationship. If in the j{ th} generation an offspring is generated as offspring by mutating some individual that is represented by a node i in a random family we have an edge (i, j) in this family tree. This definition of edges defines in which family tree a new node j is to be inserted. It becomes a node in that family tree where its parent is located. Only the family tree containing the parent that was selected for reproduction changes, and we have no change in any other family tree T t (x), so that V t + 1 = V t , , and hold for these other family trees.

Our definition of family trees implies that the nodes have strictly increasing values on any path from the root to a leaf. For each edge (i, j) ∈ E t we have i < j. Note that family trees represent the reproduction in the (μ + 1) ∗  EA. They do not represent the selection for replacement. Thus, in general random family trees contain many nodes that are not members of the current population. In particular, the function values on paths from the root to any leaf are not necessarily increasing. We can have for nodes with an edge (i, j) ∈ E t between them.

The intuition behind family trees that is the key to their use in the analysis is the following. When analyzing the (1+1) EA we have seen that for a fitness function f we need a not too short sequence of mutations to mutate a random individual into an optimal one with probability close to 1. For the (μ + 1) ∗  EA this implies that we need to have random family trees that have a sufficient depth. If the depth is as long as the length of the sequence of mutations we expect to need, we expect the (μ + 1) ∗  EA to have found an optimum. If the depth is smaller than that, we expect the (μ + 1) ∗  EA not to have found an optimum yet. We see that we need lower bounds on the depth of random family trees if we want to prove upper bounds on the expected optimization time of the (μ + 1) ∗  EA. Analogously, we can use upper bounds on the depth of random family trees for lower bounds on the expected optimization time.

Simple upper bounds for the (μ + 1) ∗  EA can often be proved without making use of random family trees. This can be achieved by adapting the method of fitness-based to the analysis of the (μ + 1) ∗  EA. We demonstrate this for ONEMAX and LEADINGONES.

Theorem 5.26.

Proof.

We adopt the method of fitness-based partitions and consider the function value of the current best, L t : = max{{ LEADINGONES}(x)∣x ∈ P t }. Consider the first generation when an optimum is reached, . Since the (μ + 1) ∗  EA makes one function evaluation in each generation and μ function evaluations for the initial population T (μ + 1)  ∗   { EA}, { LEADINGONES} = μ + T holds.

Due to the strict plus-selection L t can only increase. It increases by at least 1 and at most n times. Thus, we obtain if I is an upper bound on the expected time needed to increase L t in one generation. A sufficient condition to increase L t is to select a best member of the current population and increase its number of leading 1-bits by mutating exactly its leftmost 0-bit. Let i denote the number of best individuals in the current population. Then the probability to increase L t is bounded below by (i ∕ μ) ⋅(1 ∕ n)(1 − 1 ∕ n) n − 1 ≥ i ∕ (enμ). Depending on i this probability may be small. One way of increasing it is to increase the number of current best i. This can be done by selecting one current best and do not change at all in mutation. This has probability . In the worst case we have i = 1. This is not an unrealistic case. Each time L t is increased we have i = 1 in the subsequent generation. The expected time to increase this number of current best to some value j (or increase L t before that) is bounded above by

In order to obtain a good upper bound on T we look for a value of j such that the expected time needed to create j current best matches the expected time needed to improve the current best given j current best. We observe that

holds and follows as bound. Thus we want to have μlogj ≈ μn ∕ j and see that having j = n ∕ logn is a good choice. Now our reasoning is complete.

In order to increase L t we wait until the number of current best is increased to min{n ∕ logn, μ} and after that until one of the at least min{n ∕ logn, μ} current best is selected for reproduction and improved via mutation. This yields

For the expected optimization time this implies

as claimed.

We see that the (μ + 1) ∗  EA is asymptotically not slower than the (1+1) EA on LEADINGONES as long as its population size is bounded by . We can prove a similar bound for the (μ + 1) ∗  EA on ONEMAX in the same way. Things are a bit more complicated since the probability for a mutation that increases the function value of a current best now depends on the function value of the current best.

Theorem 5.27.

Proof.

We carry out this proof very similarly to the proof of Theorem 5.26. For ONEMAX we define and i(n − L t ) ∕ (enμ) as lower bound on the probability to increase L t if P t contains at least i current best. Here we want to increase the number of current best from 1 to min{μ, n ∕ (n − L t )} and have an expected waiting time of less than 6μln(n ∕ (n − L t )).

Different from the proof of Theorem 5.26, we sum up the expected times in two different sums. On one hand, we sum up all expected waiting times for increasing the number of current best and get

as upper bound. On the other hand, we sum up all expected waiting times for improving the current best given that there are at least min{μ, n ∕ (n − L t )} in P t . Here we have

as upper bound.

Together, we obtain

as claimed by adding up the two upper bounds and additionally taking into account the μ function evaluations for the initial population.

We see that the (μ + 1) ∗  EA is asymptotically not slower than the (1+1) EA on ONEMAX as long as its population size is bounded by .

For ONEMAX and LEADINGONES it suffices to concentrate on the current best member of the population. Given its function value we have sufficient information to compute a bound on the expected waiting time for an improvement in function value. We considered the number of copies made to have an improvement for this probability. For PLATEAU, this technique cannot work. The time until the first individual reaches the plateau is similar to ONEMAX. We expect to see the complete population on the plateau after another steps (and will prove this expectation to be correct later). At that point of time the situation is very much different from the situation for ONEMAX and LEADINGONES. The function value of the current best only tells us that it is on plateau, i.e., of the form 1 i 0 n − i with . We remember the analysis of RLS (Theorem 5.23) and of the (1+1) EA on PLATEAU (Theorem 5.24) and see that we need to consider the random walk on the plateau. For the (μ + 1) ∗  EA we need to make sure that the individuals have sufficient time to perform such a random walk. In order to do this we prove a lemma that connects the (μ + 1) ∗  EA with the (1+1) EA.

We consider the (μ + 1) ∗  EA on PLATEAU. Let

denote the first point of time when the complete population of the (μ + 1) ∗  EA is on the plateau. Consider some random family tree T t (x 0) with and some t ≥ T { PLATEAU}. Note that this T t (x 0) may be (and most probably is) a subtree of some T 0(x) for some x ∈ P 0. Now consider some y ∈ P t (x 0) and the path from x 0 to this y. If y is still member of the population, i.e., y ∈ P t , we say that the path is alive. Otherwise, if holds, we say that the path is dead. Clearly, we can always find a such that there is always a path in T t (x 0) that is still alive. For such a path we connect the (μ + 1) ∗  EA with the (1+1) EA in the following lemma.

Lemma 5.7.

Let . For some s with consider x 0 ∈ P s and some path with x l ≠1 n that is alive.

The search points are subject to the same random distribution as a sequence of l + 1 search points that is generated by the (1+1) EA on PLATEAU started in x 0 and generating only point on the plateau.

Proof.

The proof can be carried out by induction on the length of the sequence l. For l = 0 the claim holds since starting in x 0 is part of the assumption. In each step the (1+1) EA creates a new search point that by assumption is on the plateau and replaces the current bit string by it (since the two are of equal fitness). By induction the points generated by the (μ + 1) ∗  EA so far are all distributed in the same way as are the corresponding points generated by the (1+1) EA. In the next step, the (μ + 1) ∗  EA also generates a point on the plateau, otherwise the path would be dead since we consider a point of time after T { PLATEAU}. Due to the selection the offspring is included in the population. Note that the immediate preceding element cannot be removed since we assume the path to be alive. This does not change the probability distribution since the replacement is carried out uniformly at random in the population.

Using this link between the (μ + 1) ∗  EA and the (1+1) EA we can reuse parts of the proof of Theorem 5.24 to prove the following result.

Theorem 5.28.

Proof.

Until the plateau is reached the situation is equivalent to ONEMAX. Once the first member of the population is on the plateau, the expected time until the complete population is on the plateau is since the probability to increase the number of members of the population on the plateau from i to i + 1 is bounded below by and

holds. Together with Theorem 5.27 this yields . We know from Theorem 5.24 that holds. If T k denotes the random time until we have a random family tree of depth k (with sufficiently large), we have . Thus, we need an upper bound on .

We are interested in the expected time until there is a path in T t (x 0) of depth at least k, all paths in T t (x 0) are dead, or the global optimum has been found. Let

denote the set of individuals we consider. We are interested in the depth of these individuals and thus consider

their maximal depth L t′ . We see that L t′ is nondecreasing with increasing t′. We are interested in seeing L t′ increased. This happens if some individual with depth L t′ is selected (with probability at least 1 ∕ μ), its offspring is also on the plateau (with probability at least (1 − 1 ∕ n) n  ≥ 1 ∕ (2e)) and it is deleted prior to its offspring (with probability 1 ∕ 2 for symmetry reasons). Thus, the probability to increase L t′ is bounded below by 1 ∕ (4eμ). This implies and

follows.

When introducing family trees we argued that they can be used to prove upper and lower bounds. We make this concrete by proving a general lower bound for functions with a unique global optimum. Before we do so we introduce a technical lemma that bounds the probability for having random family trees with large depth. This is useful because for lower bounds on the expected optimization time we need upper bounds on the depth of the random family trees.

Lemma 5.8.

Consider the (μ + 1) ∗  EA on an arbitrary function  . For any and any

holds.

Proof.

Consider a specific path of depth l with that is labeled with search points . The probability of creating exactly this path is bounded above by

Considering a tree at time step t there are ways of choosing the t points of time and thus { t l} ways of choosing such a path. We conclude that the probability to have a path of depth l is bounded above

Setting t = lμ ∕ 3 we get as upper bound for the probability of having a path of depth l = 3t ∕ μ

as claimed.

Theorem 5.29.

Let be an arbitrary function with a unique global optimum, let

Proof.

We make a case distinction with respect to the population size μ and begin with the case of small population size. Let μ ≤ log(n) ∕ 2 hold. Now it suffices to prove . Consider one specific bit, say the i{ th} one. With probability 1 ∕ 2 a specific initial individual differs at this position from the unique global optimum. Thus, with probability all individuals in the initial population have this bit wrong. This holds for any bit position i. Thus, the expected number of bits that are not set correctly in any individual of the initial population is bounded below by . Application of Chernoff bounds yields that with probability we have at least such bits. Now consider the first (n − 1)ln(n) ∕ 2 generations. The probability that of these at least bits at least one is never flipped is bounded below by

This yields

in this case.

In the case of larger populations we have μ > log(n) ∕ 2 and it suffices to prove . We consider the first cμn generations, where c > 0 is some small constant. We know that the probability to have a family tree of depth at least 3cn is bounded above by (Lemma 5.8). This implies that the expected depth of any family tree is bounded above by 3cn (since ). Application of Chernoff bounds yields that the probability to have a family tree of depth at least 6cn is bounded above by . In the same way as in Theorem 5.11 we see that in only 6cn function evaluations the optimum is not found with probability close to 1. This implies .

The general lower bound in Theorem 5.29 is asymptotically tight. The function ONEMAX shows this since ONEMAX is a function with a unique global optimum and can be optimized in steps. We summarize these findings from Theorems 5.27 and 5.29 in the following corollary.

Corollary 5.3.

For LEADINGONES the general lower bound from Theorem 5.29 is not tight. We show that by proving a lower bound that asymptotically matches the upper bound from Theorem 5.26.

Theorem 5.30.

Proof.

The upper bound is contained in Theorem 5.26. For the proof of the lower bound we consider the two parts and separately. The lower bound can be proved in the same way as for the (1+1) EA (see proof of Theorem 5.16). We use

as drift function and see that expected decrease in a single generation is bounded above by . Since the expected initial distance is for any polynomially bounded population size μ the bound follows.

For the part of the lower bound we want to prove that with probability close to 1 the unique global optimum is not found within the first cμnlogn generations for some small constant c > 0. We have considered LEADINGONES before and know that for each member x of the initial population we have { LEADINGONES}(x) ≤ n ∕ 2 with probability . For such an individual we know from the analysis of the (1+1) EA (Theorem 5.16) that the probability to have it mutated into the unique global optimum within n ∕ 6 mutations that increased its function value is bounded above by . For the (μ + 1) ∗  EA we know from the proof of the lemma used in the proof of the general lower bound (Lemma 5.8) that the probability for having a specific path of depth l in the family tree is bounded above by

In the proof of the general lower bound we only used the trivial bound since a better bound needs some knowledge about the objective function. Here we consider LEADINGONES and thus know that holds on any path of such improvements. This yields

as upper bound for the probability of the existence of such a path. We observe that ((ecnlogn) ∕ l) l is maximal for l = cnlogn, so that we obtain

as upper bound for the same event given that c < 1 ∕ (6loge) holds. Since we are free to chose the constant c > 0 as small as we wish, this concludes the proof.

We now have for ONEMAX and for LEADINGONES. Remembering the corresponding bounds for the (1+1) EA (see Theorems 5.2 and 5.16), we see that we have bounds of the form with for both functions. Continuing this pattern for PLATEAU we would expect an upper bound with for the (μ + 1) ∗  EA on LEADINGONES. Theorem 5.28, however, only yields . We are neither able to prove a lower bound of that order nor to improve the upper bound. In the following we prove a lower bound of , leaving a gap between upper and lower bound of a factor .

Theorem 5.31.

Proof.

The main idea is similar to the one used in the proof of the lower bound on LEADINGONES (Theorem 5.30). We show that for a small constant c > 0 we need to have a family tree of depth at least 3cn 3 ∕ logμ with probability close to 1. We concentrate on paths that are alive and know that the probability distribution on such paths is equal to that of the (1+1) EA (Lemma 5.7). At any point of time we have at most μ paths that are alive. It may be less since a path may contain more than a single individual of the current population. Let, as before, denote the first point of time when the complete population is on the plateau. In addition, we consider the first point of time when a first individual enters the plateau and denote this point of time by T { PLATEAU}, 1. By definition T { PLATEAU}, 1 ≤ T { PLATEAU} holds.

In the initial population each individual contains at most (5 ∕ 9)n 1-bits with probability . Then the situation is essentially the same as on n − { ONEMAX}, and thus the number of 1-bits cannot increase until the first point enters the plateau. In the mutation that first enters the plateau with probability at most a small linear fraction of all bits are mutated. Thus, at T { PLATEAU}, 1 at most one member of the population contains at most (4 ∕ 7)n 1-bits, and all other members of the population contain at most (5 ∕ 9) 1-bits with probability . Since the situation before T { PLATEAU}, 1 is equivalent to ONEMAX, we have that holds.

Now we consider the next generations. We already know that holds. Using standard arguments of repeating phases we obtain that

holds. We also already know that within these generations with probability the depth of the family trees is bounded by . In effective mutations the Hamming distance to the unique global optimum is almost surely not decreased linearly. Thus, all members of the population have at most (3 ∕ 5) 1-bits almost surely after T { PLATEAU} generations.

Now we consider the subsequent generations. Consider some path in that is alive (assuming that T (μ + 1)  ∗  { EA}, { PLATEAU} > l holds). Let A denote the event that p contains a subpath  such that the following three conditions all hold. The subpath  only contains nodes after T { PLATEAU}. All nodes are labeled 1 i 0 n − i with n ∕ 4 ≤ i ≤ (3 ∕ 4)n. There are changes of labels on the subpath . We prove that holds.

Remember that we can assume that the first point on the plateau contains at most (3 ∕ 5)n 1-bits. All his successors are also on the plateau, otherwise the path would be dead. The interval of 1-bits we consider will with overwhelming probability not be jumped over in a single mutation since this would require a mutation of a linear number of bits. The only condition that is not obvious is the lower bound on the number of changes of label.

The interval on the plateau we consider has linear length . We know from the analysis of the RLS on PLATEAU (Theorem 5.23) that with probability  this distance is not crossed in steps. The (μ + 1) ∗  EA can make larger steps, of course. However, the situation is with respect to mutations the same as for the (1+1) EA (see Theorem 5.24). The expected number of mutating bits is and large mutations are exponentially unlikely (in the number of mutating bits). Thus we obtain the same result and follows.

Now we can additionally assume to have changes of labels on the subpath . With probability between two changes of labels we have generations. This holds independently for each change of label. Remember that the probability to make a copy of a parent equals . Thus, in these generations between two changes of label the path will grow in length by due to these copies with probability very close to 1 (by means of Chernoff bounds). Together we obtain that with overwhelming probability the subpath  has length . We already know that with probability close to 1 we need generations to have a path of length (Lemma 5.8). This implies .

5.9 Remarks

This chapter is the most important for this book as it lays out the methods needed for analyzing evolutionary algorithms. It is therefore not surprising that the remarks at the end of this chapter are by far the most extensive. We have neither presented the newest nor the strongest methods known today. Instead, we opted for methods that are particularly easy to understand and apply and that are applicable in many situations. We give pointers to stronger methods and more recent developments here.

All the results we have represented are given using Landau notation (see Appendix A.1). Even where multiplicative constants showed up in the proofs we have dropped them as soon as possible and have not paid attention to them or tried to optimize these. It is possible to derive much more precise results; the work by Böttcher et al. [12] is an excellent example. However, this line of research has the obvious disadvantage of being at odds with the spirit of evolutionary computation. Evolutionary algorithms are heuristics that we apply with the hope to obtain good solutions in reasonable time. We do not expect them to be optimal. We see and are willing to call the (1+1) EA efficient on ONEMAX in spite of the fact that a performance of is easy to achieve. Admitting that we consider a factor of to be unimportant, we should acknowledge that it does not make much sense to care about constant factors. Moreover, performing such a precise analysis as the one in [12] leads to additional problems that make the results questionable, as Jansen and Zarges [68] point out.

Fitness-based partitions are probably the oldest analytical method. Trivial f-based partitions were used by Mühlenbein in 1992 [90] to prove that holds. The first use of nontrivial f-based partitions can be traced back to 1996: Rudolph [111] used them to obtain upper bounds for . The functions were introduced by Horn et al. [51] for k = 3, later formalized and generalized by Rudolph [112]. The definition here is somewhat more elegant than Rudolph’s definition by using the empty path as base case and is due to Sudholt [118]. The introduction of long paths was motivated by the wish to demonstrate that unimodal problems can contain exponentially long paths to the optimum (which is true) and that crossover helps on that path. Based on experiments the authors claimed that the (1+1) EA would require exponentially long time on { LONGPATH} k , a claim that was shown to be false by Rudolph [111]. That exponentially long paths can actually imply exponential optimization time for the (1+1) EA was later proven by Droste et al. [31] for . The result was later generalized for { LONGPATH} k with arbitrary k by Garnier and Kallel [45].

Fitness-based partitions were spelled out as a method explicitly by Wegener [128]. He also presents a lower bound technique based on fitness layers, but a very weak one. Better lower bounds using fitness-based partitions can be obtained by taking into account the so-called viscosity that helps to estimate how many fitness layers are skipped on average. This is introduced by Sudholt [119, 120]. The approach can also be used to obtain better upper bounds [120]. It is worth noting that fitness-based partitions as a method have been extended to deal with non-elitist selections mechanisms by Lehre [78].

The proof for the general lower bound was contained in [30] (although only implicitly, and actually spelled out for linear functions). It is explicitly stated for all evolutionary algorithms that use standard bit mutations as the only variation operator in [70]. It can easily be extended to fitness functions with a larger number of global optima. The coupon collector scenario is a classic in probability. The result and its proof here are particularly close to the description in the textbook by Motwani and Raghavan [88].

The method of typical events is implicit in the proof for the general lower bound [30]. The typical event is the initialization with a search point that has linear Hamming distance to the unique global optimum. The technique is the basis of the proofs for lower bounds on , , both in [31].

The introduction of drift analysis to evolutionary computation is due to He and Yao [47]. They presented both the upper and the lower bound (Theorems 5.15 and 5.19), but the proofs have been criticized as being incomplete. A more complete proof is provided by Jägersküpper [54]. The main application in [47] is a simplified proof of an for any linear function . The proof, however, turned out to be incorrect and was ‘corrected’ by reducing the mutation probability to p m  = 1 ∕ (2n) (so that also fitness-based partitions could be used). Finally, a correct proof of the bound for the (1+1) EA on linear functions with the standard mutation probability p m  = 1 ∕ n by means of drift analysis was presented by He and Yao in [48]. As we know, the proof becomes much simpler when using multiplicative drift (Lemma 5.6) instead of additive drift (Lemma 5.19). The introduction of multiplicative drift is due to Doerr et al. [23], who shortly after slightly improved the upper bound obtained [24]. A further improvement was the introduction of bounds on the probability to deviate from the expected time due to Doerr and Goldberg [20]. The for the (1+1) EA on linear functions can be extended to mutation probabilities p m  = c ∕ n for arbitrary constants by means of drift analysis if the drift function depends on c and the actual linear function [19]. The final word in the case of the (1+1) EA on linear functions is due to Witt [135], who presents a very precise analysis.

Drift analysis has been extended to make use of more complicated drift functions. Apart from the example of adapting the drift function to the concrete problem and parametrization of the algorithm [19], the extension of the notion of drift by Lehre and Yao [78] is worth mentioning. Multiplicative drift can also be extended to prove lower bounds [135]. More importantly, drift analysis can yield exponential lower bounds [96].

Typical runs as a proof method were explicitly laid out for the first time by Wegener [128]. It has been used implicitly before, perhaps most pronouncedly in the proof of an upper bound for the expected optimization time of an evolutionary algorithm with uniform crossover on { JUMP} k [56, 60]. The upper bound for was proved considerably earlier [59] than the lower bound [14], even though essentially both rely on the method of typical runs.

The introduction of random family trees as a method for the analysis of evolutionary algorithms is due to Witt [134]. In his seminal paper on the (μ + 1) EA he used them to derive tight bounds, and he was the first to do so for a population-based evolutionary algorithm.

The introduction of delay sequences as a method for the analysis of evolutionary algorithms is due to Dietzfelbinger et al. [17]. They use them to derive much more precise bounds than was common at that time. The evolutionary algorithm they consider, the (1+1) GA (Algorithm 3), is very artificial, of course. It is worth noting that similar results can be obtained with more realistic evolutionary algorithms if appropriate measures for maintaining diversity in the population are employed [44]. The example function, { H-IFF}, was introduced by Watson et al. [126] in the context of the building blockhypothesis.

While almost all of the methods presented in this chapter and in the references mentioned here were developed for the analysis of evolutionary algorithms their applicability is by no means restricted to this. They can and have been successfully applied to the analysis of many other randomized search heuristics, including ant colony optimization (for example, [27, 93, 122]), artificial immune systems (for example, [69, 138]), particle swarm optimization (for example, [121]), and simulated annealing (for example, [63, 131]).

References

12.

S. Böttcher, B. Doerr, F. Neumann, Optimal fixed and adaptive mutation rates for the LeadingOnes problem, in Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN XI), ed. by R. Schaefer, C. Cotta, J. Kołodziej, G. Rudolph. Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, 2010), pp. 1–10

14.

D. Brockhoff, T. Friedrich, N. Hebbinghaus, C. Klein, F. Neumann, E. Zitzler, On the effects of adding objectives to plateau functions. IEEE Trans. Evol. Comput. 13(3), 591–603 (2009)CrossRef

17.

M. Dietzfelbinger, B. Naudts, C.V. Hoyweghen, I. Wegener, The analysis of a recombinative hill-climber on H-IFF. IEEE Trans. Evol. Comput. 7(5), 417–423 (2003)CrossRef

19.

B. Doerr, L.A. Goldberg, Adaptive drift analysis, in Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN XI), Kraków, ed. by R. Schaefer, C. Cotta, J. Kołodziej, G. Rudolph. Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, 2010), pp. 32–41

20.

B. Doerr, L.A. Goldberg, Drift analysis with tail bounds, in Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN XI), Kraków, ed. by R. Schaefer, C. Cotta, J. Kołodziej, G. Rudolph. Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, 2010), pp. 174–183

23.

B. Doerr, D. Johannsen, C. Winzen, Drift analysis and linear functions revisited, in IEEE Congress on Evolutionary Computation (CEC 2010), Barcelona (IEEE, Piscataway, 2010), pp. 1–8

24.

B. Doerr, D. Johannsen, C. Winzen, Multiplicative drift analysis, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2010), Portland (ACM, New York, 2010), pp. 1449–1456

27.

B. Doerr, F. Neumann, D. Sudholt, C. Witt, Runtime analysis of the 1-ANT ant colony optimizer. Theor. Comput. Sci. 412(17), 1629–1644 (2011)MathSciNetMATHCrossRef

30.

S. Droste, T. Jansen, I. Wegener, A rigorous complexity analysis of the (1 + 1) evolutionary algorithm for linear functions with Boolean inputs, in Proceedings of the IEEE International Conference on Evolutionary Computation (ICEC 1998), Anchorage, ed. by D. Fogel, H.-P. Schwefel, T. Bäck, X. Yao (IEEE, Piscataway, 1998), pp. 499–504

31.

S. Droste, T. Jansen, I. Wegener, On the optimization of unimodal functions with the (1 + 1) evolutionary algorithm, in Proceedings of the 5th International Conference on Parallel Problem Solving from Nature (PPSN 1998), Amsterdam, ed. by A. Eiben, T. Bäck, M. Schoenauer, H.-P. Schwefel. Lecture Notes in Computer Science, vol. 1498 (Springer, Berlin, 1998), pp. 13–22

44.

T. Friedrich, P.S. Oliveto, D. Sudholt, C. Witt, Analysis of diversity-preserving mechanisms for global exploration. Evol. Comput. 17(4), 455–476 (2009)CrossRef

45.

J. Garnier, L. Kallel, Statistical distribution of the convergence time of evolutionary algorithms for long-path problems. IEEE Trans. Evol. Comput. 4(1), 16–30 (2000)CrossRef

47.

J. He, X. Yao, Drift analysis and average time complexity of evolutionary algorithms. Artif. Intell. 127(1), 57–85 (2001)MathSciNetMATHCrossRef

48.

J. He, X. Yao, A study of drift analysis for estimating computation time of evolutionary algorithms. Nat. Comput. 3(1), 21–35 (2004)MathSciNetMATHCrossRef

51.

J. Horn, D. Goldberg, K. Deb, Long path problems, in Proceedings of the 4th International Conference on Parallel Problem Solving from Nature (PPSN IV), Jerusalem. Lecture Notes in Computer Science, vol. 866 (Springer, Berlin, 1994), pp. 149–158

54.

J. Jägersküpper, Algorithmic analysis of a basic evolutionary algorithm for continuous optimization. Theor. Comput. Sci. 379(3), 329–347 (2007)MATHCrossRef

56.

T. Jansen, I. Wegener, On the analysis of evolutionary algorithms – a proof that crossover really can help, in Proceedings of the 7th Annual European Symposium on Algorithms (ESA 1999), Prague, ed. by J. Nesetril. Lecture Notes in Computer Science, vol. 1643 (Springer, Berlin, 1999), pp. 184–193

59.

T. Jansen, I. Wegener, Evolutionary algorithms – how to cope with plateaus of constant fitness and when to reject strings of the same fitness. IEEE Trans. Evol. Comput. 5(6), 589–599 (2002)CrossRef

60.

T. Jansen, I. Wegener, On the analysis of evolutionary algorithms – a proof that crossover really can help. Algorithmica 34(1), 47–66 (2002)MathSciNetMATHCrossRef

63.

T. Jansen, I. Wegener, A comparison of simulated annealing with simple evolutionary algorithms on pseudo-Boolean functions of unitation. Theor. Comput. Sci. 386, 73–93 (2007)MathSciNetMATHCrossRef

68.

T. Jansen, C. Zarges, Analysis of evolutionary algorithms: from computational complexity analysis to algorithm engineering, in 11th ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA 2011), Schwarzenberg, ed. by H.-G. Beyer, W.B. Langdon (ACM, New York, 2011), pp. 1–14

69.

T. Jansen, C. Zarges, Analyzing different variants of immune inspired somatic contiguous hypermutations. Theor. Comput. Sci. 412(6), 517–533 (2011)MathSciNetMATHCrossRef

70.

T. Jansen, K.A. De Jong, I. Wegener, On the choice of the offspring population size in evolutionary algorithms. Evol. Comput. 13(4), 413–440 (2005)CrossRef

78.

P.K. Lehre, Fitness-levels for non-elitist populations, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2011), Dublin (ACM, New York, 2011), pp. 2075–2082

88.

R. Motwani, P. Raghavan, Randomized Algorithms (Cambridge University Press, Cambridge, 1995)MATH

90.

H. Mühlenbein, How genetic algorithms really work: mutation and hillclimbing, in Proceedings of the 2nd International Conference on Parallel Problem Solving from Nature (PPSN II), Brussels (Elsevier, Amsterdam, 1992), pp. 15–26

93.

F. Neumann, C. Witt, Runtime analysis of a simple ant colony optimization algorithm. Algorithmica 54(2), 243–255 (2009)MathSciNetMATHCrossRef

96.

P.S. Oliveto, C. Witt, Simplified drift analysis for proving lower bounds in evolutionary computation. Algorithmica 59(3), 369–386 (2011)MathSciNetMATHCrossRef

111.

G. Rudolph, How mutation and selection solve long path problems in polynomial expected time. Evol. Comput. 4(2), 195–205 (1996)CrossRef

112.

G. Rudolph, Convergence Properties of Evolutionary Algorithms (Kovac, Hamburg, 1997)

118.

D. Sudholt, Computational complexity of evolutionary algorithms, hybridizations, and swarm intelligence. Ph.D. thesis, Technische Universität Dortmund, 2008

119.

D. Sudholt, General lower bounds for the running time of evolutionary algorithms, in Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN XI), Kraków, ed. by R. Schaefer, C. Cotta, J. Kołodziej, G. Rudolph. Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, 2010), pp. 124–133

120.

D. Sudholt, A new method for lower bounds on the running time of evolutionary algorithms. Technical report abs/1109.1504v2, CoRR, 2011. http://arxiv.org/abs/1109.1504

121.

D. Sudholt, C. Thyssen, Running time analysis of ant colony optimization for shortest path problems. J. Discret. Algorithms 10, 165–180 (2012)MathSciNetMATHCrossRef

122.

D. Sudholt, C. Witt, Runtime analysis of a binary particle swarm optimizer. Theor. Comput. Sci. 411(21), 2084–2100 (2010)MathSciNetMATHCrossRef

126.

R.A. Watson, G. Hornby, J.B. Pollack, Modeling building-block interdependency, in Proceedings of the 5th International Conference on Parallel Problem Solving from Nature (PPSN 1998), Amsterdam, ed. by A. Eiben, T. Bäck, M. Schoenauer, H.-P. Schwefel. Lecture Notes in Computer Science, vol. 1498 (Springer, Berlin, 1998), pp. 97–108

128.

I. Wegener, Methods for the analysis of evolutionary algorithms on pseudo-Boolean functions, in Evolutionary Optimization, ed. by R. Sarker, X. Yao, M. Mohammadian (Kluwer Academic, New York, 2002), pp. 349–369

131.

I. Wegener, Simulated annealing beats Metropolis in combinatorial optimization, in Automata, Languages and Programming, 32nd International Colloquium (ICALP 2005), Lisbon, ed. by L. Caires, G. Italiano, L. Monteiro, C. Palamidessi, M. Yung. Lecture Notes in Computer Science, vol. 3580 (Springer, Berlin, 2005), pp. 589–601

134.

C. Witt, Runtime analysis of (μ + 1) EA on simple pseudo-Boolean functions. Evol. Comput. 14(1), 65–86 (2006)MathSciNet

135.

C. Witt, Optimizing linear functions with randomized search heuristics – the robustness of mutation, in 29th International Symposium on Theoretical Aspects of Computer Science (STACS 2012), Paris, ed. by C. Dürr, T. Wilke. Leibniz International Proceedings in Informatics, vol. 14 (Dagstuhl Publishing, Saarbrücken, 2012), pp. 420–431

138.

C. Zarges, On the utility of the population size for inversely fitness proportional mutation rates, in 10th ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA 2009), Orlando, ed. by I. Garibay, T. Jansen, R.P. Wiegand, A.S. Wu (ACM, New York, 2009), pp. 39–46

Thomas JansenNatural Computing SeriesAnalyzing Evolutionary Algorithms2013The Computer Science Perspective10.1007/978-3-642-17339-4_6© Springer-Verlag Berlin Heidelberg 2013

6. Select Topics in the Analysis of Evolutionary Algorithms

Thomas Jansen^1

(1)

Department of Computer Science, University College Cork, Cork, Ireland

Abstract

Now that we have a collection of analytical tools tailored for the analysis of evolutionary algorithms, we are ready to consider concrete evolutionary algorithms and derive results, gain insights, prove facts about their behavior and performance. Of course, what one finds interesting and worth investigating in evolutionary computation is a personal question that is a matter of taste and other circumstances. It is almost inevitable that some questions that a reader finds most interesting will not be considered here. Therefore, the most important purpose this chapter serves is to be an example of how evolutionary algorithms can be analyzed using the methods described in the previous chapter. We do this considering four different topics that cover four different aspects of evolutionary computation. This includes considering effects of specific variation crossovers, dipping into the topic of design of evolutionary algorithms, considering a specific variant of evolutionary computation that we have not covered before, and, finally, considering an example for an application of evolutionary algorithms in combinatorial optimization.

Now that we have a collection of analytical tools tailored for the analysis of evolutionary algorithms, we are ready to consider concrete evolutionary algorithms and derive results, gain insights, prove facts about their behavior and performance. Of course, what one finds interesting and worth investigating in evolutionary computation is a personal question that is a matter of taste and other circumstances. It is almost inevitable that some questions that a reader finds most interesting will not be considered here. Therefore, the most important purpose this chapter serves is to be an example of how evolutionary algorithms can be analyzed using the methods described in the previous chapter. We do this considering four different topics that cover four different aspects of evolutionary computation. This includes considering effects of specific variation crossovers, dipping into the topic of design of evolutionary algorithms, considering a specific variant of evolutionary computation that we have not covered before, and, finally, considering an example for an application of evolutionary algorithms in combinatorial optimization.

6.1 Crossover

So far, we have not analyzed a single evolutionary algorithm that makes proper use of crossover. Almost every single one of our analyses was restricted to evolutionary algorithms using mutation as the only variation operator. The only exception is the (1 + 1) GA (Algorithm 3) that uses 1-point crossover. It is not a realistic evolutionary algorithm and it sidesteps the analytical difficulties introduced by crossover. We have already discussed (see Sect. 5.8) that increasing the population size μ complicates the analysis more than increasing the offspring population size λ. The reason is that increasing the population size increases the number of different populations, the natural state space of a Markov chain that is associated with an evolutionary algorithm. Increasing the offspring population size only changes the transition probabilities of this Markov chain, not its state space. Introducing crossover introduces another fundamental change. When using mutation it is completely sufficient to know about the distribution of single individuals in the population. When using crossover, however, at least two members of the current population are involved and the process becomes dependent on correlations between members of the population. One can argue that an evolutionary algorithm using a crossover operator that creates offspring based on two parents has a quadratical dynamical system as its most natural formal description (as opposed to a linear system like a Markov chain). It is well known that the analysis of quadratical dynamical systems is much harder. This exposes how the (1 + 1) GA sidesteps the difficulty usually introduced by crossover. It uses a population of size 1 so that no correlations have to be taken into account. The second parent used in crossover is completely determined by the first one (since it is its bitwise complement). This turns crossover essentially into another mutation operator. In this sense it is for the first time that we consider the analysis of an evolutionary algorithm with crossover in this section.

Using crossover as the only search operator is not a good idea. All crossover operators we consider have the property that { crossover}(x, x) = x holds. Moreover, when considering the search space {0, 1} n this holds for each individual bit: if the two parents coincide at position i than the offspring is guaranteed to have the same value at this position. This implies that evolutionary algorithms with crossover as only variation operator may get stuck and may never find a global optimum. Since they will get stuck with positive probability the expected optimization time is infinite. Thus, it makes more sense to introduce crossover as additional variation operator instead of replacing mutation by crossover. Note that for the (1 + 1) GA all this is not the case, another hint that for this algorithm crossover is actually more like a mutation operator.

There are different reasons why one may want to introduce crossover as additional variation operator. If one is interested in evolutionary algorithms from a biological perspective, it may make sense to have crossover since it can be observed in nature. From our perspective of computer science, however, this is not a good reason. What we are interested in is understanding how crossover can help optimize fitness functions more efficiently. Since crossover is computationally more expensive in comparison to common mutation operators we would hope for a fairly large speedup due to crossover.

Let us a start with considering { JUMP} k (Definition 5.5) and speculating what crossover may be able to do for us there. We restrict our interest to the case of rather small k and assume k ≪ n in the following. The function { JUMP} k (x) is very similar to { ONEMAX} if the number of 1-bits in x is bounded above by n − k. In order to increase the function value beyond that of a bit string with n − k 1-bits one needs a direct jump to the unique global optimum 1 n . The waiting time for such a jump dominates the expected optimization time of the (1 + 1) EA, it is (see Theorem 5.13).

Consider 2 bit strings with exactly n − k 1-bits and uniform crossover. We know that it matters what 2 bit strings of this kind we consider. For the sake of simplicity let us consider 2 bit strings that are selected uniformly at random among all bit strings with exactly n − k 1-bits. There are two possible cases. If there is a single position where the 2 bit strings both have a 0-bit we know that crossover cannot yield the global optimum 1 n . If there is no such position, we know that there are exactly 2k positions where exactly one of the 2 bit strings has a 0-bit. At all other n − 2k positions both bit strings have a 1-bit so that the offspring is guaranteed to have a 1-bit there, too. The probability to produce the optimum equals the probability to select the 1-bit at all these 2k positions, thus it equals 2 − 2k . We observe that this is much larger than n  − k , the probability to produce the optimum by means of standard bit mutation. In particular, the expected waiting time for such a crossover operation 22k is polynomially bounded for , where the waiting time n k for such a mutation is superpolynomial for .

We still need to determine the probability for being in one of the two cases. Let us consider the case where there is no position where both bit strings have a 0-bit. If we consider the first bit string to be fixed, then the second bit string can have its k 0-bits at any of the n − k positions where the first bit string has a 1-bit. Hence, there are such bit strings so that the probability for this case equals

Thus, we have a lower bound of to produce 1 n by means of uniform crossover of two such bit strings so that the expected waiting time is polynomially bounded for . We see that we can expect a substantial speedup in comparison to the (1 + 1) EA when using an evolutionary algorithm with uniform crossover. This speedup, however, depends on a not too small probability to have two parents that do not share some position where both bit strings have a 0-bit. There is no particular reason to believe that once an evolutionary algorithms arrives at a population where all bit strings have n − k 1-bits the members of this population are distributed uniformly among all such bit strings. In fact, usually they are not. We will investigate this in more detail.

Clearly, taking specific measures to guarantee diversity within the population would help to have a population that is closer to a uniform distribution. However, this means tailoring the evolutionary algorithm toward our specific fitness function. This is not what we have in mind. We want to investigate the potential of evolutionary algorithms as general-purpose problem solvers. Hence, we consider evolutionary algorithms without any measure to increase the diversity within the population. In order to get a clearer picture we do not fix a specific evolutionary algorithm but consider a class of evolutionary algorithms where not all details are fixed. We stick to the common notion in the field and call these algorithms steady-state genetic algorithms (GAs).

The steady-state GA has three parameters, the population size μ, the crossover probability p c , and a flag a ∈ {0, 1} that we discuss later. Moreover, the selection for reproduction is not fixed. We allow for all kinds of selection that depend only on fitness (and nothing else like the particular bit strings) and that do not select against the aim for fitness maximization. The second requirement implies in formal terms that for with we always have . The weakest form of selection that guarantees this is uniform selection. Strong selection mechanisms like tournament selection, fitness-proportional selection, and Boltzmann selection all work, too. Even truncation selection (where in this case two individuals with maximal function value would be selected while breaking ties randomly) is allowed. We observe that one consequence of our requirements is that individuals with equal function value are selected with equal probability. This is a crucial property that we will exploit when we prove good performance of these evolutionary algorithms on { JUMP} k .

The third parameter, a ∈ { 0, 1}, allows for a very mild measure to increase the diversity of the population. We discussed that such a measure would help uniform crossover on { JUMP} k . With a = 0 no such measure is taken. With a = 1 we avoid duplicates: In the case that no crossover is performed and mutation does not flip a single bit, i.e., the offspring is a duplicate of its parent, the offspring is discarded. We are actually able to prove good performance even without any measure to ensure diversity, i.e., for a = 0. However, with avoidance of duplicates (a = 1) we can prove a better upper bound on the expected optimization. We denote the optimization time of the steady-state GA with parameters , p c  ∈ (0, 1), and a ∈ { 0, 1} as .

Theorem 6.1.

For with , μ ≥ klog^2 n, p c ≤ 1∕(9kn), the following holds.

Proof.

We make use of the method based on typical runs. We divide a run of the GA into three disjoint phases and denote their respective lengths by T 1, T 2, and T 3. The first phase starts with an arbitrary population and ends when a population is reached where every member of the population contains exactly k 0-bits. Note that the number of 0-bits in the members of the population can only change when the global optimum is found. For the definition of the second phase we consider the number of 0-bits at position i in the current population, i.e., . The second phase starts after the first phase ends, and it ends when for all we have Z t [i] ≤ μ ∕ (4k) for the first time. Note that Z t [i] are random variables and may grow beyond μ ∕ (4k) for later populations, again. The third and final phase starts after the second phase ends, and ends when the global optimum 1 n is found. By definition holds.

For the first two phases we will rely on mutation only. For the first phase it is easy to see that crossover cannot hurt. With probability we do not use crossover to create an offspring. In the first phase { JUMP} k behaves like{ ONEMAX}. With a population of size μ we obtain as upper bound in the case a = 0. We observe that this (rather weak) upper bound does not rely on making duplicates. Duplicates only become important for better upper bounds, like the upper bound for the (μ + 1) EA on { ONEMAX} (Theorem 5.27). Thus, the same bound holds for a = 1.

The proof of an upper bound for the second phase is by far the most complicated part of the proof. We start with the case a = 1, i.e., the variant of the steady-state GA that does not allow to produce duplicates. We discuss changes for a = 0 afterwards.

We know that each member of the current population contains exactly k 0-bits. Thus, in total there are μk 0-bits in the population. If these were evenly distributed in the population we would have μk ∕ n 0-bits at each position. Unfortunately, there is no reason to assume that the 0-bits will be evenly distributed. In fact, they tend to be rather concentrated at a few position at the beginning of the second phase. This is due to an effect that is called hitchhiking. Some members of the population have better fitness than others. Depending on the selection employed these tend to have more offspring. These offspring tend to have the relatively few 0-bits at the same positions as their parents. Hence we can expect to have a concentration of 0-bits at the beginning of the second phase. We will prove that, for any initial distribution of 0-bits, the 0-bits become more evenly distributed over time. We prove that mutation is sufficient to do this. We make the pessimistic assumption that crossover tends to increase the concentration of 0-bits. It is this pessimistic assumption that requires us to restrict the crossover probability to small values p c  ≤ 1 ∕ (9kn). This limits the potential negative influence crossover may have.

We consider one generation and one of the n positions, say the first. Let z denote the number of 0-bits at the beginning, i.e., z = Z t [1]. We assume z ≥ μ ∕ (8k) in the following. We have since at most one member of the population is replaced by the offspring that is created in this generation. Let A z  +  denote the event that the number of 0-bits at this position increases. Let A z  −  denote the event that the number of 0-bits at this position decreases. We would like to know and . Without more information about the current population, determining these probabilities cannot be done. However, we are able to prove upper and lower bounds that will turn out to be sufficient. In order to do this we introduce some more random events. These random events can be used to characterize necessary conditions for A z  +  and sufficient conditions for A z  − . Moreover, their probabilities are easier to determine.

Let B z denote the event that crossover is used to create the offspring. We have .

Let C z denote the event that in the selection for replacement an individual with a 1-bit at position 1 is selected. It is important to note that in the second phase all members of the population have equal function value. Thus, selection for replacement (as well as selection for reproduction) is uniform. This implies .

Let D z denote the event that y 1[1] = 0. We have .

The steady-state GA (Algorithm 7) always applies mutation to generate an offspring, either after crossover or as the only variation operator. Let E z denote the event that in this mutation the first bit does not flip. We see that holds.

We define two more families of events. Their definition is more abstract. We will see later how they are related to the algorithm. For the moment it suffices to note that their definition is precise and we can easily assign probabilities.

Let (for ) denote the event that in a mutation i 0-bits out of k − 1 0-bits and i 1-bits out of n − k 1-bits mutate and n − 2i − 1 bits do not mutate. By definition holds.

Let (for ) denote the event that in a mutation i 0-bits out of k 0-bits and i − 1 1-bits out of n − k − 1 1-bits mutate and n − 2i bits do not mutate. By definition holds.

To prove an upper bound on we observe that

holds. Note that B z covers the case of crossover. If no crossover takes place, the number of 0-bits can only increase if one individual with a 1-bit is replaced (event C z ) and either an individual with a 0-bit was selected and turned into another individual with a 0-bit at this position (event D z for selecting such an individual, event E z for not mutating this 0-bit and the events to create an individual with k 0-bits in total), or an individual with a 1-bit was selected (event ), the bit was mutated (event ) and the remaining bits are mutated in a way that yield an offspring with k 0-bits (taken care of by the events ). This observation yields the upper bound

for the probability to further increase the number of 0-bits. Using the probabilities for the events we have

and obtain

as upper bound after some calculations.

We want to prove that the number of 0-bits gets smaller if it is large in the beginning. Thus, we need a lower bound on . To this end we use the same events as we did for A z  +  and define some more.

Let (for ) denote the event that in a mutation i − 1 0-bits out of k − 1 0-bits and i 1-bits out of n − k 1-bits mutate and n − 2i bits do not mutate. By definition holds.

Let (for ) denote the event that in a mutation i 0-bits out of k 0-bits and i 1-bits out of n − k − 1 1-bits mutate and n − 2i − 1 bits do not mutate. By definition holds.

For decreasing the number of 0-bits we rely on a generation without crossover (event ). In order to decrease the number of 0-bits it is necessary that the individual that is replaced in the final selection step has a 0-bit (event ). Now we can distinguish two disjoint cases. Either we select a parent that already has a 0-bit (event D z ), mutate this bit (event ), and end up with an individual that has exactly k 0-bits (which is described by for ). Alternatively, we select a parent that has 1-bit (event ), do not change this bit (event E z ), and again end up with an individual that has exactly k 0-bits (which is now described by for ). Together, this yields

and we obtain

as an immediate consequence. Remember that for all the events involved we know the exact probabilities (except for A z  − , of course). Plugging these in yields

and we obtain

after some calculations.

Let p z  −  denote this lower bound on and denote our upper bound on . We want to prove that there is a noticeable tendency to decrease the number of 0-bits. Thus, we would like to have quite a bit larger than 0. We have

and observe that this is not always the case. Depending on z and μ, the decisive term zn − μk could even be negative. But remember that we wanted to prove this tendency to decrease the number of 0-bits to be present only if the current number of 0-bits is fairly large. Actually, we decided to work under the assumption that z ≥ μ ∕ (8k) holds in the beginning. Using this assumption we obtain

which is a considerable tendency.

In order to get a grip on the change in the number of 0-bits it would be useful to consider only relevant steps of the algorithm, i.e., generations where z actually changes. The event that z changes equals . In order to obtain an upper bound on we first apply the union bound and obtain . We already have an upper bound on . For an upper bound on we remember that we have

and get

by again assuming the worst with respect to crossover (and being a bit more generous with respect to the mutation case). Plugging in the exact probabilities, using z ≥ μ ∕ (8k), and calculating as above we obtain

as upper bound. For A z  +  we already have

and using z ≥ μ ∕ (8k) as well as p c  ≤ 1 ∕ (9kn) we get

so that follows.

On the other hand, we need a lower bound for the probability to have a relevant step, i.e., a lower bound on . Clearly, holds and we already have

Remember that we still work under the assumption that z ≥ μ ∕ (8k) holds. Under this assumption we know that we have with 1 ∕ (8k) ≤ α ≤ 1, i.e., . Using this we have

as lower bound. Taking into account that holds we obtain

as lower bound on the probability of a step being relevant.

We consider cμnk 2 steps in the second phase (c > 0 a constant) and prove an upper bound on the probability not to finish the second phase in this number of steps. Since the probability of having a relevant step is we know that we have at least c′μk 2 relevant steps (c′ > 0 a constant with c′ < c) with probability by application of Chernoff bounds. Since we have μ ≥ klog^2 n this probability is very close to 1.

Consider one of these relevant steps. We denote the conditional probability to increase the number of 0-bits in such a step (conditioned on having an essential step) by q z  + , the conditional probability to decrease it is denoted by q z  − . We have

and

and by our results above

since and hold. We observe that yields that the number of 0-bits will be decreased by at least ( a constant with ) with probability , again by application of Chernoff bounds. Remember that we have made no assumption about the constant c in the number of steps cμnk 2. Clearly, for larger values of c we obtain larger constants c′ and c ′′ . Thus, we can make c so large that c ′′  ≥ 1 holds.

Reducing the number of 0-bits by at least c ′′ μ ≥ μ with probability appears to be impossible. Remember that all our calculations are valid only as long as z ≥ μ ∕ (8k) holds. Clearly, when the number of 0-bits z is reduced it will at some point of time fall below μ ∕ (8k). After that it may increase again. We consider the last point of time within these cμnk 2 steps where z falls below μ ∕ (8k). At this point of time some number of relevant steps are yet to be taken. We make a case distinction with respect to this number of steps. First, let us assume that this number is less than μ ∕ (8k). In that case, at the end the number of 0-bits is less than μ ∕ (8k) + μ ∕ (8k) = μ ∕ (4k) since the number of 0-bits can increase by at most 1 in each relevant step. Since μ ∕ (4k) is all that we need at the end of the second phase, for this case there is nothing to do. Therefore, let us now consider the other case where at least μ ∕ (8k) relevant steps are still to be taken. Since the number of essential steps is we can still apply Chernoff bounds as above. Thus, at the end of this phase we have that the number of 0-bits at the position we considered is at most μ ∕ (4k) with probability . Application of the union bound yields that we have this at all n positions with probability , and this concludes our consideration of the second phase for a = 1.

If the algorithm does not avoid duplicates, i.e., in the case a = 0, things change slightly. We need to take into account the events and since viable offspring can be constructed without mutating any bit. We have

and can easily verify that still holds. However, the probability for a step to be relevant increases considerably. Before we had ; this is now no longer true. We use the trivial upper bound instead. Due to this change we obtain the weaker bound of (instead of ). We compensate for this by increasing the number of steps we consider to (from cμnk 2), again c > 0 a sufficiently large constant. The rest of the proof remains unchanged.

Finally, we come to consider the third and final phase. We know that we start with at most μ ∕ (4k) 0-bits at each position. In the same way as we did for the second phase we show that with probability the number of 0-bits does not increase above μ ∕ (2k) at any of the n positions. In the third phase we create an optimal search point if we perform crossover (with probability p c ), select 2 bit strings that do not share a 0-bit at any position (with some probability ρ), select at all 2k positions where on of the parents has a 0-bit the 1-bit from the other parent (with probability (1 ∕ 2)2k ) and the following mutation does not flip any bit (with probability ). Thus, the total probability is and we need to bound ρ from below. Consider the situation after the first parent has been selected. It has k positions where the second parent should not have a 0-bit. At each position there are at most μ ∕ (2k) individuals in the populations that have a 0-bit at this position. Hence, the number of individuals that collide at least one of the k positions is bounded above by . Since selection is uniform (since all individuals have equal fitness) we have ρ ≥ 1 ∕ 2. Thus, we have probability to find the optimum for each generation in the third phase. This implies as upper bound on its expected length. Adding up the (expected) lengths of the three phases, , concludes the proof. □

Obviously, the proof of Theorem 6.1 is rather long (it started on page 8) and may appear complicated. However, it demonstrates a huge benefit due to uniform crossover for a class of evolutionary algorithms (remember that there is freedom with respect to selection for reproduction) on an example function that was not designed for this purpose. When we design purpose-built example functions we can prove even stronger results with less effort. This is the route that we take next, starting with a result for k-point crossover.

While it is nice to see that crossover can indeed be proven to be a crucial operator for a ‘realistic’ evolutionary algorithm (one that is pretty much standard and that does not have anything special built in to make the proof work) we are not completely happy with Theorem 6.1 for at least two reasons. One is that the gap between the performance of the EA with crossover and one without, say the (1 + 1) EA, is not exponential. If we set , i.e., as large as we can, we have a polynomial expected optimization time for the steady-state GA and . It would be nicer to have a gap that is actually exponential. The second reason is that the crossover probability we have in the proof is exceptionally low, p c  ≤ 1 ∕ (9kn). Usually, the crossover probability is set to some positive constant (but not 1). We have seen that we need this unusually small crossover probability for the proof. In the second phase we rely on mutation to distribute the few 0-bits more evenly in the population and simplify the analysis by assuming that crossover is always harmful. This simplistic approach requires us to have a crossover probability that is so small that its effects are outweighed by mutation. It is, however, not clear if such a small crossover probability is actually required for the algorithm to be efficient on { JUMP} k . Actually, experiments seem to indicate that this is not case. However, proving a result similar to Theorem 6.1 for a larger crossover probability, i.e., a result for the same algorithm on the same function but with is an open problem.

Therefore, we take another route in the following. We design an example function specifically for this purpose. We want to be able to prove an exponential gap between the optimization time of an EA with crossover and one without where the crossover probability is some constant. We will do this twice, once for uniform crossover and once for k-point crossover. We start with k-point crossover as it will turn out that it is easier to do for this crossover operator. The reason is not difficult to see. Since one k-point crossover operation can only create up to n different offspring, each of these has a pretty good chance of actually being created. Uniform crossover, on the other hand, creates any of 2 h offspring with equal probability (where h is the Hamming distance of the parents). If we want to have an exponentially long waiting time for an equivalent mutation we need to have . But for such large Hamming distances each potential offspring only has probability of being the actual offspring. Thus, we cannot expect to find a unique global optimum efficiently. We will discuss later how we can overcome this difficulty.

Again we make use of a steady-state GA, but a slightly different one from Algorithm 7. The algorithm differs in two ways. First of all, we do not make use of the mechanism of avoidance of duplicates (that is optional in Algorithm 7 and active for a = 1). The more important difference is the modified selection for replacement. The steady-state GA (Algorithm 7) selects uniformly at random one of the worst individuals for replacement if the new offspring is not worse. We modify this slightly by being a bit more selective. If the new offspring is not worse we also consider the worst individuals. However, among those we do not simply select one uniformly at random but restrict our choice to those individuals that have a maximal number of identical copies in the current population. It is easy to see that one does not want to have many copies of a worst individual in the population. In this sense this very slightly stronger selection mechanism is a very natural one. It is rather obvious a weak mechanism that aims at having some degree of diversity in the population. However, it will be crucial for our proofs and cannot simply be omitted. We do not specify the crossover operator that is used and will first consider the algorithm using 1-point crossover and afterwards with uniform crossover. As we did for Algorithm 7, we also do not specify the concrete selection for reproduction. We make the same assumptions as we did there: Selection is based only on fitness, and for x, y ∈ P t we have that f(x) ≥ f(y) implies that holds. Remember that this implies that individuals with equal fitness are selected with equal probability.

We define an example function that we call R 1, k and that can be explained as a modification of { JUMP} k . It shares its major structure with jump: The fitness values increase with the number of 1-bits as long as this number is at most n − k. The unique global optimum is the all ones bit string 1 n . For the remaining bit strings the fitness increases with an increasing number of 0-bits. However, we add a differentiation for bit strings with equal number of 1-bits. For these bit strings we consider the length of the longest block of consecutive 1-bits, . This is added to the ‘normal’ function value and therefore encourages to collect all 1-bits in a single block.

Definition 6.1.

Let and be given. Let denote the length of the longest block of consecutive 1-bits in x. The fitness function is defined by

The idea of R 1, k is pretty simple. Since the number of 1-bits is the most decisive factor we expect an EA to behave on R 1, k in the same way as it would on { JUMP} k . Thus, for a steady-state GA we expect the population to only consist of bit strings with exactly n − k 1-bits fairly quickly. After that the length of the longest block of consecutive 1-bits matters. Due to this the n − k 1-bits will be gathered in a single block. Note that there are k + 1 different positions where this block could start. If the population size is at least that large the selection mechanism of our steady-state GA (Algorithm 8) will make sure that for each possible starting position there is at least one member of the population where a block starts at that position. In this situation it is easy for 1-point crossover to create the unique global maximum. We show that these ideas are correct in the proof of the following theorem.

Theorem 6.2.

Let δ and be constants with . For with , μ > k, , the following holds for the steady-state GA Algorithm 8 with 1-point crossover.

Proof.

We consider four phases of the algorithm with lengths such that . We prove an upper bound on the expected length for each of the four phases.

The first phase starts with an arbitrary population at initialization and ends when all members of the population contain n − k 1-bits for the first time (or the optimum is found). In this phase it is important for us to have a probability of at least to have a step without crossover. Consequently, we see that just like for { JUMP} k we have ET 1 = Oμnlogn.

The second phase starts after the end of the first phase and ends when all members of the population have the n − k 1-bits collected in a single block of 1-bits (or the optimum is found). Note that in this phase all members of the population always contain exactly n − k 1-bits. Moreover, is monotone increasing with t, again due to selection. Moreover, when S(P t ) = μ(n − k) holds the second phase ends. As long as this is not the case we distinguish two cases. First, let us consider the case where b(x) = j holds for some value of j and all x ∈ P t . In this case we can increase S(P t + 1) in comparison to S(P t ) by having a generation without crossover, mutate one of the 1-bits that is not yet part of the longest blocks of 1-bits, mutate one of the 0-bits that is adjacent to this block, and do not mutate any other bit. This has probability at least

We observe that this case can occur at most once for each value of . Thus, the total contribution of this case toward is bounded above by

The other (and much more common) case is that not all members of the population have a longest block of 1-bits of equal length. In this case we concentrate on the number j of members of the population that have a longest block of 1-bits, i.e., . Note that j < μ since we treated the case j = μ in the first case. We can increase j by having a generation without crossover, select one of these j individuals for reproduction, and make a copy of it. This has probability

For each length of a longest block this can happen for all values . Thus, the total contribution of this case toward is bounded above by

so that

holds.

The third phase starts after the end of the second phase and ends when all possible bit strings that have exactly n − k 1-bits in a single block are present in the population (or the optimum is found). There are k + 1 different bit strings that we would like to see in the population. Here it is important to have μ > k, otherwise this phase could never end. In the beginning of the third phase we have at least one such individual (in this case all members of the population would be identical). As long as at least one is missing we can create a new one by means of a mutation that flips exactly 2 bits, one 0-bit adjacent to the block of 1-bits and a 1-bit at the other end of the block. Clearly, this creates another bit string with n − k 1-bits, all in one block, and a position of the block that is shifted by 1. We observe that it cannot be the case that this bit string is already member of the population for all possible parents. Hence, there is always at least one member of the population we can select as a parent for this operation. Therefore, it has probability at least

Here it is important that all members of the population are selected with equal probability. This is the case, since in phase 3 all members of the population have equal fitness. Due to our selection for replacement we need to have such a mutation at most k times. Once a bit string is introduced into the population it can never be lost again since the population size is at least k + 1 and the last occurrence of such a bit string cannot be the most frequent worst bit string that is removed. Thus, we have

as upper bound on the expected length of the third phase.

The fourth and final phase starts after the end of the third phase and ends when the optimum is found. Here we rely on crossover. We have all bit strings with n − k 1-bits in a single block as members of the population in this phase. Thus, we can select and as parents for crossover. Now there are n − 2k + 1 crossover points where the 1-point crossover of the two leads to 1 n as offspring. Since we have we have . Therefore, the probability for such an event is bounded below by

and we obtain as upper bound on the expected length of the fourth phase.

Adding the four bounds yields

as claimed. □

If we consider the (1 + 1) EA on R 1, k we see that the expected optimization time is not different from that on { JUMP} k and equals . When we set k = n ∕ 4 we have for the (1 + 1) EA and for the steady-state GA with μ = n and p c  = 0. 5, an exponential speedup due to the use of 1-point crossover. Note that the result for the steady-state GA is very robust with respect to the choice of the population size μ and the crossover probability p c .

For uniform crossover we need a completely different example function to accomplish a comparable speedup. We begin with an informal discussion of the example function that we are going to call R 2.

We have already noticed that we cannot expect uniform crossover to efficiently find a unique global optimum in a situation where mutation needs exponentially long on average. Thus, we design a function with a rather large number of global optimum that form a kind of target area. The target area will be so large the it is easy to hit by means of uniform crossover (provided one uses the right kind of parents) but not so large that it can be found ‘by accident’ without crossover. To achieve this we conceptually divide a bit string of length n into two halves of length n ∕ 2 each. We imagine these two halves ‘living’ in two separate Boolean hypercubes of dimension n ∕ 2. We use this to force the random population into a configuration that we can control. This helps in two different ways. On the one hand it guarantees (in a probabilistic sense, of course) that the global optimum is not found ‘by accident’. On the other hand it helps to generate the kind of parent population that we need for uniform crossover to work. We define the function R 2 in the following definition. A graphical representation of R 2 is given in Fig. 6.1.

Fig. 6.1

Graphical representation of R 2 with some arbitrary , corresponding to two arbitrary points in the respective parts of the search space. Note that the target zone is shown disproportionally large: It occupies parts of a single layer, only

Definition 6.2.

Let with . We define m = n ∕ 2 and k = m ∕ 3. For x ∈ { 0, 1} n we define such that . Moreover, we define such that . We define a so-called circle C and a so-called target area:

For some y ∈ { 0, 1} n and some set we define the Hamming distance of y and A as .

Using this notation the fitness function is defined by

If we pick a polynomial number of search points uniformly at random, none of them will have the x′-part equal to 0 m and none of them will be in C or T with the -part, both simultaneously with probability exponentially close to 1. In this case the fitness value equals and the algorithm is guided toward the circle C. In that time x′ has no influence on the fitness and can perform an unbiased random walk. Once the circle is reached the fitness equals and the algorithm is guided toward 0 m in the left half, x′. The right half is stuck in the circle, , otherwise the fitness value would drop to 0. Once we have x′ = 0 m and x ′′  ∈ C the only possible change is a direct jump into the target zone, x ′′  ∈ T.

All randomized search heuristics will more or less follow this path. We see that the Hamming distance between C and T is , thus we cannot hope to reach it by means of a big mutation. Since fitness points toward C the best one can do with mutation is to ignore fitness and to try an unbiased random walk with . Since the size of the search space is 2 n ∕ 2 this is hopeless. Thus, mutation-based algorithms like the (1 + 1) EA will need an exponential number of steps with overwhelming probability. Uniform crossover, on the other hand, is able to make this jump. We will see in the proof of the following theorem how that works.

Theorem 6.3.

Let be a constant. With , μ ≥ n, , the following holds for the steady-state GA (Algorithm 8) with uniform crossover.

Proof.

We follow the same proof strategy as we did in the proof of Theorem 6.2: We consider four disjoint phases and prove upper bounds on the expected length of each phase in separation.

The first phase starts with an arbitrary population after initialization and ends when we have x ′′  ∈ C for all x ∈ P t for the first time (or the optimum is found). We see that the fitness is defined by the Hamming distance to C. Therefore, this part is no more difficult than optimizing { ONEMAX}; it is in fact simpler due to the larger number of ‘optima’. Thus, we obtain as before.

The second phase starts after the end of the first phase and ends when we have x′ = 0 m and x ′′  = C for all x ∈ P T for the first time (or the optimum is found). Again, this is similar to { ONEMAX}, and we obtain as upper bound.

The third phase is in some sense similar to the third phase in the proof of Theorem 6.2. It starts after the end of the second phase and ends when we have all possible bit strings with x ′′  ∈ C in the population for the first time (or the optimum is found). We observe that there are 2m = n such bit strings and it is important that we have μ ≥ n. In the beginning of this phase we have at least one such bit string in the population already. As long as we do not have all, there is always some with such that holds for some x ∈ P t . Let r be the number of copies x has in the population. If there is more than 1 bit string of this kind (for different y) we pick one where r is maximal. Since the probability to generate y is bounded below by

Trivially, we have r ≥ 1 and obtain

as upper bound since we need to introduce new such bit strings less than n times. If μ is large, however, 1 ∕ μ is really small and it pays to pay attention to the value of r. In each generation, the probability to create copy of x (and thus to increase r by 1) is bounded below by

as long as r ≤ μ ∕ n holds because then this x cannot be removed in the selection for replacement. The expected time needed to increase r to μ ∕ n is bounded above by

Having r ≥ μ ∕ n reduces the expected waiting time to create y to

so that we have

as another upper bound on the expected length of the third phase. Since we have μ ≥ n the second upper bound is better, and we have .

The fourth and final phase starts after the third phase ends and ends when the optimum is found. Remember that we have all possible with in the population in this phase. We concentrate on the case where an optimum is found by means of uniform crossover of and . Let X be the event that the uniform crossover of and produces a global optimum. Then we have

as lower bound to produce the optimum in one generation. Remember that , and the optimum is found if all contain exactly 1-bits. Since the two parents are complementary in the right half the offspring that are created by means of uniform crossover are distributed uniformly at random among all bit strings. Thus, we have

for the probability of such a ‘good’ crossover operation. Together this yields

as upper bound on the expected length of the fourth phase. Adding up the upper bounds we obtain

as claimed. □

Using fairly common choices for the population size and the crossover probability we obtain in comparison to , again an exponential speedup due to the use of crossover.

6.2 Mutation

We have seen in the last section that crossover can speed up evolutionary algorithms exponentially in certain situations. Still, mutation is a very important variation operator that in many situations is able to help locate a solution efficiently without the use of crossover. In comparison to crossover it has the advantage that it does not need an actual population (a single search points suffices) and that it is not influenced by the distribution of the population: If the members of a population become too similar to each other crossover becomes ineffective. In this situation mutations are crucial. Therefore, it is worthwhile to consider mutation in some more depth. In this section we will consider two different aspects. First, we will consider the choice of the mutation probability p m in standard bit mutations. Afterwards we discuss what can be done if something is known about the fitness function that makes standard bit mutations appear not to be an optimal choice. For one such situation we discuss design and analysis of problem-specific mutation operators.

So far, when using standard bit mutations we always have the mutation probability p m  = 1 ∕ n. There are good reasons for this. With this mutation probability the expected number of mutation bits in a single mutation equals 1. This is a good value when we want to locate a single point exactly. If we use a different mutation probability p m  = v ∕ n, the expected waiting time for a specific mutation of a single bit becomes

and we observe that having smaller mutation probabilities (with v < 1) increases the expected waiting time linearly while having larger mutation probabilities (v > 1) increases the expected waiting time exponentially. In this sense, larger mutation probabilities are much more dangerous than smaller ones. On the other hand, if mutation is the only variation operator that is applied, it appears to be wasteful to have p m  < 1 ∕ n since then we expect in one mutation less than 1 bit to flip: In many mutations nothing will change at all. This explains why p m  = 1 ∕ n is actually a good choice.

However, there are situations where p m  = 1 ∕ n is far from being optimal. It is not difficult to see this. Consider the function { JUMP} n and the (1 + 1) EA. We know that holds with mutation probability p m  = 1 ∕ n (Theorem 5.13). If we use p m  = 1 ∕ 2 instead we know that holds: With p m  = 1 ∕ 2 each new offspring is selected uniformly at random. Since { JUMP} n has a unique global optimum the expected time to find this by pure random sampling equals 2 n . However, this example is not convincing us that 1 ∕ 2 is a useful mutation probability. The problem is that while 2 n is very much smaller than n n (smaller by the exponential factor (n ∕ 2) n , actually) it is still exponential. So, while the (1 + 1) EA with p m  = 1 ∕ n will never ever find the optimum of { JUMP} n it will still never find it with p m  = 1 ∕ 2 (unless n is fairly small, of course). An improvement to something that is still exponential is not too impressive. We would like to see an example where the expected optimization time is reduced to a polynomial from something that is superpolynomial or even exponential. In order to see this we create an appropriate example function.

Consider the situation where you want to locate a unique global optimum by means of standard bit mutation that has Hamming distance h. The probability to accomplish this equals . If we want to decide about the optimal mutation probability for this we can set the first derivative to 0, , and observe that p m  = h ∕ n is the only solution and in fact maximizes the probability. How large can a jump be with this mutation probability? If we want to make a jump over h bits we require n − h bits to remain unchanged. This has probability

and the expected waiting time is only polynomial as long as holds. A second observation is that going from p m  = 1 ∕ n to p m  = h ∕ n does not change that much as long as hitting some point exactly is concerned. If we consider expected waiting times we obtain

and see that the expected waiting time is polynomial if and only if n h is. Thus, with respect to polynomial vs. superpolynomial we gain nothing. We realize, that similar to R 2 (see Definition 6.2), we need a larger target area.

These insights present us with all the ingredients we need for our example function. We want to lead the (1 + 1) EA to a place where a jump over bits is required in order to reach a target area of sufficient size. This target area must not be reachable otherwise. All this is the case for the following example function that we call P. A graphical representation is given in Fig. 6.2.

Fig. 6.2

Graphical representation of P. The set F 0 is shaded in gray

Definition 6.3.

Let be given. Let n = 2 k  > 32. We define the following five sets:

Using this notation the fitness function is defined by

We consider the (1 + 1) EA on P with the whole range of useful mutation probabilities p m  ∈ (0, 1 ∕ 2]. We have designed P so that it can be efficiently optimized with a mutation probability p m  = log(n) ∕ n. Therefore, we will use this value as our base case and express other mutation probabilities as deviations from this value.

Theorem 6.4.

Consider the (1+1) EA with mutation probability p m (n) ∈ (0,1∕2]. Let

(0)

Proof.

We know that the initial search point belongs to F 1 with probability . This implies that with this probability the (1 + 1) EA will never enter F 0. We now make a case distinction with respect to the mutation probability p m (n). Large mutation probabilities cause different difficulties than small ones.

First, we consider the case of large mutation probabilities, i.e., , which is equivalent to . With probability the initial search point contains at least n ∕ 4 0-bits and at least n ∕ 4 1-bits. Since we will not enter F 0, the number of 0-bits cannot decrease. The set F 4 is the set of all global optima. In order to enter it we need that at least (n ∕ 4) − logn bits do not mutate. We consider these n ∕ 4 0-bits and know that the expected number of bits that are mutated in a single mutation equals . We call this random number of mutating bits M. We can apply Chernoff bounds to bound the probability that at least (n ∕ 4) − logn bits do not mutate and obtain

for some constant c > 0. The probability to see such an event in at most n cα(n) ∕ 2 steps is bounded above by n  − cα(n) ∕ 2 (by application of the union bound). Since n cα(n) ∕ 2 is not polynomially bounded and we have the desired statement for the case of large mutation probabilities.

Now we consider small mutation probabilities, i.e., which is equivalent to . Remember that with probability the (1 + 1) EA will never enter F 0. For all we have that holds. Thus, the final mutation leading to some optimum in F 4 needs to flip at least logn bits simultaneously. The expected number of mutating bits in one mutation equals . We call this number of mutating bits M′ and bound the probability to mutate at least logn bits by

for some constant c′ > 0. Remember that we have since lim n → ∞ α(n) = 0 holds. As before, the probability to see such an event in n  − (c′ ∕ 2)lnα(n) steps is bounded above by n (c′ ∕ 2)lnα(n). Since , n  − (c′ ∕ 2)lnα(n) is not polynomially bounded and we have the desired statement for small mutation probabilities, too. □

We see that for P the (1 + 1) EA cannot be successful if for the mutation probability does not hold. If the mutation probability is much larger it cannot even come close to an optimum. If the mutation probability is much smaller it cannot make the final jump over the distance of logn. Note that this does not imply that the (1 + 1) EA is actually efficient on P with . It could be that P is just too difficult for the (1 + 1) EA with any mutation probability. We prove that this is not the case and that the (1 + 1) EA with an appropriately chosen mutation probability finds an optimum of P fairly quickly.

Theorem 6.5.

Let c > 0 be constant. Consider the (1+1) EA with mutation probability p m (n) = cln (n)∕n.

for c = 1∕(4ln 2)

Proof.

We use the trivialfitness layers for the proof. For there are always at least n ∕ 4 mutations of single bits that increase the function value. Thus, the probability to leave the current fitness layer is bounded below by

and we obtain

as upper bound on the expected time to leave .

Now we consider the case . We can increase the fitness by increasing the number of 1-bits among the first n ∕ 4 bits from i to i + 1, decrease the number of 1-bits in the other (3 ∕ 4)n bits by one, and do not mutate any other bit. The probability for such a mutation equals

and we obtain

as upper bound on the expected time for this case.

For we have with i > 0, and the fitness increases if we remove the rightmost 1-bit. The probability for this equals

so that we have

as upper bound for the expected time for this case.

Finally, only the case x = 0 n is left. We need to mutate exactly logn bits that must not belong to the leftmost 2logn bits. The probability for such a mutation equals

so that we have as upper bound for expected waiting time for the final mutation.

We add the upper bounds and obtain

as claimed. The upper bound of for c = 1 ∕ (4ln2) is easy to verify numerically. □

We see that the choice of the correct mutation probability can make a decisive difference. However, the example function P also reveals that the choice of the mutation probability is very difficult. Without detailed knowledge about the fitness function it is impossible to choice ‘the right’ mutation probability. This is easy to see. The functions { ONEMAX} and { JUMP} n are structurally almost identical: The fitness depends only on the number of 1-bits in the bit string and is strictly increasing (for { ONEMAX}) or strictly decreasing (for { JUMP} n ) with this number. The decisive difference is that the global optimum for { ONEMAX} coincides with this tendency, whereas for { JUMP} n it is at the opposite end of the search space. With mutation probability p m  = 1 ∕ n the (1 + 1) EA optimizes { ONEMAX} efficiently, whereas for { JUMP} n using p m  = 1 ∕ 2 reduces the expected optimization time by a factor of . Clearly, we cannot know this if we do not know the fitness function precisely. Since this is knowledge that we do not have in practical situations, this is unsatisfactory. We would like to have a more practical simple solution for setting the mutation probability.

One may argue that p m  = 1 ∕ n is very often a good choice and deviations for this are hard to find. However, they exist. Therefore, we consider a simple alternative where we equip the (1 + 1) EA with a dynamic parameter setting.

When setting the mutation probability for the (1 + 1) EA we always have p m  ≤ 1 ∕ 2. We have already discussed that having p m  ≥ 1 ∕ n makes sense if standard bit mutations are the only variation operator and that larger mutation probabilities are ‘more dangerous’ than smaller ones. Therefore, we consider a scheme for setting the mutation probability dynamically that uses small mutation probabilities much more often than large ones. We achieve this by setting p m  = 1 ∕ n initially and doubling it in each step until it grows larger than 1 ∕ 2. In that case we set it back to 1 ∕ n. The resulting algorithm is called the dynamic (1 + 1) EA and is formally defined as Algorithm 9.

We see that the dynamic (1 + 1) EA uses mutation probabilities , together different mutation probabilities of the form 2 i  ∕ n.

If we have a proof of (for a mutation probability p m that is also used by the dynamic (1 + 1) EA, e.g., for p m  = 1 ∕ n) by means of fitness layers or drift analysis we can usually easily obtain : We adopt the proof for the mutation probability that is used by the (1 + 1) EA and the dynamic (1 + 1) EA and ignore the other mutation probabilities used by the dynamic (1 + 1) EA. While this appears to be quite wasteful, it leads to asymptotically tight upper bounds for many functions, e.g., for{ ONEMAX},{ BINVAL},{ LEADINGONES}, and{ RIDGE}. We see that using the dynamic parameter control instead of setting the mutation probability ourselves comes at the price of an additional factor of . Taking into account that we are using a general randomized search heuristic, such a small additional factor is usually considered to be acceptable. For example, when optimizing linear functions the (1 + 1) EA needs expected time (Theorem 5.22) and is considered to be efficient even though such functions can easily be optimized in time . However, we clearly need to see more positive news to consider using the dynamic (1 + 1) EA.

The exponential parameter setting scheme of the dynamic (1 + 1) EA guarantees that it uses a mutation probability that is less than a factor 2 away from any desired mutation probability p ∈ [1 ∕ n, 1 ∕ 2]. We can use this observation to prove an improved general upper bound on the expected optimization time of the dynamic (1 + 1) EA (in comparison to the (1 + 1) EA). The general upper bound remains exponential, of course.

Theorem 6.6.

For any function , holds.

Proof.

The dynamic (1 + 1) EA uses a mutation probability p m  ∈ [1 ∕ 4, 1 ∕ 2]. In steps with this mutation probability an optimum of f is reached with probability at least . Ignoring the steps with other mutation probabilities yields the upper bound 4 n logn. □

When considering the example function P we have seen that setting the mutation probability appropriately can be really difficult. The dynamic (1 + 1) EA has no difficulties optimizing this function.

Theorem 6.7.

Proof.

For all steps except the final mutation we consider only the mutation probability p m  = 1 ∕ n. For there are always at least n ∕ 4 mutations of single bits that improve the fitness value. Thus, the probability for leaving a fitness layer is bounded below by , and we obtain as upper bound on the number of such steps for this case.

For we can leave the current fitness level by increasing the number of 1-bits among the first n ∕ 4 bits from i to i + 1, decrease the number of 1-bits in the other (3 ∕ 4)n bits by one, and do not mutate any other bit. The probability for such a mutation equals

and we obtain

as upper bound on the expected number of mutations with mutation probability p m  = 1 ∕ n for this case.

For we have with i > 0 and the fitness increases if we remove the rightmost 1-bit. The probability for this equals so that we have as upper bound for this case.

Only for the final mutation, i.e., for x = 0 n , we consider the mutation probability that is used. We know from the proof of Theorem 6.5 that the expected waiting time for such a mutation with mutation probability cln(n) ∕ n is . We know that we have some c ∈ [1, 2]. Considering Fig. 6.3 we observe that is an upper bound for the expected number of mutations with this mutation probability, we have to wait until finally finding an optimum.

Fig. 6.3

Graph of c − log(c) − log(ln(2)) and 1. 53 for c ∈ [1, 2]

Adding up the expected waiting times for mutations with the specific mutation probabilities we obtain

Since we need to wait for the specified mutation probability, each time we have an additional factor of so that follows. □

We see that we obtain a better upper bound for the dynamic (1 + 1) EA than for the (1 + 1) EA even with a carefully chosen mutation probability. The reason is that for P one would like a small mutation probability like p m  = 1 ∕ n most of the time. Only in the end for the final mutation is desirable. While with the (1 + 1) EA we have to look for a compromise, the dynamic (1 + 1) EA uses the appropriate mutation probability (together with many inappropriate ones) all the time. This enables the dynamic (1 + 1) EA to efficiently optimize functions where the (1 + 1) EA fails completely, regardless of the mutation probability.

Definition 6.4.

We define a target area T:

The fitness function is defined by

A graphical representation of the example function can be found in Fig. 6.4. The function is very similar to { RIDGE} except for the global optima. The target set T is introduced and contains all global optima (making the all ones bit string that is the unique global optimum of { RIDGE} a local optimum). Due to the large Hamming distance of at least n ∕ 16 to each point on the ridge, these global optima are difficult to reach by means of mutation unless the mutation probability is very large. Since the dynamic (1 + 1) EA also uses very large mutation probabilities it has no difficulties on D 1.

Fig. 6.4

Graphical representation of D 1

Theorem 6.8.

Proof.

We know that the expected optimization time of the (1 + 1) EA with mutation probability p m  = 1 ∕ n on { RIDGE} is (Theorem 5.14). The upper bound was proved by means of fitness-based partitions and carries over with an additional factor of . This gives an upper bound of to reach the local optimum 1 n .

We now need to consider the final mutation. We only consider steps with a mutation probability p m  ∈ [1 ∕ 8, 1 ∕ 4]. The expected number of flipping bits equals . The number of flipping bits is binomially distributed and is concentrated around the mean value. It is therefore not difficult to show that the number of flipping bits is in the interval [n ∕ 8, n ∕ 4] with probability . What we are missing is the probability that a search point with k 0-bits (with k ∈ [n ∕ 8, n ∕ 4]) belongs to T. We know that this probability equals

and prove a lower bound as follows. Some x with { ONEMAX}(x) = k is not in T if its Hamming distance to any point on the ridge is less than n ∕ 16. The only points on the ridge that are candidates for this are with i ∕ n ∈ [11 ∕ 16, 15 ∕ 16]. Consider one such point. Now we pick a point with j 0-bits uniformly at random, j ∕ n ∈ [3 ∕ 4, 7 ∕ 8]. The expected Hamming distance of these two points equals since it is the sum of two hypergeometrically distributed random variables: Having fixed the ridge point of length n with i 1-bits we can describe the random experiment of determining the other point as picking the positions of the j 0-bits randomly (without replacement). Each time we pick one of the i 1-bits in the ridge point the Hamming distance increases by one. Since we select without replacement the number is hypergeometrically distributed. The same holds for the 1-bits. We observe that holds (we have equality for i ∕ n = 15 ∕ 16 and j ∕ n = 7 ∕ 8). The hypergeometric distribution is even more concentrated around the mean than the binomial distribution. Thus, the probability of a Hamming distance of less than n ∕ 16 = (4 ∕ 64)n is exponentially small. Thus, the probability that such a mutation generates a search point from T is . Therefore, the expected waiting time for such a mutation is and together the upper bound follows. □

While the dynamic (1 + 1) EA has no difficulties with D 1, a (1 + 1) EA with a fixed mutation probability has no chance of finding an optimum in polynomial time. The proof is relatively short and straightforward

Theorem 6.9.

With any fixed mutation probability p m  ∈ (0,1∕2], the probability that the (1+1) EA finds an optimum of D 1 in a number of steps that is even bounded above by a sufficiently small exponential function is exponentially close to 0.

Proof.

The initial search point contains at most (5 ∕ 8)n 1-bits with probability . Each search point with a fitness that is not smaller than that of the initial search point has Hamming distance at least n ∕ 16 to each global optimum of D 2. Therefore, in order to reach T a mutation that flips at least n ∕ 16 bits simultaneously is required. Chernoff bounds yield that the probability for such a mutation is exponentially close to 0 if p m  < 1 ∕ 32 holds.

We consider the case of p m  ∈ [1 ∕ 32, 1 ∕ 2]. We start with a search point with at most (5 ∕ 8)n 1-bits with probability . Since the mutation probability is we expect to decrease the number of 1-bits in each mutation by . Deviations by more than constant factors happen only with probability exponentially close to 0. The fitness can only be improved if a point on the ridge is found with at most (5 ∕ 8)n 1-bits. Even if this happens, the probability to find a point from T or another point on the ridge from there is exponentially close to 0. □

What we have seen is quite encouraging: The dynamic (1 + 1) EA is easy and convenient to use since we need not worry about setting the mutation probability. In many cases it loses a factor of in comparison to the (1 + 1) EA with a fixed mutation probability, which we consider to be not too bad. And it can outperform the (1 + 1) EA on functions where an appropriate mutation probability is difficult to find (like P) and even more so on functions where no good fixed mutation probability exists (like D 1). So, the dynamic (1 + 1) EA is indeed an interesting and potentially more robust alternative. However, before we get too excited we should remind ourselves that no search heuristic can outperform any other on all problems. This is a simple consequence of the NFL (Theorem 4.3). It may be even more convincing to see an actual example and experience how large the disadvantage of the dynamic (1 + 1) EA can become and what the source of such a disadvantage may be. We accomplish this by considering another example function, called D 2. A graphical representation of D 2 can be seen in Fig. 6.5.

Definition 6.5.

We define a trap area T:

The fitness function is defined by

We first prove that the (1 + 1) EA is efficient on D 2. This even holds for the most commonly used mutation probability p m  = 1 ∕ n. Note, however, that we cannot show this with respect to the expected optimization time. The fitness function D 2 is one of the cases where a large expected optimization time would be misleading: An algorithm may get trapped in the trap area T with very small probability but be caught in it for so long that this dominates the average optimization time. Therefore, we make a statement with high probability instead.

Theorem 6.10.

Consider the (1+1) EA with mutation probability p m = 1∕n.

Fig. 6.5

Graphical representation of D 2

Proof.

The probability that the (1 + 1) EA optimizes n − { ONEMAX}(x) in time is exponentially close to 1. In this time it visits different search points x with { ONEMAX}(x) ∕ n ∈ [1 ∕ 4, 3 ∕ 4]. Because of symmetry each point with i 1-bits has equal probability of being visited. The number of points with i 1-bits for i ∕ n ∈ [1 ∕ 4, 3 ∕ 4] is bounded below by . Therefore, the probability to hit a point in T in this time on D 2 is exponentially small. Once the (1 + 1) EA is on the ridge T can only be reached by means of a mutation that flips at least n ∕ 24 bits. Such a mutation has probability at most (24 ∕ n)! . So, the probability to have such a mutation in is still exponentially small. Once on the path we need at most n specific mutations of single bits to reach the optimum. Each of these mutations has probability . The probability not to have at least n such mutations in 2en 2 steps is exponentially small. This implies the claim. □

We see that the (1 + 1) EA with mutation probability p m  = 1 ∕ n has no difficulties in locating the optimum in steps. Since the dynamic (1 + 1) EA uses this mutation probability once in each phase of length it should have no difficulties doing the same in steps—unless the use of larger mutation probabilities leads it somewhere else before it finds the global optimum. We will prove that this is indeed the case: The use of larger mutation probabilities lets the dynamic (1 + 1) EA find the trap where it is caught for a very long time with probability very close to 1. Note that this is almost the same as we proved for D 1 with the decisive difference that for D 1 the area T contained the global optima where for D 2 it does not.

Theorem 6.11.

The probability that the dynamic (1+1) EA finds the optimum of D 2 in a number of steps that is even bounded above by a sufficiently small exponential function is exponentially close to 0.

Proof.

Let us first assume that the trap T is entered at some point of time. If that happens a mutation from the current search point into the unique global optimum is needed. The Hamming distance is at least n ∕ 4, there the probability for such a mutation is bounded above by p m n ∕ 4. For any p m  ≤ 1 ∕ 2 this is exponentially small and the probability to see such a mutation even in an exponential number of steps is still exponentially small.

We need to prove that the probability to eventually fall into the trap is exponentially close to 1. We know that the dynamic (1 + 1) EA is also efficient on n − { ONEMAX}. Hence we can conclude that it has either fallen into the trap or found a point on the ridge with at most n ∕ 4 1-bits after steps with probability exponentially close to 1.

We now consider the time during which with i ∕ n ∈ [1 ∕ 4, 3 ∕ 4] holds. This time interval has a length of at least or one of the following two events happens: It finds another point on the ridge with Hamming distance at least or it falls in the trap. The probability to find another point on the ridge in such a large distance is with and p m  ≤ 1 ∕ 2. Therefore, it is exponentially close to 0. Now we prove a lower bound on the probability to reach the trap. We consider a mutation probability p m  ∈ [1 ∕ 12, 1 ∕ 6]. The expected number of mutating bits is in the interval [n ∕ 12, n ∕ 6] and the actual number of mutating bits is in this interval with probability . The fraction of points with such a Hamming distance to the current point on the ridge that have a Hamming distance of at most n ∕ 24 to the ridge is exponentially small. Thus, in each of these mutations the dynamic (1 + 1) EA has a probability of at least to fall into the trap. In the time interval we consider there are chances for such a mutation. Therefore, the probability not to fall into the trap is bounded above by , which is exponentially small. Hence, the probability to fall into the trap is and hence exponentially close to 1. □

One other topic that we want to discuss with respect to mutation is the design of problem-specific mutation operators. The idea is that we know something about the kind of problem we are dealing with and that we would like our evolutionary algorithm to benefit from this knowledge. Black-box complexity as well as the NFL suggest that this is actually a good idea. We consider one specific example and will design a mutation operator that takes advantage of the special properties. One consequence is that the new mutation operator is biased. We have seen in Sect. 2.5 that there are good reasons not to design such variation operators and that one should be careful when still doing this. We keep this in mind and discuss benefits as well as drawbacks of the new mutation operator.

Let us consider a problem class where we know that an optimal solution will have a very small number of 1-bits. A natural example for such a problem class is the search for a minimum spanning in a dense graph: Consider a graph with v nodes. We call such a graph dense if the number of edges n is much larger than linear in the number of nodes, say . Let the graph be weighted, i.e., each edge has some positive weight. The problem is to find a selection of edges so that these edges connect every two nodes in the graph, and among all these selections one where the total sum of the weights of the selected edges is minimal. Since the weights are all positive the solution will be a tree (hence the name, minimum spanning tree (MST)). A tree with v nodes has exactly v − 1 edges, which is very small in comparison to the number of edges in our dense graph. If we use standard bit mutations for this problem we know that the mutation operator has the tendency to steer the search toward points with n ∕ 2 edges: If the current number of 1-bits is k the expected number of 1-bits after mutation (with mutation probability p m  = 1 ∕ n) equals

which is larger than k for k < n ∕ 2 and smaller than k for k > n ∕ 2. In a graph with nodes the mutation operator tends to look for solutions with edges. But we know that the solution is not there, it is a selection of v − 1 edges.

We know that this tendency of standard bit mutations is not necessarily a big issue: Remember { ONEMAX} where the unique global optimum is 1 n . That standard bit mutations steer toward search points with n ∕ 2 1-bits is not an issue, selection is easily able to counteract that. However, it is this tendency toward the middle that slows the search down toward the end. The closer we get to 1 n the less likely it becomes to get closer. It would be nice to have a mutation operator that does not have such a tendency away from the area where we know that the solution is going to be.

To this end, we define asymmetric mutations. When mutating some x ∈ { 0, 1} n we make the probability to flip a bit depend on the value of the bit. We decide for each bit independently to flip it with probability if it is a 1-bit and with probability otherwise. In some sense the factor 2 is an arbitrary choice. The perhaps most obvious choice 1 is not a good idea: If we have x ∈ { 0, 1} n with { ONEMAX}(x) = 1 the only remaining 1-bit would be flipped with probability 1. Since mutation should be random we avoid this deterministic behavior.

It is easy to see that this mutation operator has no tendency to change the number of 0-bits. If the current number of 1-bits is k the expected number of 1-bits after one asymmetric mutation equals

so that with respect to the number of 1-bits we expect no change. Such random processes where in one step no change is expected are known as martingales (see Appendix A.3). We keep this in mind since it may be useful in an analysis.

We remark that this mutation operator is not unbiased in the sense of Sect. 2.5: Consider 10 n − 1 and Hamming distance as natural metric. Clearly, and 0 n both have Hamming distance 1. However, we have

and

We consider a variant of the (1 + 1) EA where we replace standard bit mutations by these asymmetric mutations and leave everything else unchanged. We call this algorithm the asymmetric (1 + 1) EA.

We have seen that the probability for mutations of single bits can be very different for asymmetric mutations in comparison to standard bit mutations. Since such mutations are important to locate search points exactly we care about them. The next lemma shows that they are still not too unlikely. In fact, they are never asymptotically less likely than when using standard bit mutations where they occur with probability . In order to shorten notation we use and as abbreviations with and .

Lemma 6.1.

Let x ∈{ 0,1} n . The probability to mutate exactly one specific b-bit in x (with b ∈{ 0,1}) is bounded below by

Proof.

We consider the case b = 1, the other is symmetric. If the probability to flip exactly one 1-bit equals

since (and the other term is even larger). For x = 1 n we have

so the claim follows. □

An immediate consequence is that the general upper bound for unimodal functions (Theorem 5.6) carries over.

Corollary 6.1.

Let be aweakly unimodal function, and let denote the number of different function values.

We remember the generalization of a function f to f a via f a (x) = f(x ⊕ a) (see Definition 4.5). The motivation for this was that for the (1 + 1) EA dealing with f or f a makes no difference. This is different for the asymmetric (1 + 1) EA. It is true that it is symmetric with respect to 0-bits and 1-bits (in the sense that we can exchange all 0-bits by 1-bits and vice versa and nothing changes for the algorithm), but it is sensitive with respect to the number of 1-bits. We will see examples that it behaves very differently on f and f a . However, due to its symmetry with respect to 0-bits and 1-bits, it behaves identically on f a and (for all a ∈ { 0, 1} n ). Therefore, we restrict ourselves to the case in the following without loss of generality.

The first concrete function we consider is { ONEMAX}. It is an interesting test case because the unique global optimum is a bit string with a very small number of 0-bits: It contains no 0-bits at all. Thus, we expect asymmetric mutations to be a good mutation operator for this function. And our expectations are actually met, we outperform the (1 + 1) EA and get rid of the factor as the following theorem shows.

Theorem 6.12.

Proof.

We use the method of fitness-based partitions and use the trivial fitness layers. If the number of 0-bits equals i the probability to remove one of these 0-bits and flip no other bit is bounded below by (Lemma 6.1). This yields

as upper bound on the expected optimization time. □

It is nice to see that our simple idea of speeding up the search works so nicely. However, one may fear that the NFL theorem (Theorem 4.3) suggests that we will pay with worse optimization times for { ONEMAX} a when the transformation moves the unique global optimum closer to the middle of the search space. We observe that the NFL theorem itself does not allow for any such prediction for two reasons: First, it speaks about distinct function evaluations and not about expected optimization times. If a search operator would simply reduce resampling of search points it could lead to a general speedup without any negative consequences. Second, { ONEMAX}(b) is not closed under permutation of the search space. Pessimists may still believe that not steering the search toward bit strings with n ∕ 2 1-bits slows the search down when the optimum actually has n ∕ 2 1-bits. The following theorem shows that at least asymptotically this is not the case. Note that the 2 in the logarithm takes care of the case , where the logarithm would either be undefined or become 0.

Theorem 6.13.

For all a ∈{ 0,1} n the following holds:

Proof.

As we have already discussed, it suffices to do the proof for since the other case is symmetric. Remember that the unique global optimum of { ONEMAX} a is and we have . For the statement follows from Theorem 6.12. Thus, we consider the case with .

We consider two phases with lengths T 1 and T 2 such that holds. The first phase starts after initialization and ends when holds for the first time. The second phase starts after the end of the first phase and ends when the optimum is found. We use the method of fitness-based partitions and use the trivial fitness layers to derive upper bounds on the expected length for both phases separately.

In the first phase we have by definition of the phase. Therefore, there are at least positions where x has a 0-bit while has a 1-bit. Flipping such a 0-bit decreases the Hamming distance to and therefore increases the function value. Due to Lemma 6.1 the probability for such a mutation is bounded below by . This implies .

In the second phase we have by definition of the phase. We consider the all ones bit string as reference point and have on the one hand, and on the other hand. This implies since the Hamming distance is a metric. We obtain as a direct consequence for each x in the second phase. The probability to increase the current fitness layer is bounded below by (n − { ONEMAX} a (x)) ∕ (8n) (again by Lemma 6.1) and

follows. This yields

for the case . If we can exchange the roles of 0-bits and 1-bits. This yields

as claimed. □

We see that the asymmetric (1 + 1) EA is asymptotically never outperformed by the (1 + 1) EA on { ONEMAX} a ; its expected optimization time is always bounded above by . The upper bound can be smaller than that but this is only the case if the number of 1-bits in the target string is either really large or really small. We need to have for all . This is a realistic assessment. A matching lower bound proves that the asymmetric (1 + 1) EA is just as fast as Theorem 6.13 suggested.

Theorem 6.14.

For all a ∈{ 0,1} n the following holds:

Proof.

Due to Theorem 6.13 we only need to prove the lower bound. As before we can restrict ourselves to . We distinguish three cases with respect to .

First we consider the case . The expected initial Hamming distance to the optimum equals n ∕ 2. The expected number of flipping bits in one mutation is bounded above by 1: It is 1 ∕ 2 for and 1 otherwise. Thus, the expected decrease in Hamming distance in one generation is bounded above by 1. Drift analysis (Theorem 5.15) implies .

Now we consider the case . Since the mutation operator is completely symmetric with respect to bit positions, we can assume without loss of generality. With probability 1 ∕ 2 the initial search point contains at least 1-bits in the first positions. Also, with probability 1 ∕ 2 it contains at least 1-bits in the other positions. Thus, with probability at least 1 ∕ 4 both are the case. We work under the assumption to have such an initial search point x in the following.

We first make a statement about the number of 1-bits during the run. For this we make use of the lower bound on the number of 1-bits in the rear part of x. Clearly, on { ONEMAX} a the Hamming distance to the optimum cannot increase. The unique global optimum is . We see that the number of 1-bits is always bounded below by since at the end we will (again) have all 1-bits that we had initially in the rear part, many, and will have lost all 1-bits in the front part, at most many. Since we have in this case follows. Therefore, the probability to flip a 1-bit is bounded above by 4 ∕ n in this phase.

Now we remember that we have as lower bound on the number of 1-bits in the front part. All these bits need to flip at least once for the optimum to be reached. The probability that after generations there is still at least one of these bits that has never flipped is bounded below by

Using the described initialization and the unflipped bits jointly as events, the method of the typical event yields

for this case.

We are left with the case . The initial Hamming distance to the global optimum is bounded below by n ∕ 4 and above by (3 ∕ 4)n with probability exponentially close to 1. We have already seen that the expected number of bits that mutate in a single asymmetric mutation is bounded above by 1. Chernoff bounds yield that the probability to mutate at least n ∕ 16 bits in a single mutation is exponentially close to 0. Thus, with probability there is a point of time when holds. Since the Hamming distance cannot increase, the number of 1-bits is always bounded below by (n ∕ 4) − (n ∕ 8) = n ∕ 8 and bounded above by (3n ∕ 4) + (n ∕ 8) = 7n ∕ 8. We see that we always have 1-bits and 0-bits in the current bit string in this case. Thus, the mutation probability is always for each bit. Thus, the lower bound can be proved in the same way as the general lower bound for p m  = 1 ∕ n (Theorem 5.11). □

It is interesting to observe that the speedup due to asymmetric mutations can be more significant than it is for { ONEMAX} for another simple example problem that has its unique global optimum also in the all ones bit string. For { LEADINGONES} this is the case as the following theorem shows.

Theorem 6.15.

Proof.

We consider two disjoint phases with length T 1 and T 2. The first phase starts after initialization and ends when holds for the first time. The second phase starts after the first phase and ends when the optimum is found.

We know that the probability for a specific mutation of a single bit is always bounded below by 1 ∕ (8n) (Lemma 6.1). Since such mutations are sufficient to end the first phase we have .

For the second phase we employ drift analysis and use as distance measure. Consider some generation t with current search point x t , offspring y and new search point x t + 1. Let A denote the event that { LEADINGONES}(y) ≥ { LEADINGONES}(x t ) holds. We have

since selection guarantees . The offspring y is not worse than its parent if the leading 1-bits are not flipped. Thus, holds. In such a mutation we have

This implies

Therefore, we have

as lower bound on the drift. Since the initial distance is bounded above by n, follows and summing up the two bounds yields the result. □

So far we have seen that the asymmetric (1 + 1) EA can beat the (1 + 1) EA with standard bit mutations in cases where the optimum is a bit string with very large or very small number of 1-bits. However, this is not necessarily the case. We consider { RIDGE} a as an example where the location of the optimum has no influence at all.

Theorem 6.16.

Proof.

Since { RIDGE} is unimodal (and therefore, { RIDGE} a is too) the upper bound follows from Corollary 6.1. For the lower bound we employ drift analysis and use the Hamming distance to the global optimum as distance measure. With probability the first point on the ridge the algorithm encounters has Hamming distance from the optimum. Thus, there is a sequence of Hamming neighbors with strictly increasing function values that all have Hamming distance to the global optimum and that need to be traversed by the asymmetric (1 + 1) EA. We have already seen that the number of bits flipped in a single mutation is bounded above by cn for any constant c > 0 with probability exponentially close to 1. Therefore, possibly after some initial steps, the number of 1-bits and the number of 0-bits in the current search point are both while the Hamming distance to the optimum is still . Thus, the mutation probabilities are , and this implies that the drift is , too. This implies as claimed. □

The influence of the variation operator is strongest when the influence of selection is weakest. We consider two extreme examples where this becomes obvious. The first is{ PLATEAU}, which is similar to { RIDGE} but where on the ‘ridge’ (on the plateau, obviously) fitness is not there to guide the algorithm toward the optimum. One may speculate that this is not a problem on { PLATEAU} since the global optimum is once again the all ones bit string, an ideal situation for asymmetric mutations. It may therefore come as a surprise that the asymmetric (1 + 1) EA is very inefficient on { PLATEAU}.

Theorem 6.17.

Proof.

The probability to flip at least n 1 ∕ 4 bits simultaneously in one mutation is . Application of the union bound implies that we see such a mutation in steps is bounded above by . Thus, we can ignore such huge mutations in the following.

The first search point on the plateau has at most 3n 2 ∕ 3 1-bits with probability . We restrict our attention to the time on the plateau and to steps where the offspring replaces its parents. We consider phases of n 5 ∕ 12 of these steps. We concentrate on the subplateau . The first phase starts when S is entered for the first time and ends after n 5 ∕ 12 steps where the offspring replaced its parent. The i{ th} phase starts after the (i − 1){ th} phase ends and when a point from S is current search point. We do not count steps between two phases. Clearly, this can only lead to underestimating the actual optimization time. We prove that during one phase it is extremely unlikely to leave S in the direction of the global optimum.

Let C  +  denote the event that in one of the steps we consider the number of 1-bits is increased. A necessary condition for a step to increase the number of 1-bits by i () is that i specific 0-bits are flipped. Since we have it follows that holds. Thus, we have

as upper bound on the probability to make a step in the direction toward the optimum.

Let C  −  denote the event that in one of the steps we consider the number of 1-bits is decreased. In order to do this it suffices to flip the rightmost 1-bit (note that we have x ∈ S and therefore there is a 1-bit). By Lemma 6.1 we have

and notice a tendency toward decreasing the number of 1-bits, i.e., moving away from the optimum. Considering only steps where we actually move we have

as upper bound for the conditional probability to increase the number of 1-bits. Let be this probability.

We said that we increase the number of 1-bits by at most n 1 ∕ 4 with sufficiently large probability. We consider n 5 ∕ 12 relevant steps where we increase the number of 1-bits R times. In the remaining n 5 ∕ 12 − R steps we decrease the number of 1-bits. Clearly, we decrease it by at least 1 each time. Thus, at the end of the phase we have increased the number of 1-bits by at most

and see that this is only positive if holds. Thus, we need to have at least n 1 ∕ 6 such steps to have moved toward the global optimum within this phase. The probability for this is bounded above by

In order to reach the optimum at least one such event needs to occur and the claim follows. □

It is important to note that it is not the case that asymmetric mutations are per se unable to cope with plateaus. The precise location of the plateau in the search space matters, as the next theorem shows.

Theorem 6.18.

Let

Proof.

Finding some point on the plateau is not harder than optimizing { ONEMAX} a . Thus, Theorem 6.13 implies that on average this is the case after steps. Once the plateau is reached it is never left. Original plateau points have the form . Thus, for { PLATEAU} a the plateau contains only points with n ∕ 2 or (n ∕ 2) + 1 1-bits and these are alternating. Thus, the two neighbors of each plateau point that are also on the plateau have equal number of 1-bits. Therefore, they are reached with equal probability. Moreover, the mutation probability for each bit is always . This implies that follows as it does for the (1 + 1) EA (Theorem 5.24). □

We conclude our consideration of asymmetric mutations with one extreme example. On { NEEDLE} selection has no say at all since all points have equal fitness (unless the optimum is found). Since the global optimum is the all ones bit string one may hope that asymmetric mutations have an advantage. This is actually the case to an extreme degree.

Theorem 6.19.

Proof.

Like in the proof of Theorem 6.17, we restrict our attention to steps where the number of 1-bits in the current search point changes. Lemma 6.1 implies that the probability for such a step is bounded below by . Thus, the expected optimization time is at most by a factor 8 larger than the bound that we prove when ignoring other steps.

Since asymmetric mutations are completely symmetric with respect to bit positions we can characterize the current state by the number of 1-bits. Remember that with respect to the number of 1-bits the process is a martingale. Consider a martingale with and for all . Let T be the first point of time where X i  ∈ { 0, n} holds. We claim that and hold and prove this at the end. Until then we work under the assumption that both hold.

We observe that the process of steps we consider is such a martingale. Let p i, j denote the probability to change the number of 1-bits from i to j in one step. We assume (and will later prove) that holds. Thus, regardless of the starting point after at most n 2 ∕ 4 steps we have either reached 0 n or 1 n . If the process reached 1 n we are already done. Let T′ denote the random number of steps needed to get from 0 n to 1 n . We know that if the first point we reach from 0 n contains i 1-bits we will return to 0 n with probability 1 − i ∕ n. Therefore,

holds, and we obtain

from this. Together this implies by summing up the upper bounds for and .

We still need to prove the claims about the martingale, and . For an arbitrary random variable Z we define to shorten notation.

We use to define a random process and consider

In the sum, we have

for each of the summands. Since for any random variable Z

holds we have

where the last equation holds since is a martingale. Together this yields

and we observe that is also a martingale (with respect to ). We have T < ∞ since there is always a positive probability to reach either 0 or n in at most n ∕ 2 steps. Moreover, for t ≤ T we have and so we can apply the optional stopping theorem (Theorem A.3). Thus, we have as well as

Since we have , follows, and we obtain . Note that holds so that we have

Since is also a martingale we can apply the optional stopping theorem to it and obtain . On the other hand, holds, so that follows, which in turn implies as claimed. □

Note that { NEEDLE}(b) is closed under permutation of the search space. Thus, the NFL theorem (Theorem 4.3) implies that the huge advantage on { NEEDLE} is offset by a disadvantage on other { NEEDLE} a .

6.3 Cooperative Coevolution

Evolutionary algorithms perform a random search based on a population of search points. Coevolutionary algorithms are similar but (usually) have more than one population. The most important difference is that there is not a clearly defined fitness function that serves as objective function for the algorithm. The different populations need to act together to yield a fitness.

One example where such coevolutionary algorithms could be used is the evolution of sorting nets. A sorting net is a graph where each node takes two numbers as inputs, compares them, and outputs them in sorted order. A sorting network is made up of such nodes and is supposed to sort n numbers that are given as input. Clearly, one would like to have small sorting networks (because they are cheap to build) and sorting networks of small depth, i.e., the longest path from an input to an output has small length (because they sort fast). Moreover, one would like the sorting network to sort correctly, obviously. In order to assess the latter one could in theory use all n! possible inputs and count how many of them as sorted correctly. Clearly, in practice this cannot be done since the computational effort is too big (unless n is very small). One could use some test cases or try random inputs. However, it may be that some inputs are harder to sort correctly than others, but it is hard to tell which inputs this could be. A solution can come here in the form of coevolution: One uses two populations, one containing sorting networks, the other inputs. Both have fitness values that depend on the other population. The fitness of sorting networks depends on the inputs since the number of correctly sorted inputs is important. The fitness of inputs depends on the sorting network because we prefer inputs which are sorted incorrectly (since we want to evolve difficult to sort test inputs).

Another popular example is evolving strategies to play some game. Strategies compete against each other, so that the fitness of a strategy depends on the other strategies that are currently present. This is an example where coevolution with a single population could be applied because strategies and their opponents are conceptually the same. However, the common trait of coevolutionary algorithms remains, and the fitness of an individual can only be assessed with the help of other individuals.

In the context of optimization of some (fitness) function one can also apply coevolution in spite of a fitness function being given. The examples for coevolution above have in common that we have competing populations. An increase in fitness for some individual implies that the fitness of other individuals decreases. This concept is called competitive coevolution. This is in contrast to cooperative coevolution where individuals work together in order to jointly maximize fitness.

In cooperative coevolution we want to maximize a fitness function . In order to do this we apply k evolutionary algorithms that work independently but cooperate for fitness evaluations. We (as designer of the cooperative algorithm) divide the bit string (and by this the search space) into k disjoint components. Each of these k components is a bit string that contains an arbitrary collection of the original bits. Jointly the k components form the original bit string x ∈ {0, 1} n . How we choose the number of components k and how we distribute the bits to the components is a design decision that we have to make. Clearly, the choices will have impact on the performance of the coevolutionary algorithm. We observe that this approach provides us with a principled way to incorporate domain knowledge: If we know that potential solutions consist of several pieces we can divide a solution into these pieces and have evolutionary algorithms work separately on these components. Since the individual algorithms join forces for fitness assessment, it is not necessary that we are able to assign a fitness value for an individual component (which we usually are unable to do).

In designing a coevolutionary algorithm we have to make even more decisions than for an (ordinary) evolutionary algorithm. Let us assume that our domain knowledge has helped us in deciding about partitioning the bit strings into k components. Clearly, for each of the k evolutionary algorithms we need to make all design decisions that we make for evolutionary algorithms. In addition to this we have to decide about the concrete form of cooperation. This consists of two main decisions. Cooperation is required when one of the k evolutionary algorithms needs to assess the fitness of one individual. In order to do this one needs to pick one representative from each of the other evolutionary algorithms in order to assemble a complete bit string. One could decide to pick a best individual from the population, a random one, or use some other decision mechanism. Moreover, one could do this only once or do this multiple times and use some form of averaging or other aggregation mechanism that in the end yields some fitness. So, deciding about the representatives from other populations and fitness aggregation (in the case of multiple samples) is the first design decision. The second design decision is the question of fitness assignment. The fitness evaluation was initiated because it was needed for some individual in some specific evolutionary algorithm. Clearly, the fitness value that is computed is used for this individual. However, it could also be used for other representatives that have contributed to it.

Our analysis of evolutionary algorithms has concentrated on example functions. We have only considered specific fitness functions that we considered to be interesting or instructive. At times, we have designed specific example functions in order to prove some specific point. ‘Real’ problems have not been part of our research agenda so far. (We will deviate from this restriction and consider some combinatorial optimization problems in the next section.) Due to this restriction to example functions there is not much sense in talking about how we want to divide a bit string of length n into k components. We therefore use the following simple mechanism. We assume that and divide a bit string x ∈ { 0, 1} n into k components of equal length. The components are called and are simply consecutive chunks from x. We define this formally in the following definition.

Definition 6.6.

Let such that . We define l : = n ∕ k. For x ∈ { 0, 1} n we call the partition of x into with for each the canonical partition of x. We have .

When analyzing evolutionary algorithms we started with the (1 + 1) EA as a particular simple example that helped us to understand important aspects of evolutionary algorithms without being overwhelmed by the complications of complete evolutionary algorithms with populations and crossover. Going from there we have been able to develop results for more complex evolutionary algorithms using analytical methods that we have developed for the (1 + 1) EA. We adopt this useful research strategy here, too. We will define a cooperative coevolutionary (1 + 1) EA (CC (1 + 1) EA for short) that uses k (1 + 1) EAs. Since the population size of each (1 + 1) EA is one we do not need to make a decision about the choice of representatives: There is only one option. With respect to fitness assignment we consider a simple (and in some sense extreme) option: The computed fitness is only assigned to that individual that initiated the fitness evaluation. All other components are not affected. These design decisions lead to the following CC (1 + 1) EA that we formulate for the canonical partition. In order to stick with our usual assumptions about mutation we have to deviate from the usual choice of the mutation probability very slightly. Remember that the length of one component is l. Thus, we use mutation probability p m  = 1 ∕ l. Only in the extreme case l = 1 (when we have k = n and have one (1 + 1) EA for each single bit) we set p m  = 1 ∕ 2 because 1 ∕ l = 1 in this case and we want to have p m  ≤ 1 ∕ 2. Moreover, p m  = 1 would make the algorithm deterministic which clearly violates one major idea of evolutionary computation.

We see that we have two loops in the CC (1 + 1) EA: The a-loop iterates over the k components, and the usual evolutionary loop t increases after each component has been updated. We call one execution of lines 3–5 onegeneration. We call k consecutive generations (where we have a = 1, a = 2, …, a = k) a round.

The motivation for using cooperative coevolution is that a bit string consists of several pieces that we can identify. We believe that there may be an advantage in treating these pieces separately. Clearly, it reduces the size of the search spaces from to . This appears to promise a tremendous speedup if l is quite small (equivalently, if the number of pieces k is quite large) and the pieces can in fact be optimized separately. This is the case if the total fitness can be expressed as the sum of partial fitness values, where each partial fitness depends only on a single piece. We call such functions separable and give a precise definition of this crucial notation of separability.

Definition 6.7.

Let and be given. The function f is called (r,s)-separable if there exist such that the following five conditions all hold.

1.

2.

3.

4.

5.

with

The function f is called exactly (r,s)-separable if f is (r, s)-separable but not (r′, s′)-separable for any r′ > r or any s′ < s.

The definition may look more complicated than it actually is. The first two conditions simply state that the are a partition of . The third condition just defines notation: it defines the names of the individual elements as well as the sizes of the sets. The fourth condition states that each set has a size of at most s. Finally, the fifth condition is the most important one: It states that function values under f can be computed as the sum of function values for r individual functions, where each function depends only on a piece of x that is defined by one of the index sets .

Of the two parameters r and s, the parameter s is more important. If we know about the individual functions g i we can compute an optimum of f in time . We observe that n ∕ r ≤ s ≤ n − (r − 1) holds. It is obvious that every function is (1, n)-separable: We simply use and g 1 = f. Linear functions are (n, 1)-separable.

In the context of separability it is useful to note that each function has a unique representation as polynomial

where denotes the set of all subsets of (the power set of ) and for all I. We call

the degree of the polynomial. It is easy to see that a function f cannot be (r, s)-separable for any s < { deg}(f).

The separability of the fitness function f and the partition used by the cooperative coevolutionary algorithm may or may not match. We try to make more visible the fact that the separability of the function and the partitioning of the bit string for the algorithm are not necessarily related by our choice of terms. When we are referring to the parts defined by the separability of the function we call these parts pieces. When we are referring to the parts defined by the partitioning done for the algorithm we call these parts components. Since pieces and components may coincide, this distinction is not always clear. Since we hope to see advantages for the CC (1 + 1) EA in these cases we give these cases a special name.

Definition 6.8.

Let be exactly (r, s)-separable for some , and let be the index sets from Definition 6.7. We say that a partitioning for a cooperative coevolutionary algorithm matches the separability of the function if for each component its index set is contained completely in some I i . We say that it exactly matches the separability of the function if in addition the number of components equals r.

We have argued that we expect the approach of cooperative coevolution to be promising if the fitness function is separable (in a nontrivial way) and this separability is exploited in the partitioning of the bit strings. Therefore, it may come as a surprise that this is not necessarily the case. We demonstrate that by considering linear function which are (n, 1)-separable. It turns out that the CC (1 + 1) EA is unable to benefit from this separability regardless of the partitioning used.

Theorem 6.20.

For any linear function with , , , holds regardless of the partitioning of x by the algorithm.

Proof.

To simplify the notation we assume that n is even. Since all weights are different from 0 f has a unique global optimum. With probability at least 1 ∕ 2 we initially have at least n ∕ 2 bits that are different from the value the global optimum has for that bit.

First, we consider the case k < n. We consider the CC (1 + 1) EA after (l − 1)lnn rounds, i.e., after generations. The probability that at least one of the n ∕ 2 bits that all need to be flipped at least once has not yet flipped is bounded below by

This implies for this case.

For the case k = n we observe that in expectation the Hamming distance to the optimum is halved in each round. Since the single bits are treated independently in each piece, the probability that in one round the Hamming distance is reduced from d to d ∕ 3 is bounded above by . We consider i rounds and assume pessimistically that the Hamming distance is n ∕ 3 initially and is reduced by a factor of 1 ∕ 3 in each round. Thus, we have Hamming distance n ∕ 3 i + 1 after i such rounds. The probability of having an even smaller Hamming distance is bounded above by . Clearly, with i = (log_3 n) ∕ 2 this probability is less than 1 and we obtain also for this case. □

In order to understand why we see no advantage for linear functions in spite of their separability, we compare the (1 + 1) EA and the CC (1 + 1) EA with respect to the number of bits they flip. We consider a time interval of one round, i.e., k generations. The (1 + 1) EA uses mutation probability p m  = 1 ∕ n, and the CC (1 + 1) EA mutation probability min{1 ∕ l, 1 ∕ 2} in each component. Let us consider the case l ≥ 2 as it is more typical. First of all, we note that the mutation probability 1 ∕ l = k ∕ n is by a factor of k larger than the mutation probability p m  = 1 ∕ n of the (1 + 1) EA. Thus, we should expect to see more bits flipped by the CC (1 + 1) EA. We consider the expected number of mutations that flip exactly b bits in one component. Since in the CC (1 + 1) EA this component was active exactly once, in one round the expected number of mutations that flip exactly b bits equals . For the (1 + 1) EA we consider the same component. Note that the (1 + 1) EA does not actually have components but that we are still able to consider the same bits that make one component in the CC (1 + 1) EA. Clearly, each bit in this component had k generations where it could flip. Thus, the expected number of mutations that flip exactly b of these l bits equals . In order to compare these expected values we consider the quotient and obtain

and observe that k b − 1 is the decisive term. For b = 1 there is no significant difference between the two algorithms. Only for b > 1 do we expect to see many more such mutations for the CC (1 + 1) EA than for the (1 + 1) EA. Since linear functions can easily be optimized by mutations of single bits and joint mutations of many bits are not important, they are not a good candidate to showcase the strength of cooperative coevolution. We now understand the result on linear functions (Theorem 6.20) and find it no longer surprising.

In order to demonstrate the strengths of cooperative coevolution we design an example function where many mutations are needed where b > 1 are flipped simultaneously. While the concrete definition of the function (Definition 6.9) may look complicated, its structure is actually quite simple. We take { LEADINGONES} as an inspiration and define a function over l bits where the function value is given by the number of leading consecutive blocks of length b that only contain 1-bits. We concatenate k such functions and add their function values. Finally, we subtract { ONEMAX} to encourage the algorithm to have only ‘pure’ blocks that contain either only 0-bits or only 1-bits. This ensures that we actually need to have mutations of b bits flipping simultaneously. The function is called { CLOB} b, k , which is short for { CONCATENATEDLEADINGONESBLOCKS}.

Definition 6.9.

Let . Let and be given such that holds. We define l = n ∕ k. The function is defined by

It is easy to see that { CLOB} b, k is exactly (k, l)-separable: The i{ th} piece is . For each value of h, the inner sum operates on the h{ th} piece and no other bits. The factor n can easily be moved within the outer sum. Since { ONEMAX} is (n, 1)-separable, it can also be split up and added to the summation of the inner sum. Modified in this way the inner sums are the functions g from Definition 6.7. Therefore, { CLOB} b, k is (k, l)-separable. For i = l ∕ b the product goes over l bits. This term is part of the polynomial representing { CLOB} b, k . Therefore, { CLOB} b, k is exactly (k, l)-separable. In order to see clear advantages for the CC (1 + 1) EA we exploit this separability fully.

Theorem 6.21.

Let be given such that . Consider the CC (1+1) EA on { CLOB} b,k such that the components exactly match the separability of { CLOB} b,k

Proof.

We use the notation from Definition 6.6. Note that pieces and components coincide since the partition used exactly matches the function’s separability.

We first prove an upper bound. Let us consider only x (1). We know that the number of leading ones blocks cannot decrease. It increases if there is a mutation of at most b specific bits. The probability for a such a mutation is bounded below by . If we have seen at most l ∕ b such mutations we know that holds. We consider rounds, i.e., generations. In these rounds the component x (1) is active 10el b ((l ∕ b) + lnk) times. Since the probability for one of the mutations we are waiting for is bounded below by 1 ∕ (el b ) we expect to see 10((l ∕ b) + lnk) of them. Using Chernoff bounds, we bound the probability to see less than (l ∕ b) + lnk of them by

from above. For k = 1 we are done. For k > 1, using the union bound we see that the probability to have at least one of the k components where we have not reached the all ones string after 10el b ((l ∕ b) + lnk) rounds is bounded above by

In this case we can consider the following 10el b ((l ∕ b) + lnk) as a second attempt. The expected number of attempts needed is bounded above by . This implies .

For the lower bound we prove two things. First, we prove that the probability to have components equal to the all zeros bit string 0 l at some point of time is bounded below by . Given this, we prove that with probability after a sufficiently large number of steps at least one of these components is still different from the all ones bit string 1 l .

We distinguish two cases with respect to the length of one component. For the probability that a component is initially already 0 l equals . The expected number of such components after random initialization equals , and we have at least k ∕ 2 l − 1 with probability . For we observe that for each component the two leftmost bits are 0-bits with probability 1 ∕ 4. The number of 0-bits can only decrease if the all ones bit string is created in this mutation. The probability for such a mutation is bounded above by 1 ∕ l 2. On the other hand, reaching the all zeros bit string is equivalent to optimizing { ONEMAX}. We see that with probability the all zeros bit string is reached in these components before the all ones bit string. Thus, with probability we have such components.

Now we can work under the assumption to have at least ck components equal to 0 l , where c > 0 is a constant. We distinguish two cases with respect to the number of components. First, we consider the case lnk ≥ l ∕ b. In this case it suffices to prove . For each of the ck components the probability for a mutation that leads to the all ones bit string is bounded above by 1 ∕ l b . We consider rounds and one such component. The probability that the component is still equal to 0 l after these rounds is bounded below by

In the CC (1 + 1) EA on { CLOB} b, k the components evolve independently. Thus, the probability to have at least one component equal to 0 l after rounds is bounded below by . This implies . The other case is lnk < l ∕ b. In this case it suffices to prove . We employ drift analysis and use Theorem 5.15. We use the number of blocks that is equal to 0 b as distance measure and we already know that the initial distance is . In one round the expected decrease in distance is bounded above by k ∕ l b since the probability to change such a block is bounded above by 1 ∕ l b and in each of the k components we have at most one such block where this can be done. Since one round consists of k generations, we have . □

The expected optimization time is . We see that it grows exponentially in b and that this growth is less hurtful if l is small. Clearly, b is a parameter of the problem { CLOB} b, k that we cannot influence. But we have some influence on l. It was a design decision to make k as large as possible (and thus l as small as possible). We see that good design decisions can actually help a cooperative coevolutionary algorithm to be efficient.

Of course, we do not believe that the (1 + 1) EA is nearly as efficient as the CC (1 + 1) EA is. Since beliefs may be wrong and ideas may be misleading we make sure that we are right in this point and perform an analysis for the (1 + 1) EA on { CLOB} b, k , too.

Theorem 6.22.

Let be given such that

Proof.

We begin with the proof of the lower bound that consists of two major steps. In the first step we prove that we need specific mutations where b bits flips simultaneously with probability . In the second step we prove a lower bound on the expected waiting time for this number of such mutations.

We consider some x t and its offspring y. Since we are dealing with { CLOB} b, k we still have this structure of k pieces of length l and l ∕ b blocks of length b in each piece. We keep using the notation for this.

We do not care about the steps where holds since either x t + 1 is unchanged or at least the number of 0-bits is not decreased in comparison to x t . We also neglect the case where the number of leading blocks of 1-bits is not increased in any piece in y in comparison to x t since then, again, the number of 0-bits cannot decrease. This leaves us with the case where the number of blocks of leading 1-bits has increased.

We have already argued that we need a specific mutation of at least 2 bits for this if initially the number of 0-bits was bounded below by 2 in the block we consider. Note that in each piece we consider only the leftmost block that is different from 1 l . This block contains at least two 0-bits with probability at least 1 ∕ 4. The expected time before such a block becomes 1 l is bounded below by . The expected time before this block becomes 0 l is bounded above by . Thus, with probability we have a situation where in each piece we have some number of leading all ones blocks and all other blocks are equal to 0 l before we increase the number of all ones blocks. Since the expected number of blocks initially equal to 1 l is , we have blocks equal to 0 l after steps with probability .

Let Z t denote the number of blocks equal to 0 l at time step t. Since in each piece only the leftmost of these blocks can be mutated into 1 l with a mutation of b bits, the number of blocks for such mutations is bounded above by min{Z t , k}. Therefore, the expected waiting time for all these blocks to become 1 l is bounded below by

for some constant c > 0. Together this proves .

For the upper bound we first need a bit of notation. Remember that x t is divided into k pieces, , each piece of length l, and that each piece contains l ∕ b blocks of length b each. For x ∈ { 0, 1} n and we denote the set of starting positions of these blocks in the piece x (i) by B i ,

For j ∈ B i , we consider the block in x (i) starting at x[j] and denote the set of positions of 0-bits in this block by Z i, j (x),

Given this, we concentrate on the leftmost bit in the leftmost block that is different from 1 l and denote its position by z i (x),

Note that the maxB i component ensures that z i (x) is well defined even if the piece is already equal to 1 l . Using this notation we denote the event that we are interested in by A(x t ), the event that there is some such that all bits x t [h] with mutate and no other bit mutates. The probability for this event is bounded below by . We observe that after l ∕ b such events in each of the k pieces the optimum is reached. We know that the expected waiting time for one such event is . The situation is similar to coupon collecting (see Lemma 5.3), where we have k different coupons and would like to have not just one but at least l ∕ b of each. We claim that the expected waiting time in this generalized coupon collecting scenario is (and prove this in Lemma 6.2, see below). Together, this proves . □

Lemma 6.2.

Let n be the number of coupons in the coupon collector’s problem, k the number of times one would like to collect each coupon, T the number of coupons obtained when the collection becomes complete.

Proof.

For n = 1 the process is deterministic and the statement is obviously true. Thus, let n > 1. For the proof of the lower bound we first observe that we need to collect k copies of each coupon, a total of coupons. This implies . On the other hand, we know that since we expect to have to wait this long even for k = 1. Together this implies .

For the proof of the upper bound we consider the situation after obtaining the first 4nk + 4nlnn coupons. Let K 1 denote the number of times we obtained the first coupon. Clearly, we have . By means of Chernoff bounds we obtain

The union bound yields that the probability that there is some coupon that we have not yet collected k times is bounded above by . In this case we reconsider the situation after we obtained another 4nk + 4nlnn coupons. The expected number of times we need to repeat this is (since we have n ≥ 2) and we obtain . □

Now we know that the cooperative coevolutionary approach can have immense advantages in comparison with an evolutionary algorithm. We have noticed, however, that separability of the fitness function alone is not sufficient. But exploiting separability together with the increased mutation probability leads to this advantage. Note that the advantage is not in the increased mutation probability alone. We cannot obtain a result like Theorem 6.21 for the (1 + 1) EA by simply increasing the mutation probability. In the case the mutation probability of the CC (1 + 1) EA is . With such a huge mutation probability the (1 + 1) EA is not going to get anywhere. The large mutation probability the CC (1 + 1) EA employs is only sustainable because it is employed in isolation in a single piece; the other pieces are protected from mutation. This constitutes a specific advantage for cooperative coevolution. Before we investigate if this advantage can also turn into a disadvantage, we consider one other question first. Can coevolution be harmful even if the function is separable and the separability is exploited? We need to be careful here. An example would not be very interesting or convincing if the disadvantage for the CC (1 + 1) EA was only due to the increased mutation probability it employs. So, we ask a stronger question: Are we able to find an example function that is separable (in a nontrivial sense) where we employ the CC (1 + 1) EA exactly matching the function’s separability and use an optimal mutation probability in each component (instead of the standard choice min{1 ∕ l, 1 ∕ 2}) and have the CC (1 + 1) EA still outperformed by the (1 + 1) EA?

Before we actually consider one such example, let us try to find out how bad it can actually get. We are interested in some (r, s)-separable function f (with s < n) such that the (1 + 1) EA is efficient on f. Clearly, the (1 + 1) EA is also efficient on every sub-function g of f (see Definition 6.7 for notation). Now, for the CC (1 + 1) EA we use for each component the same mutation probability that the (1 + 1) EA employs. Let t(n) denote the expected optimization time of the (1 + 1) EA on f. We consider the situation after rounds of the CC (1 + 1) EA. For each component, Markov’s inequality yields that the probability not to be done is bounded above by 1 ∕ n 2. Thus, the probability not to be done in any one component is bounded above by . Thus, with probability the CC (1 + 1) EA optimizes f with generations and is (in this sense) also efficient. We see that we cannot expect the CC (1 + 1) EA to be very much slower than the (1 + 1) EA.

Definition 6.10.

Let . We define and functions and .

It is not difficult to see that g l is exactly (1, l)-separable and, therefore, f k, l is exactly (k, l)-separable. Since f k, l is defined as sum of k copies of g l it suffices to discuss properties of g l . The all ones bit string 1 l is the unique global optimum and there are two good ways to reach it. Randomized search heuristics are guided toward 0 l , and it is very unlikely that they will not have a Hamming distance of to 0 l before they start to increase the number of 1-bits. From that point on, search points are either of the form or . In order to improve a search point of the form it suffices to reach a Hamming neighbor. In order to improve a search point of the form a mutation of at least three specific is needed (unless i = 4 or ). Thus, reaching the optimum via can be expected to be much more efficient. However, reaching search points of the form is clearly much less likely. Thus, we can expect that it does not contribute that much to the expected optimization time. This holds for g l . If we consider f k, l we have k copies of g l . This makes it much more likely to observe the less likely event that the path via is entered. We will see that for the (1 + 1) EA, however, this is less of a problem than for the CC (1 + 1) EA.

Theorem 6.23.

Let . We define . Let

Proof.

First we consider the (1 + 1) EA on g n , i.e., the case k = 1. We claim that holds in this case.

Let . Let F denote the event that there is some with x t  ∈ P. We know that holds (this follows from the result on { RIDGE}, see Theorem 5.14). Replacing the mutations of single bits sufficient for { RIDGE} by mutations of 3 specific bits we obtain . By the law of total probability we have

and see that it suffices to prove .

As long as we have and all bit strings with the same number of 1-bits have equal probability. This implies for the first time when holds that this search point x t belongs to P with probability . If we have the algorithm may still make a mutation leading it to P. For i = 0 we have and for i > 0. Thus, for i = 0 the probability to mutate into P is bounded above by . For i > 0 we have probability . Thus, follows.

For f k, l we have for each piece that the expected time needed to reach its optimum is if this happens without entering P, and otherwise. Moreover, for each component the probability to enter P is . This implies

□

Theorem 6.24.

Let . We define k := l 4, . Consider the CC (1+1) EA on f k,l matching exactly the separability of f k,l . In each component the optimal mutation probability p m is used.

Proof.

Note that k = l 4 and implies so that this parameterization is actually covered by Theorem 6.23. We make a case distinction with respect to the mutation probability p m . Since the k components are identical we can assume that for each component the same mutation probability is used.

First consider the case p m  = 1 ∕ l. We know that the probability to enter P is . We have k = l 4 so that the probability to have this in at least one component is . If this happens the expected number of rounds needed to optimize this component is so that follows.

With p m  > 1 ∕ l the probability to enter P increases. The optimal mutation probability once in P equals 3 ∕ l. Thus, asymptotically we cannot improve over the case p m  = 1 ∕ l here.

Now consider the case p m  < 1 ∕ l with . Now the probability to enter P is decreased to . However, the probability that this happens in at least one of the k components is still . Since the smaller mutation probability slows the algorithm down the expected optimization time is actually larger than for p m  = 1 ∕ l.

Finally, we consider the case . For this case we concentrate on the time needed to reach the optimum via . The expected number of rounds for one component is bounded below by . This implies . □

We have not yet dealt with the case that the fitness function is either not separable or that the partitioning of the bit strings does not match its separability. Remembering the motivation for introducing cooperative coevolution, we realize that this case implies that a design error has been made due to an error of judgment with respect to properties of the fitness function. We would like to know what consequences such an error may have.

We consider { JUMP} n (see Definition 5.5) as an extreme example. We know that holds. Thus, the (1 + 1) EA is completely inefficient on { JUMP} n and we do not expect the CC (1 + 1) EA to change that. However, as the next theorem shows, things can be even worse.

Theorem 6.25.

For any partition in k ≥ 2 components the CC (1+1) EA never finds the optimum of { JUMP} n with probability

Proof.

We call a part x (i) (with ) optimal if holds. As long as there are at least two components that are both not optimal the number of 1-bits cannot increase in any component. Thus, the optimum cannot be reached.

We need to prove that the probability to have at most one nonoptimal component initially is . For we have that each component individually is optimal after random initialization with probability . The probability to have two that are both nonoptimal is therefore and the claim follows. For we have that a component is optimal after random initialization with probability 2 − l . Thus, the expected number of optimal components is bounded above by k ∕ 2 l . We apply Chernoff bounds and obtain as upper bound for the probability to have at least optimal components

For k ≥ 4 we have at least two nonoptimal components with probability which is sufficient since we are dealing with the case . □

It is important to observe that this problem is fundamental and is not restricted to the artificial and for practical purposes irrelevant { JUMP} n . We can consider concatenated copies of { JUMP} n and see that as long as we have at least two components for each copy we will end up with a solution that is bad with respect to function value (different from { JUMP} n where we still find a second best search point). The same holds if the number of components is equal to the number of copies but the components do not coincide with the pieces. It also holds when we use larger population sizes. Also the use of crossover makes no difference.

We conclude that cooperative coevolutionary algorithms are (at least in general) not global optimizers. They may fail to find an optimal solution even when given infinite time. So far, we have not talked much about finding an optimum with probability converging to 1 with time. The reason is practical: It does not really matter if it takes steps to find an optimum or if we do not find an optimum at all. Therefore, we take note of this deficiency of cooperative coevolution but are not too concerned by it.

Let us consider another and final example for an inseparable problem, i.e., a fitness function that is (1, n)-separable. It is easy to see that { LEADINGONES} is (1, n)-separable. We know that holds (Theorem 5.16). Since { LEADINGONES} is not misleading (very different from { JUMP} n ), it is not surprising that the CC (1 + 1) EA is eventually able to optimize it. However, it may be surprising that the partitioning of the bit string into pieces has no influence at all.

Theorem 6.26.

Consider the CC (1+1) EA with an arbitrary partitioning.

Proof.

We know that in expectation rounds are sufficient to optimize the leftmost component that is still different from 1 l . Thus, the expected number of function evaluations is bounded above by . Since there are k components we obtain .

For the lower bound we distinguish three different types of components. The components x (i) with we call optimized. The component x (m + 1) is called active. We observe that all remaining components are distributed uniformly at random since they have been so initially and never had an influence on selection (compare the proof of Theorem 5.16). This implies that for each active component we require rounds (equal to generations) to optimize it. Since the initial number of optimized components is we obtain . □

We observe that separability can be completely irrelevant. On the other hand, we have seen that not matching the separability of the fitness function can hinder a cooperative coevolutionary algorithm to find an optimal solution at all. Thus, separability has some role to play. However, it is far from being a decisive factor and more properties of the fitness function need to be taken into account.

6.4 Combinatorial Optimization Problems

So far we have only considered the case that we want to apply some evolutionary algorithm to a fitness function , where S is a search space where the evolutionary algorithm works. We have not really considered the question where this fitness function came from. We have touched the subject in Sect. 2.5 but have been more concerned with the design or adaptation of evolutionary algorithms here. A very different and quite common case is one that we find for example in combinatorial optimization. In combinatorial optimization one is confronted with an instance of a general problem. Examples that we already mentioned include satisfiability problems (SAT) where one is looking for an assignment of variables that satisfies as many clauses as possible (the concrete instance being defined by the set of clauses) and the traveling salesperson problem (TSP) where one is looking for a tour visiting n cities that minimizes the total travel distance (the concrete instance being defined by the distances between each pair of cities). We will be considering two more examples in this section.

When confronted with the instance of a combinatorial optimization problem one way of tackling the problem instance by means of an evolutionary algorithm is to convert the problem instance into a fitness function over some search space. The choice of the search space, the mapping from this search space to problem solutions (the so-calledgenotype–phenotype mapping), and the design of the fitness function are all design choices that together define the encoding. It is possible to encode each problem instance individually. However, it is much more common to design a common encoding for a combinatorial optimization problem in an algorithmic way so that a problem instance can be turned into a search space and fitness function automatically. Clearly, the choice of the encoding makes the difference between an easy or hard to solve problem for the evolutionary algorithm. We will not discuss this problem in a principled way but consider one example in some detail. This will highlight some of the choices one is faced with and some of the consequences these choices have.

The combinatorial optimization problem we consider is the computation of a minimum spanning tree. A problem instance is an undirected, weighted, connected graph G = (V, E, w) where V is a set of vertices and a set of edges. Being connected means that for each two nodes u, v ∈ V there is a path of edges to get from u to v. The graph is weighted, so there is a weight function that associates a positive weight with each edge. A solution is also an undirected, weighted, connected graph G′ = (V, E′, w) with , i.e., a subgraph of G that includes all nodes. Among all these graphs we are looking for one that minimizes . The problem is called minimum spanning tree (MST) and is a well-known combinatorial optimization problem. It is easy in the sense that efficient algorithms for solving it (computing exact solutions deterministically in polynomial time) are known. Thus, it is not a good candidate for applying evolutionary algorithms. However, it is interesting to see how randomized search heuristics perform on simple problems: If someone does not recognize the problem as MST (or is unaware of good problem-specific algorithms) and decides to apply a randomized search heuristic, will the heuristic be efficient?

The problem definition does not explicitly include that the solution needs to be a spanning tree. However, this is an immediate consequence from the problem definition. The solution needs to be a spanning graph since it needs to be connected. If it is not a tree we can remove at least one edge and the remaining graph is still connected. Since all edges have positive weights, the resulting graph has smaller weight so that the original graph could not have had minimum weight. Thus, only trees can be optimal solutions.

Since the set of nodes V in any solution G′ is fixed, we can define a candidate solution by defining the set of edges E′. We say that every is a candidate solution and distinguish candidate solutions that are spanning graphs (not necessarily spanning trees) from those that are not. Candidate solutions that are spanning graphs are called feasible, the others infeasible.

Since the problem is an edge selection problem we use the following search space. Let (so we use ). The search space S is defined as S = { 0, 1} m and x ∈ { 0, 1} m defines an edge selection . Note that for some evolutionary algorithms the ordering of the bits is important. For such cases it can be useful to spend some time considering different mappings from edges to positions in the bit string. However, we will only consider simple mutation-based search heuristics here so that this is not important for us. In order to simplify notation we will usually consider x ∈ { 0, 1} m and talk about it as if it was E′(x). Since we can represent any edge selection this way the algorithms will be able to find infeasible solutions. We need to remember this when defining the fitness function.

The MST is a minimization problem. We are looking for an edge selection that is spanning and has minimum weight. Fitness functions, on the other hand, are usually maximized. However, we will deviate from this and define a fitness function that is to be minimized. When we apply an algorithm that aims at maximizing we silently replace f by − f.

One first and very naive attempt at defining a fitness function could be

that simply assigns the weight of the edge selection as function value. Since this ignores that the edge selection needs to be a spanning graph we immediately get x = 0 m as global optimum with and f 1(x) = 0, clearly not what we had in mind. We observe that we must not forget any of the requirements for a solution.

Learning from our attempt, we define

as an improved alternative. We observe that optimal solutions for the MST correspond to global minima of f 2. In this sense f 2 is a good encoding of the problem. However, it is an encoding that makes the problem unnecessarily hard. Consider the chain graph G = (V, E, w) with , and w(e) = 1 for all e ∈ E. A graphical representation of the graph can be found in Fig. 6.6.

Fig. 6.6

Graphical representation of the chain graph G for n = 8

Clearly, the only (and thus optimal) spanning tree is defined by E′(x) = E and represented by x = 1 m . We have f 2(1 m ) = m and f 2(x) = ∞ for all . Thus, for the chain graph the fitness function is essentially equal to { NEEDLE} and is very hard for randomized search heuristics. Since the problem instance is very simple we see that f 2 is not a good choice and that in general a badly defined fitness function can turn an easy problem into a hard one.

Motivated by the failure of f 2, we are willing to define a slightly more complex fitness function. Let w b  = w(E) and c(x) denote the number of connected components in (V, E′(x)). Using this we define

as our third attempt at finding a good fitness function for MST.

The function value f 3(x) is the sum of three components. The first component, , is a penalty term that increases with the number of connected components. Feasible solutions consist of exactly one connected component, for these this penalty term becomes 0. Due to the weight w b 2 that is added for each additional connected component this term is the most important. The second component,, is a penalty term that increases with the number of edges. It becomes 0 if the number of edges equals n − 1. This is the number of edges in a tree on n nodes. Since the weight for this term is w b , it is outweighed by the first penalty term if the number of edges is smaller than n − 1. Clearly, this is the second most important term. The fitness function f 3 first pays attention to the number of connected components. If this is 1 it pays attention to the number of edges. If this number indicates that a tree has been found the final component, w(E′(x)), makes sure that we favor spanning trees with small weights. We see that the set of global minima of f 3 coincides with the set of minimum spanning trees, the optimal solution of the MST instance. We observe that we encode additional problem knowledge into the fitness function. The number of edges in an optimal solution is not part of the problem description. While it is, of course, a good idea to code knowledge about the problem into the fitness function, one may question if this degree of knowledge is present in common applications. Thus, one may wonder if f 3 is still an appropriate example fitness function in our context. Taking this critique into account we define the next fitness function as slight variant of f 3:

While f 4 maintains the penalty term for the number of connected components, it has no penalty term for a too large number of edges. We observe that this term is not actually needed; the set of global minima of f 3 and f 4 are identical.

While f 4 appears to be the most natural fitness function we have defined, one may wonder if it really is ‘natural’ to combine the number of connected components and the weight of the edge selection into one single number. This leads to the definition of with

The two different criteria are explicitly enumerated as two different objectives. Now we need a search heuristic that searches solutions that simultaneously minimize both components. Many such heuristics are known, in particular evolutionary algorithms are generally thought of as being especially good for solving such multiobjective optimization functions. However, since we have not considered evolutionary algorithms for multiobjective optimization we do not investigate this further.

We discard our first two attempts at defining fitness functions and start with considering f 3. We consider the (1 + 1) EA and random local search (RLS). For RLS, we do not consider the direct Hamming neighborhood N 1 but a larger neighborhood N  ≤ 2 that includes all points with Hamming distance at most 2, . We select a neighbor from N  ≤ 2 not uniformly at random but with equal probability 1 ∕ 2 either uniformly at random from N 1(x) or uniformly at random from . Note that this favors direct Hamming neighbors since and . We observe that N  ≤ 2 is more appropriate than the direct Hamming neighborhood N 1 for MST. In order to see this, assume that the current search point is a spanning tree that has no minimum weight. For N 1 this is a local optimum since we can neither remove any edge (since this would increase the number of connected components from 1 to 2) nor add any edge (since this would increase the total edge weight). With N  ≤ 2 we can at the same time remove one edge and include another one. In addition to RLS, we also consider the (1 + 1) EA under the same perspective.

Theorem 6.27.

Using RLS with neighborhood N ≤2 and selecting a neighbor with Hamming distance 1 or 2 with equal probability, the expected time until some spanning tree is found using f 3 is

Proof.

We define an f 3-based partition S i by

The sets S i with contain exactly those search points that encode solutions with n + 1 − i connected components. Since f 3 decreases with a decreasing number of connected components, these S i are ordered correctly for an f 3-based partition. The sets S i with i ≥ n all contain search points that encode spanning graphs, i.e., subgraphs that form a single connected component. As the number of edges decreases, the value of i increases so that these S i are also ordered correctly. We observe that S m + 1 contains exactly all spanning trees. We observe that for all x ∈ { 0, 1} m  ∖ S m + 1 there is always a direct Hamming neighbor in a higher fitness level. For i < n we have n + 1 − i connected components. For each of these components there exists at least one edge that connects it to some other connected component since the graph (V, E) is connected. Thus, we have as lower bound on the probability to leave S i for . For i ≥ n we have m + n − i edges and can select any one of m + 1 − i for removal without losing connectedness. This implies as lower bound to leave S i for . This implies

as upper bound on the expected waiting time to reach a spanning tree. □

Theorem 6.28.

Using the (1+1) EA, the expected time until some spanning tree is found using f 3 is

Proof.

The (1 + 1) EA (with mutation probability p m  = 1 ∕ m as m is the length of the bit strings x ∈ { 0, 1} m ) can make the same mutations of single bits with asymptotically the same probability. Therefore, the proof of Theorem 6.27 carries over. □

While it is nice to see that we can find some spanning tree using RLS or the (1 + 1) EA we are, of course, interested in finding minimum spanning trees. In order to prove that we can also achieve this we need to have some insight into the structure of the problem. Note that this insight is needed to prove the algorithms to be efficient, not for the algorithms to be efficient or for us to be able to define an appropriate encoding. In the following we identify an edge set that defines a subgraph (V, E′) with this subgraph.

Lemma 6.3.

Let G = (V,E,w) be an instance of the MST, a minimum spanning tree, a spanning tree with w(T) > w(T ∗ ).

There is a , such that there are k different spanning trees with such that on average holds.

Proof.

We define . Since we have w(T) > w(T  ∗ ) and , we know that 1 ≤ k ≤ n − 1 holds. We can exchange an edge from against some edge in T  ∗  such that the result is still connected and the resulting weight is bounded above by w(T). More formally, there exists a bijection such that for all we have a circle in containing α(e) with w(α(e)) ≥ w(e). We obtain the new spanning tree as that has Hamming distance 2 to T and weight at most w(T). Performing all k of these edge exchanges leads to T  ∗  with weight w(T  ∗ ). Thus, the average decrease in weight caused by one such edge exchange equals . □

In order to avoid difficulties due to the unknown number of edge exchanges k we can formulate a similar lemma using a simple trick.

Lemma 6.4.

Let G = (V,E,w) be an instance of the MST, a minimum spanning tree, a spanning tree with

There are n different spanning trees with such that on average holds, where s(T′) is the result of selection between T and T′ in RLS or the (1+1) EA.

Proof.

We know from the proof of Lemma 6.3 that there are k ≥ 1 such edge exchanges. We define n − k more by identifying search points with Hamming distance 2 that have worse function values. These are not accepted in selection. Due to the increased number of operations (n instead of k) the average decreases as claimed. □

Using the structural insight from Lemma 6.4 it is not difficult to prove an upper bound on the time needed to find a minimum spanning tree. We do this first for local search.

Theorem 6.29.

Consider an MST instance G = (V,E,w) with and define . Using RLS with neighborhood N ≤2 and selecting a neighbor with Hamming distance 1 or 2 with equal probability, holds.

Proof.

We know that on average after steps some spanning tree is found. Thus we consider RLS from this point on. We use drift analysis in the form of Lemma 5.6, i.e., multiplicative drift. Let T  ∗  denote an optimal solution. We use d(T) = w(T) − w(T  ∗ ) as distance function. Lemma 6.4 yields that for each current spanning tree there are n different spanning trees with a Hamming distance of 2 such that moving to either one of them decreases d(T) on average by a factor of 1 ∕ n. Since the probability to move to such a neighbor is , the expected decrease in distance is . Since each spanning tree contains n − 1 edges and all edge weights are bounded below by 1 and bounded above by w max we have that the initial distance is bounded above by . Application of Lemma 5.6 yields . □

Again, the result can be easily transferred to the (1 + 1) EA.

Theorem 6.30.

Consider an MST instance G = (V,E,w) with and define

Proof.

The only relevant difference of the (1 + 1) EA to RLS is the generation of the next search point. For the proof of Theorem 6.29 only search points with Hamming distance 2 are considered. The (1 + 1) EA mutates to one of these with asymptotically equal probability in comparison to RLS, . Thus, the result carries over without any change. □

If we consider graphs where the edge weights are polynomially bounded we have that . This yields an expected optimization time of which is ‘efficient’ in the sense of ‘polynomial’, but is clearly slower than problem-specific algorithms that achieve running times of (Kruskal’s algorithm) or (Prim’s algorithm). However, RLS and the (1 + 1) EA are very much simpler to implement than both these problem-specific algorithms and know nothing about the problem at all. Thus, losing a factor of is not too bad. Moreover, the simple proof of Theorem 6.29 may lead to speculations that our upper bound is overly pessimistic. This, however, turns out not to be the case. In order to prove this we consider the following problem instance. A graphical representation can be seen in Fig. 6.7.

Definition 6.11.

Let with . We define ,

and G n : = (V, E, w).

We see that G n consists of two parts with an equal number of nodes. The part containing the nodes v i with i ≤ (n ∕ 2) + 1 consists of triangles that each contain two edges with weight 2n 2 and one edge with weight 3n 2. The part containing the nodes v i with i ≥ (n ∕ 2) + 1 forms a clique; all nodes have weight 1. A minimum spanning tree consists of all the edges with weight 2n 2 and an arbitrary subset of nodes from the clique that are a spanning tree for the clique. The motivation for G n is the following. The part with the triangles is relatively difficult to solve. There is only one unique optimal solution for this part and a wrong choice for each triple of nodes. The clique is easy to solve but increases the number of edges so that the graph is dense. This decreases the probability to reach a specific edge selection, which hurts in the part with the triangles which is sparse.

Theorem 6.31.

Using RLS with neighborhood N ≤2 and selecting a neighbor with Hamming distance 1 or 2 with equal probability, holds.

Proof.

The upper bound follows from Theorem 6.29 for RLS and Theorem 6.30 for the (1 + 1) EA since G n contains n nodes and edges. For the proof of the lower bound we fix the following notation. If in a triangle all three edges are selected we call the triangle complete. If the two light edges (with weight 2n 2) are selected we call the triangle good; if one light edge and one heavy edge (with weight 3n 2) are selected we call the triangle bad. For an edge selection x ∈ { 0, 1} m let g(x) denote the number of good triangles and c(x) denote the number of complete triangles. Moreover, we denote the number of connected components in the triangle part as c T (x) and the number of connected components in the clique part as c C (x). Finally, the total number of connected components is .

Fig. 6.7

Graphical representation of the graph G n for n = 12. All edges shown without a weight (edges with i, j ≥ 7) have edge weight 1

We consider the situation after random initialization and are interested in a situation where we have (a linear number of bad triangles) and (the clique connected). For each of the n ∕ 4 triangles the probability to be bad initially equals 1 ∕ 4. Thus, the expected number of bad triangles is n ∕ 16, and Chernoff bounds yield that with probability we have initially at least n ∕ 32 and at most n ∕ 8 bad triangles. For each node in the clique we observe that its expected degree after initialization is ((n ∕ 2) − 1) ∕ 2 so that with probability it has at least degree n ∕ 8. Let N be the random number of neighbors. For each of the other nodes in the clique it holds that the expected number of neighbors it shares with the first node we consider equals N ∕ 2. Since N ≥ n ∕ 8 with probability we conclude that the number of common neighbors is at least n ∕ 32 with probability . Thus, with probability the clique is connected initially. We now assume that this is the case.

We consider one step of the algorithm (RLS or (1 + 1) EA) and concentrate on the edges in the triangle part that differ in x t and y. Let k be the Hamming distance between x t and y restricted to the bits that represent edges in the triangle part (ignoring the edges in the clique part). We call a generation with such x t , y, and k a k-step. For k ∈ { 1, 2} the probability for such a step is

for both, RLS and the (1 + 1) EA. For k ≥ 3 the probability for such a step is 0 for RLS since we are restricted to the N  ≤ 2 neighborhood. For the (1 + 1) EA we still have probability for such a step. Since the lower bound we consider is , we have that with probability no such step with k ≥ 5 occurs in this time. Therefore, we can ignore such steps in the following.

We consider phases of n 5 ∕ 2 consecutive steps. In such a phase the number of 1-steps is with probability , and the number of 2-steps is with probability . We need to keep in mind that in a k-step the number of flipping bits can be larger than k: There may be additional changes in the clique.

Now consider the first phase of length n 5 ∕ 2, starting after initialization. Remember that we have C C (x) = 1 and at the beginning of this phase. Moreover, we can assume that the number of 2-steps is . These steps can decrease the number of bad triangles by at most . Moreover, they can increase C C (x) by at most . This holds since C C (x) can only increase in steps where C T (x) decreases by at least the same amount. Since C T (x) is decreased by at most in 2-steps the upper bound on the increase in C C (x) follows.

For RLS, k-steps with k ≥ 3 cannot occur. For the (1 + 1) EA, the expected number of such steps is . The probability to have of them is . Thus, they can also neither increase C C (x) by nor decrease the number of bad triangles by .

Remember that we can assume to have 1-steps. In such a step c C (x) cannot increase since c T can decrease by at most 1 and the edges in the triangle part are heavier than all clique edges. A 1-step may decrease the number of bad triangles by 1. This can happen by either making a bad triangle complete or by removing an edge from a bad triangle. But cases decreases the fitness so much that such a step can only be accepted if at the same time c C (x) is decreased. Since c C (x) is initially 1 and increased by at most this again cannot happen times.

However, the 1-steps have a good chance to make the graph connected. Similar to the proof of Theorem 6.27, we see that 1-steps are sufficient. Thus with probability we have c G (x) = 1 and at the end of the first phase.

Now we consider the second phase. As c G (x) = 1 holds at the beginning of the second phase, the graph will stay connected. We see that 1-steps cannot decrease b(x). We want to show that we have and c(x) = 0 at the end of the second phase, i.e., the number of bad triangles is still linear but we now have no complete triangles any more. In fact, we will prove that at the end of the second phase with probability close to 1 we have a spanning tree.

We can argue as above that b(x) can only decrease by in one phase. This implies at the end of the second phase. The probability to decrease c(x) by 1 is bounded below by for both algorithms. Since we always have c(x) ≤ n ∕ 4 (since the number of triangles is n ∕ 4) we see that the expected number of generations until we have c(x) = 0 is . Thus, this happens in the second phase with probability . We can show the same for the clique so that we have a spanning tree at the end of the second phase but still with probability .

In a spanning tree 1-steps are no longer accepted at all. The same holds for 3-steps. As mentioned above, we can ignore k steps with k > 4. A 2-step can only be accepted if it turns a bad triangle in a good one. The probability for such a step is for both algorithms. Thus, the expected waiting time for such a step is and we need such steps. This implies as expected waiting time. Since 4-steps are much less likely they cannot change this. □

The fitness function f 4 is in some sense more natural than f 3. It does not contain an additional penalty term for an ‘incorrect’ number of edges. We expect the number of edges to be reduced to n − 1 due to the additional weight of unneeded edges and find this confirmed in the following result.

Theorem 6.32.

Consider RLS with neighborhood N ≤2 , where RLS selects a neighbor with Hamming distance 1 or 2 with equal probability and the (1+1) EA, both either on an arbitrary graph G = (V,E,w) with and or on G n from Definition 6.11.

Proof.

The proof for all results changes little in comparison to Theorems 6.29–6.31. The only major difference is that we no longer wait for a spanning tree as a preparatory step. Instead, we wait for the selection to be a spanning graph, i.e., an arbitrary connected graph. This introduces additional Hamming neighbors that are an improvement and can be reached by removing a single edge. We observe that this neither hurts our upper bounds from Theorems 6.29 and 6.30 nor the lower bound from Theorem 6.31. □

We consider two more randomized search heuristics on this problem, namely the Metropolis algorithm and simulated annealing. They allow us to revisit a topic that we have considered before, the comparison of dynamic with static parameterization. We have seen for the (1 + 1) EA that the dynamic (1 + 1) EA can be efficient on problems (Theorem 6.8) where the (1 + 1) EA is very inefficient even with optimally set static mutation probability (Theorem 6.9). We will see now a similar result for simulated annealing and the Metropolis algorithm: Simulated annealing can be efficient where the Metropolis algorithm is very inefficient even with optimally chosen static temperature. Since we have not stated a complete formal definition of simulated annealing earlier we do this now here. Note that is an annealing schedule, and is a neighborhood.

Remember that the Metropolis algorithm is identical to simulated annealing with the only difference that is a constant. Both algorithms as defined as Algorithm 12 aim at minimization. Therefore, we can use the fitness functions we have defined for MST without the need to change from f i to − f i .

Clearly, simulated annealing and the Metropolis algorithm are both able to accept steps that worsen the function value, doing this with probability when the function value increases by f(y) − f(x t ). One may speculate that this implies that it is sufficient to consider only direct Hamming neighbors. Clearly, a smaller neighborhood is an advantage. And we will be able to demonstrate that this smaller neighborhood is actually sufficient in this context.

We have seen that finding some spanning graph is not a difficult problem. We simplify things by not considering this problem at all. We do this by selecting the initial search point deterministically. Since and the MST instance (V, E, w) is connected we have that x 0 encodes a feasible solution. Since its feasibility is guaranteed by the problem definition it is called the trivial feasible solution. It is not uncommon to start simulated annealing (and the Metropolis algorithm) with a trivial feasible solution. We use the fitness function f 2 for both algorithms. This choice comes with two immediate advantages. First, since f 2(x) = ∞ for any infeasible x we know that the two algorithms will never accept an infeasible solution. Moreover, f 2 is structurally very simple and possibly the most natural encoding of the minimum spanning tree problem.

We will not consider the general MST problem here but consider only one particular instance. This instance is given by the graph D n in the following definition. A graphical representation of D n can be seen in Fig. 6.8.

Definition 6.12.

Let . We define ,

and D n : = (V, E, w).

We see that D n consists of 2n triangles, n of them we call light and n of them we call heavy. Each triangle contains two edges that we call light and one edge that we call heavy. In a light triangle the light edges have weight 1 and the heavy edge has weight m. In a heavy triangle the light edges have weight m 2 and the heavy edge has weight m 3. We see that D n contains nodes and edges. There is only one minimum spanning tree, and it contains exactly all light edges.

Fig. 6.8

Graphical representation of the graph D n for n = 3

When we talk about the algorithms we adopt the notation from the proof of Theorem 6.31. For x ∈ { 0, 1} m we use c(x) for the number of complete triangles (where all three edges are selected), b(x) for the number of bad triangles (where one light and one heavy edge is selected) and introduce g(x) for the number of good triangles (where the two light edges are selected). Since the graph is always connected we have b(x) + c(x) + g(x) = 2n.

Initially, we have c(x 0) = 2n (and consequently ). Removing an edge from a complete triangle is an improvement and will be accepted. Removing a light edge turns the complete triangle into a bad triangle. Removing the heavy edge turns a complete triangle into a good triangle. Given that we remove an edge in a triangle the probability to turn it into a bad triangle is 2 ∕ 3. We see that there is a tendency to make complete triangles bad. Once a triangle is bad it needs to be made complete before it can become good. In a light triangle this corresponds to a worsening of 1, and in a heavy triangle to a worsening of m 2. Such steps should be not too unlikely because they will be needed many times before a minimum spanning tree is found. If a triangle is good it can only become bad if it first becomes complete. In a light triangle this corresponds to a worsening of m, and in a heavy triangle to a worsening of m 3. Such steps should be not too likely since we want to keep good triangles good. Now we see a problem: We would like to have the probability for increasing f 2 by m small since this makes a light good triangle complete. But we would also like to have the probability for increasing f 2 by m 2 large since this makes a heavy bad triangle complete. Since accepting to increase f 2 becomes less and less likely with increasing difference in function value, we cannot have both at the same time. Clearly, for the Metropolis algorithm this will be a big a problem. It is unclear if changing the probability over time can solve the dilemma. So, we speculate that the Metropolis algorithm has a problem on D n while there is hope for simulated annealing.

Theorem 6.33.

For any temperature the probability that the Metropolis algorithm finds a minimum spanning tree for D n using the fitness function f 2 within steps is bounded above by

Proof.

We make a case distinction with respect to the temperature T. In the case of a low temperature, T < m, we consider the heavy triangles. Since we start with n complete heavy triangles and make a complete triangle with probability 2 ∕ 3 bad the probability to find a minimum spanning tree without creating a heavy bad triangle is . The probability to turn this heavy bad triangle into a good one is bounded above by

and the expected waiting time is .

In the case of a high temperature, T ≥ m, we consider the n light triangles. Let X t denote the number of good triangles among the n light triangles in x t . We start with X t  = 0 and are looking for . The probability distribution of X t + 1 depends only on the number of good, bad, and complete light triangles at time t. We consider one light triangle in one step. If the triangle is complete it stays complete with probability 1 − 3 ∕ m. With probability 1 ∕ m it turns good, and with the remaining probability 2 ∕ m it turns bad. If the triangle is good it turns complete with probability . It cannot become bad. Thus, with the remaining probability of it stays good. If the triangle is bad it turns complete with probability . It cannot turn good and thus stays bad with probability . Using T ≥ m we obtain

as bounds on the probability to increase and decrease X t in one step. We concentrate on steps where X t actually changes and obtain

as bound. Clearly, holds. We consider the situation for X t  ∈ [(9 ∕ 11)n, n] and start our consideration when X t  = (10 ∕ 11)n holds. Within this interval we have

and observe a clear tendency to decrease the number of light good triangles. In fact, the probability to have X t  = n before we have X t  = (9 ∕ 11)n when starting with (10 ∕ 11)n is less than the probability to be ruined in the gambler’s ruin scenario (see Theorem A.4) with equal capital n ∕ 11 for both players and probability 3 ∕ 5 of winning in each step. This probability is bounded by

so that the waiting time for such an event is . Since we need to see such an event before a minimum spanning tree is found the claimed bounds follow. □

Theorem 6.33 shows that with temperatures below m it is not possible to escape from a single heavy bad triangle in polynomial time with overwhelming probability. However, with higher temperatures the light triangles have a tendency not to stay good. The only hope for simulated annealing on D n with f 2 is to start with a quite high temperature and hope that all heavy triangles turn good and stay good that way. Once this is achieved the temperature can be lowered so that the process can be repeated for light triangles at lower temperatures. Note that simulated annealing employs dynamic parameter control, not adaptive parameter control: It is not possible to check the current state and lower the temperature in response to having the bad triangles all good. But, luckily, a helpful fixed annealing schedule can be found.

Theorem 6.34.

Let p be an arbitrary polynomial in m. There is a constant c > 0 such that simulated annealing with annealing schedule finds a minimum spanning tree for D n using the fitness function f 2 within 3cmln m steps with probability 1 − 1∕p(n).

Proof.

We partition a run of simulated annealing on D n with f 2 into four phases. The first phase starts with initialization and ends when the temperature has sunk to T(t) ≤ m 5 ∕ 2 for the first time. The second phase starts after the first ends when T(t) < m 2 for the first time. The third phase starts after the second and ends when . The fourth and final phase starts after the third and ends when T(t) < 1. Note that

holds for t > 3cmlnm. Thus the claim on the optimization time follows immediately. It is easy to see that the length of each phase is bounded below by (c ∕ 4)mlnm.

We ignore the first phase completely. We only wait for the temperature to drop and do not care what happens in that time.

In the second phase we have . We consider the n heavy triangles and ignore what happens with the light triangles completely. The probability to accept a step where a heavy good triangle turns complete is bounded above by , and we see that with probability this does not happen within this phase. Since the temperature is strictly decreasing it will also not happen later with this probability.

In each step each edge is effected by the step with probability 1 ∕ m. Thus, we expect each edge to be effected (either removed or included depending on its current status) at least (c ∕ 4)lnm times in this phase. We chose c′ so small that the probability not to see the edge change its status at least c′lnm times is . We consider just one edge and only steps where its status changes. Considering the three edges of a triangle, the probability to turn a heavy bad triangle into a good one in two such subsequent steps is bounded below by

If c is sufficiently large the probability to see this for each of the n heavy triangles within the at least (c ∕ 4)mlnm steps in this phase is . Thus, with probability at the end of the second phase all heavy triangles are good. We have already argued that these triangles all stay good with probability .

The third phase we ignore similar to the first one. We do not care what happens to the light triangles in this phase and know that the heavy triangles which are all good will all stay good.

In the fourth phase we repeat what we did in the second phase for the heavy triangles for the light triangles. We have in this phase, which differs from the temperatures in the third phase by a factor of m 2. The weights of the light triangles also differ from the weights of the heavy triangles by a factor of m 2. Since this cancels out, all calculations remain unchanged. □

We consider one other combinatorial optimization problem as a final example. We consider the aspect of encoding again and use this example to show how negative results for a large class of evolutionary algorithms can be obtained.

The problem we consider is that of computing a longest common subsequence of a number of strings, LCS for short. It is an important problem in computer science and has applications in bioinformatics.

A problem instance consists of some strings . The common alphabet Σ is finite and assumed to be fixed, i.e., it is usually not part of the problem instance. In computer science the binary alphabet {0, 1} is of particular importance. In bioinformatics the alphabet Σ = { { A}, { C}, { G}, { T}} is important as these four letters usually represent the four nucleic acid bases that make up DNA. The number of strings m can be any natural number, nontrivial problem instances have m > 1.

Consider two sequences , with l ≤ l i . We call Y a subsequence of X i if there exists a sequence of indices such that holds. We see that all letters in Y need to appear in X i in the same order but they may not be contiguous in X i . We can represent the sequence of indices as a bit string with r[k] = 1 if there is some index j h that equals k and r[k] = 0 otherwise. For example, 00101 is a subsequence of 01011001 as witnessed by 1, 3, 4, 6, 8 (represented by 10110101). Note that the witnessing sequence of indices is not unique, 1, 3, 5, 7, 8 (represented by 10101011) is another valid witness.

We use this representation of an index sequence by a bit string as motivation to derive a representation of the problem to make it accessible for optimization by means of randomized search heuristics. Let be a problem instance. We assume without loss of generality that X 1 is a shortest sequence in this problem instance, i.e., . We define and use the search space {0, 1} n . We see that x ∈ { 0, 1} n represents a subsequence of X 1 as above. We let c(x) denote this subsequence. The subsequence c(x) is a feasible solution if it is also a subsequence of X 2, X 3, …, and X m , otherwise it is an infeasible solution. Among the feasible solutions we prefer search points that represent long subsequences, i.e., search points with a large number of 1-bits.

There are many ways a fitness function can be defined. We consider two options, one realistic and practical, the other only of theoretical interest and not useful in practice.

Let be a fixed problem instance with , let . In order to assess the feasibility of a candidate solution x ∈ { 0, 1} n we define a function by

We see that denotes the prefix of c(x) of length k and that { MAX}(x) yields the length of the longest prefix of c(x) that is a common subsequence of the problem instance. If x encodes a feasible solution, i.e., c(x) is a common subsequence of , we have { MAX}(x) = { ONEMAX}(x). For infeasible solutions we shorten c(x) from right to left until the remaining prefix is a common subsequence. Since this can be repeated until (the empty string) we see that { MAX}(x) is well defined. Clearly, for infeasible solutions { MAX}(x) < { ONEMAX}(x) holds.

We use { MAX}(x) to define a fitness function by using the value of { MAX}(x) and subtracting one for each letter in c(x) that needed to be removed until the remaining prefix was a common subsequence, i.e., . Clearly, a search point x  ∗  with maximal function value under f { MAX} encodes an optimal solution c(x  ∗ ) (and any optimal solution corresponds to some search point that has maximal function value under f { MAX}). In this sense f { MAX} is useful. It is not the case that infeasible solutions necessarily have smaller function values than feasible ones. In particular, the empty sequence (encoded by the all ones bit string 0 n ) is trivially feasible but has function value 0. However, for an infeasible solution the function value can always be increases by flipping a 1-bit (removing a letter that is not part of a common subsequence). Thus, we expect that it will be easy to locate feasible solutions using this problem encoding.

The idea of f { MAX} is that if x does not encode a feasible solution we shorten the represented sequence from right to left until the remaining sequence is a feasible solution. This is a very simple approach. One may wonder if the performance of randomized search heuristics improves if the fitness function is better at assessing the potential of a candidate solution. We test this idea by defining a fitness function that contains in some sense full knowledge. Note that we define this function solely to investigate this idea. It is not meant to be a ‘real’ fitness function used to solve LCS. In particular, evaluating this fitness function requires solving the problem rendering it clearly useless in a practical sense. We call this fitness function f { LCS} and use a function similar to { MAX} to define it. We define { LCS}(x) as the length of the longest common subsequence of and . In comparison to f { MAX} where for infeasible c(x) the longest prefix that is a common subsequence is used, for f { LCS} for infeasible c(x) the longest subsequence that is a common subsequence is used. Since standard bit mutations can remove arbitrary 1-bits (and not just rightmost 1-bits) f { LCS} appears to be more helpful for evolutionary algorithms.

Computing a longest common subsequence for is a difficult problem; it is NP-hard. There is a problem-specific algorithm that computes an optimal solution deterministically in time . We see that the problem becomes hard with a growing number m of strings. As long as the number of sequences is small, , an optimal solution can be computed in polynomial time. We see that we cannot expect randomized search heuristics to be efficient in the general case. However, for not too large numbers of sequences m we would hope to see polynomial expected optimization times. This hope, however, is disappointed. We will show that even for m = 2 evolutionary algorithms can be extremely inefficient when using the encoding defined by f { MAX} or f { LCS}.

The first instance we consider is

(and assume that n is a multiple of 32). Clearly, there is a large number of common subsequences. We concentrate on subsequences that cannot be made longer by adding more letters. There are two subsequences of this kind, one is 1(24 ∕ 32)n that is based on the first (24 ∕ 32)n bits in X 2 and the last (24 ∕ 32)n bits in X 1. Clearly, going with this subsequence implies that none of the first (8 ∕ 32)n bits in X 1 can be chosen. The other such subsequence is . It is based on the last (18 ∕ 32)n bits in X 2 and an appropriate selection of the 0-bits and 1-bits in X 1. We observe that the first subsequence is by (6 ∕ 32)n longer than the second and is the only optimal solution.

Will evolutionary algorithms be able to find this optimal solution using f { MAX} as encoding? The search space is {0, 1} n since n is the length of X 1 which is shorter than X 2. Let us consider some evolutionary algorithm that uses strictplus-selection and initializes its population uniformly at random. For each member of the initial population the expected number of 1-bits among the first (8 ∕ 32)n bits equals (4 ∕ 32)n, and among the last (24 ∕ 32)n bits equals (12 ∕ 32)n. Thus, application of Chernoff bounds (four times) yields that with probability the number of 1-bits among the first (8 ∕ 32)n bits is in the interval [(3 ∕ 32)n, (5 ∕ 32)n] and among the last (24 ∕ 32)n bits in the interval [(11 ∕ 32)n, (13 ∕ 32)n]. Since the population size is polynomially bounded, application of the union bound yields that with probability , the initial population only contains feasible solutions that represent a candidate solution of the second type, one that contains 0-bits followed by 1-bits. Such candidate solutions cannot be turned into global optima by flipping single bits. Using single bit mutations they can be turned into local optima representing and finding this local optimum is not harder than optimizing{ ONEMAX}, hence it will happen fast. The number of 1-bits in the first part of an individual can only decrease if at the same time the number of 1-bits in the second part increases, otherwise the fitness decreases and plus-selection does not accept that. The number of 1-bits in the second part is initially at least (11 ∕ 32)n and cannot increase above by (13 ∕ 32)n because otherwise the candidate solution become infeasible. Thus, the number of 1-bits can increase by at most (2 ∕ 32)n. Consequently, the number of 1-bits in the first part can decrease by at most (2 ∕ 32)n. Since we have at least (3 ∕ 32)n 1-bits initially, in the first part we end up with at least n ∕ 32 1-bits there. Clearly, we cannot expect to get rid of those by means of mutation or uniform crossover. Since the positions of the 1-bits have no influence on fitness it is also hard to imagine that 1-point crossover would help much. We see that a very large class of evolutionary algorithms will fail on this instance. If we defined a concrete evolutionary algorithm it would be easy to turn these observations into a formal proof. Note that the problem-specific algorithm is able to solve it deterministically in time .

When we consider f { LCS} instead of f { MAX} we consider a different problem instance to demonstrate essentially the same problems. Consider

(assuming that n is a multiple of 40). Here the longest common subsequence is 1(16 ∕ 40)n , the subsequence that is a local optimum under f { LCS} is 0(13 ∕ 40)n . Again we see that after random initialization we are with overwhelming probability in the following situation. There is a linear number N of 1-bits, , that all need to be removed in a single step. These 1-bits are randomly distributed among a number of positions that is at least , where c > 1 is a constant. Thus, it is extremely unlikely that all these 1-bits can be removed simultaneously. Thus, even providing the evolutionary algorithm with an extreme (and unrealistic) amount of domain knowledge as we did in f { LCS} does not help.

Clearly, these insights do not imply that evolutionary algorithms cannot be applied to the search for longest common subsequences. They can be much faster than the problem-specific algorithm in solving simple instances because its run time depends only on the size of the problem instance while an evolutionary algorithm may be fast on very large but simple problem instances. But we see that a simple analysis makes us aware of the fact that there are problem instances where most (if not all) realistic evolutionary algorithms will perform very badly and that we cannot claim to have an efficient heuristic for LCS in a worst-case sense. For some evolutionary algorithms, even the claim that they are efficient on average would be wrong: The worst-case instances we have seen can be produced with probability and at least mutation-based algorithms require in expectation generations to escape from the local optima, making the average expected optimization time still exponential (averaged over the problem instances, expectation taken over the random choices of the algorithm).

6.5 Remarks

Crossover is generally assumed to be the most important variation operator in genetic algorithms (while mutation is usually considered a rather unimportant ‘background operator’ there). Surprisingly, the first actual proof of a speedup by means of crossover was only presented in 1999 [56, 60]. Before that, driven by schema theory and the building block hypothesis, Mitchell, Forrest and Holland [86] define the so-called royal road function that is designed in a way that is thought to be ideal for genetic algorithms. However, experiments reveal [42] that the simple (1 + 1) EA is in fact more efficient. Only when one considers an idealized genetic algorithm that is given knowledge about optimal crossover points does the problem become easy for such an algorithm [43]. An overview article by Mitchell and Forrest [85] as well as Mitchell’s book [84] give a good overview of the development. The name ‘royal road’ motivates why originally the example functions R 1, k and R 2 were named ‘real royal road functions’ [58, 61].

The mutation rate p m  = 1 ∕ n has long been recommended [9]; Jansen and Wegener pointed out that it may be far from optimal [57]. They also proposed the dynamic (1 + 1) EA as a robust alternative to avoid having to make a choice [62]. It is now known that the (1 + 1) EA can be even much more sensitive with respect to the choice of the mutation probability. Changing the mutation probability by only a constant factor and keeping it to may change the expected optimization time from exponential to polynomial and vice versa [28].

The asymmetric mutation operator was introduced by Neumann and Wegener [92] for the minimum spanning tree problem. A more in-depth analysis than the one presented here was presented by Jansen and Sudholt [67].

The concept of coevolution was introduced to the field of evolutionary computation in the form of competitive coevolution; Hillis’ seminal work is perhaps most notable [49]. Cooperative coevolution was introduced by Potter and De Jong [103, 104]. The first run time analysis of this approach covered the CC (1 + 1) EA (Algorithm 11) and is due to Jansen and Wiegand [64–66]. Implementation details taking into account parallel or distributed computing environments are discussed in [65]. One can demonstrate even larger advantages for the CC (1 + 1) EA than we have done here, example functions where an exponential speedup can be observed are known [66]. One major open problem is the expected optimization time on linear functions in general. While the general lower bound (Theorem 6.20) can be made tight for{ ONEMAX}, only an upper bound of is known for arbitrary linear functions, even though it is speculated that this can be improved to [66].

Minimum spanning trees and longest common subsequences are by no means the only combinatorial optimization problems for which evolutionary algorithms have been analyzed. Neumann and Witt [94] give a very good introduction and overview. When tackling combinatorial optimization problems the representation becomes an important issue. Rothlauf [108] covers this subject in some depth. In theoretical studies this subject is often conveniently ignored as Jansen, Oliveto, and Zarges point out [71].

References

9.

T. Bäck, D.B. Fogel, Z. Michalewicz (eds.), Handbook of Evolutionary Computation (IOP Publishing/Oxford University Press, Bristol/Oxford, 1997)MATH

28.

B. Doerr, T. Jansen, D. Sudholt, C. Winzen, C. Zarges, Mutation rate matters even when optimizing monotonic functions. Evol. Comput. (2013). doi:10.1162/EVCO_a_00055

42.

S. Forrest, M. Mitchell, Relative building block fitness and the building block hypotheses, in Foundations of Genetic Algorithms 2 (FOGA 1993), Vail, ed. by L.D. Whitley (Morgan Kaufmann, San Francisco, 1993), pp. 198–226

43.

S. Forrest, J.H. Holland, M. Mitchell, When will a genetic algorithm outperform hill climbing? in Advances in Neural Information Processing Systems, ed. by J.D. Cowan, G. Tesauro, J. Alspector (Morgan Kaufmann, San Francisco, 1994), pp. 51–58

49.

D.W. Hillis, Co-evolving parasites improve simulated evolution in an optimization procedure. Physica D 42, 228–234 (1990)CrossRef

56.

T. Jansen, I. Wegener, On the analysis of evolutionary algorithms – a proof that crossover really can help, in Proceedings of the 7th Annual European Symposium on Algorithms (ESA 1999), Prague, ed. by J. Nesetril. Lecture Notes in Computer Science, vol. 1643 (Springer, Berlin, 1999), pp. 184–193

57.

T. Jansen, I. Wegener, On the choice of the mutation probability for the (1 + 1) EA, in Proceedings of the 6th International Conference on Parallel Problem Solving from Nature (PPSN 2000), Paris, ed. by M. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J.J. Merelo-Guervos, H.-P. Schwefel. Lecture Notes in Computer Science, vol. 1917 (Springer, Berlin, 2000), pp. 89–98

58.

T. Jansen, I. Wegener, Real royal road functions – where crossover provably is essential, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), San Francisco, ed. by L. Spector, E. Goodman, A. Wu, W. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. Garzon, E. Burke (Morgan Kaufmann, San Francisco, 2001), pp. 1034–1041

60.

T. Jansen, I. Wegener, On the analysis of evolutionary algorithms – a proof that crossover really can help. Algorithmica 34(1), 47–66 (2002)MathSciNetMATHCrossRef

61.

T. Jansen, I. Wegener, Real royal road functions – where crossover provably is essential. Discret. Appl. Math. 149, 111–125 (2005)MathSciNetMATHCrossRef

62.

T. Jansen, I. Wegener, On the analysis of a dynamic evolutionary algorithm. J. Discret. Algorithms 4(1), 181–199 (2006)MathSciNetMATHCrossRef

64.

T. Jansen, R.P. Wiegand, Exploring the explorative advantage of the cooperative coevolutionary (1 + 1) EA, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2003), Chicago, ed. by I.E. Cantu-Paz, J. Foster, K. Deb, L. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. Potter, A. Schultz, K.A. Dowsland, N. Jonoska, J. Miller. Lecture Notes in Computer Science, vol. 2723 (Springer, Berlin, 2003), pp. 310–321

65.

T. Jansen, R.P. Wiegand, Sequential versus parallel cooperative coevolutionary (1 + 1) EAs, in Proceedings of the IEEE International Congress on Evolutionary Computation (CEC 2003), Canberra (IEEE, Piscataway, 2003), pp. 30–37

66.

T. Jansen, R.P. Wiegand, The cooperative coevolutionary (1 + 1) EA. Evol. Comput. 12(4), 405–434 (2004)CrossRef

67.

T. Jansen, D. Sudholt, Analysis of an asymmetric mutation operator. Evol. Comput. 18(1), 1–26 (2010)CrossRef

71.

T. Jansen, P.S. Oliveto, C. Zarges, On the analysis of the immune-inspired B-cell algorithm for the vertex cover problem, in Proceedings of the 10th International Conference on Artificial Immune Systems (ICARIS 2011), Cambridge. Lecture Notes in Computer Science, vol. 6825 (Springer, Berlin, 2011), pp. 117–131

84.

M. Mitchell, An Introduction to Genetic Algorithms (MIT, Cambridge, 1995)

85.

M. Mitchell, S. Forrest, Royal road functions, in Handbook of Evolutionary Computation, ed. by T. Bäck, D.B. Fogel, Z. Michalewicz (IOP Publishing/Oxford University Press, Bristol/Oxford, 1997), pp. B2.7:20–B.2.7:25

86.

M. Mitchell, S. Forrest, J.H. Holland, The royal road function for genetic algorithms: fitness landscapes and GA performance, in Proceedings of the 1st European Conference on Artificial Life, Paris, ed. by F.J. Varela, P. Bourgine (MIT, Cambridge, 1992), pp. 245–254

92.

F. Neumann, I. Wegener, Randomized local search, evolutionary algorithms, and the minimum spanning tree problem, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2004), Seattle. Lecture Notes in Computer Science, vol. 3102 (Springer, Berlin, 2004), pp. 713–724

94.

F. Neumann, C. Witt, Bioinspired Computation in Combinatorial Optimization: Algorithms and Their Computational Complexity (Springer, Berlin, 2010)MATH

103.

M. Potter, K.A. De Jong, The coevolution of antibodies for concept learning, in Proceedings of the 5th International Conference on Parallel Problem Solving from Nature (PPSN 1998), Amsterdam, ed. by A. Eiben, T. Bäck, M. Schoenauer, H.-P. Schwefel. Lecture Notes in Computer Science, vol. 1498 (Springer, Berlin, 1998), pp. 530–539

104.

M. Potter, K.A. De Jong, Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evol. Comput. 8(1), 1–29 (2000)CrossRef

108.

F. Rothlauf, Representations for Genetic and Evolutionary Algorithms (Springer, Berlin, 2006)

Thomas JansenNatural Computing SeriesAnalyzing Evolutionary Algorithms2013The Computer Science Perspective10.1007/978-3-642-17339-4© Springer-Verlag Berlin Heidelberg 2013

Appendix A Fundamentals

In this appendix we collect some fundamental tools and notation that are used throughout the book. They are supposed to convey the information needed to follow the main text when something is unknown. Section  A.1 introduces Landau notation, which is used to compare the growth of functions and is most useful when talking about run times of algorithms or, in our case, expected optimization times of evolutionary algorithms. Everybody with some background in algorithms should be familiar with this notation. Section  A.2 covers the two most basic inequalities for bounding the probability that a random variable deviates from its expected value: Markov’s inequality and Chernoff bounds. It is probably safe to assume that both are covered in every introductory course in the area of probability (at least as long as the course aims at computer science students). Section  A.3 introduces martingales and the optional stopping theorem. To demonstrate their usefulness, very simple proofs for the well-known gambler’s ruin scenario are provided. This is probably the least known tool from probability theory used. Since it is not directly of interest (we only care about applying it) it appears to be more appropriate to cover it in the appendix and not in the main text.

A.1 Landau Notation

We are well acquainted with comparing numbers a and b using five different relations, namely equal ( a  =  b ), less than or equal ( a  ≤  b ), less than ( a  <  b ), greater than or equal ( a  ≥  b ), and greater than ( a  >  b ). In a very similar way we compare the growth of functions f and g using five relations. We define these relations for function .

Definition A.1.

For we define

- iff ,

- iff ,

- iff and ,

- iff , and

- iff .

We follow the usual notation and write, for example, . Note that we abuse ‘ = ’, since it is quite different from the usual equal sign. In particular, it is not symmetric: it does not even make much sense to write ‘ .’ It would be more plausible to define as the set of all functions f , such that holds in the sense of Definition A.1. Then we would write which is more meaningful. But since the notation ‘ ’ is established, we do not deviate from the standard here. We remark that it is not necessary to have functions that are defined for all values in . It suffices to consider functions where is an infinite subset of .

The correspondence to the usual relations for numbers should be clear. O corresponds to ≤ , Ω corresponds to ≥ , Θ corresponds to = , o corresponds to < , and ω corresponds to > .

A.2 Tail Estimations

We have concentrated almost exclusively on proving upper and lower bounds on the expected optimization of some evolutionary algorithm A on some fitness function f , . Sometimes, mostly in cases where this expected value is misleadingly large, we have made statements about the probability to have the optimization time T A ,  f smaller (or larger) than some value. But even in the case where we only wanted to prove a result about the expected optimization time we needed bounds on the probability that it deviates from the expected value. This is, for example, the case when the result on is used for the proof of , the expected optimization of the same algorithm on some other fitness function g , since f is in some sense a part of g . Thus, it is useful to have tools to bound the probability for T A ,  f deviating from . The most simple tool is Markov’s inequality. Since its proof is so simple and instructive, we state it complete with proof.

Theorem A.1 (Markov’s Inequality).

Let (Ω,p) be a probability space, a non-negative random variable, .

.

Proof.

We define an indicator variable Y s :  Ω  → { 0, 1} by

for some that we will set later. By definition X  ≥  s ⋅ Y s holds for any choice of s . This implies . We conclude that holds. We set and obtain . □

The strength of Markov’s inequality is that hardly anything is required. The random variable being non-negative (like time and distances always are) is sufficient. However, requiring more allows for stronger statements. The most important tool for us are Chernoff bounds. We state them in three different forms and prove only the first of them.

Theorem A.2 (Chernoff Bounds).

Let (Ω,p) be a probability space, X 1 , X 2 , …, X n : Ω →{ 0,1} independent random variables such that we have for all . We define .

Proof.

The main tools of the proof are a variable transformation, application of Markov’s inequality (Theorem A.1) and exploiting the independence of the random variables . We begin with the variable transformation and observe that

holds for all . We apply Markov’s inequality and obtain

By definition of X we have

Since the X i are independent, holds. Putting things together, we obtain

and we need no more ideas to complete the proof. We have since the X i are indicator variables. Using we obtain . We know that e x  ≥ 1 +  x holds for all and apply this with , obtaining

We define t : = ln(1 + δ) and observe that t  > 0 holds (since δ > 0 holds by assumption). This yields

as claimed.

For the proof of the second statement we start with

and can continue in the same way to obtain

for t : = ln(1 ∕ (1 − δ)). To obtain the more manageable form here and also for the third statement it suffices to consider appropriate Taylor expansions. □

A.3 Martingales and Applications

We remember Markov chains from Sect. 3.1. We have seen that evolutionary algorithms can usually be described as Markov chains. When analyzing evolutionary algorithms from this perspective we are usually interested in stopping times. We define the notion of a stopping time formally.

Definition A.2.

A random variable is a stopping time of a random process if for all the event T  =  t can be expressed in terms of .

We see that Definition A.2 forbids a stopping time to depend on future events. The question if a process has stopped at time step  t needs to be decidable by looking at the process from its start up to that time step  t (and not beyond it).

A different kind of random process that we consider here is the martingale. It is a random process that, loosely speaking, on average does not move. The formal definition is as follows.

Definition A.3.

A real-valued random process is called a martingale with respect to a random process if the following conditions all hold for all :

1.

Y n is a function of .

2.

or Y n  ≥ 0.

3.

.

We remark that a martingale may be a martingale with respect to itself, i.e., X 0  =  Y 0 , X 1  =  Y 1 , X 2  =  Y 2 , …. We observe that a martingale may or may not be a Markov chain. When we apply martingales it is usually not the case that we observe some random process to be a martingale. It is more common that we explicitly construct a martingale based on some other random process that usually is not a martingale. We will see examples for this below in the proof of Theorem A.4. The main tool for applications is the optional stopping theorem. We state this theorem here without a proof since the proof requires some effort and yields no insight that is particularly useful in our context.

Theorem A.3 (Optional Stopping Theorem).

Let be a martingale with respect to , let T be a stopping time of . If there exists some such that either T ≤ k holds almost surely or T < ∞ and holds for all t < T almost surely, then .

The formal conditions on T or are usually easy to verify. The main statement of the optional stopping theorem is that the random process that does not move in a single step, on average, will not have moved at the end, on average. While not being very surprising, we will see that it is a very useful result.

We consider the gambler’s ruin scenario. In this scenario two players are repeatedly playing a game where the random outcome is that one of them wins. Each time the losing player pays the winner one token. The two players start with s A and s B tokens, respectively. The game is unfair, and one player wins with probability p A , the other with probability p B  = 1 −  p A , and p A  ≠  p B holds. In a casino scenario where the first player is a customer and the second player is the casino, one usually has p A  <  p B and s A  ≪  s B . It is therefore quite obvious that it is highly likely that player  A will be ruined in the end. Still we would like to know the exact probability for this. Moreover, usually the first player does not really enter the game to win but to be entertained while gambling. Thus, we would like to know for how long this player can enjoy the sweet pain of losing, i.e., we are interested in the expected duration of the game.

Theorem A.4 (Gambler’s Ruin Problem).

In the gambler’s ruin scenario, let , p A ∈ (0,1) ∖{ 1∕2}. We define q := (1 − p A )∕p A . The probability that the game ends with player A being ruined equals

The expected duration of the game equals

Proof.

Let X i denote the number of tokens player A has after i rounds of the game. We have X 0  =  s A and observe that the random process is a Markov chain. By definition we have and observe that this is a stopping time for our Markov chain. Whatever the current status of the game is, the first player would ultimately win if he won the next n games where holds. This implies that

and we conclude that T  <  ∞ almost surely, a necessary condition for applying the optional stopping theorem (Theorem A.3). We define another random process based on and show that this random process is a martingale. We define p B : = 1 −  p A and and prove that this indeed defines a martingale with respect to . To this end, it suffices to see that

holds. Since is bounded (since holds) we can apply the optional stopping theorem. This yields . On the other hand, by definition of the random process, we know that

holds. Together this yields

and proves the first statement. However, we learn nothing about the from this. To this end we define a different random process by . We verify that this indeed a martingale with respect to by observing that

holds. Now the optional stopping theorem yields . On the other hand

holds and together this yields

where is known from the first part of the theorem. □

A.4 Remarks

The appendix contains only the most essential tools needed to follow the exposition given in the different chapters. It aims at being a first help, not at replacing anything else. A very useful introduction in the same spirit that is much more complete and contains more advanced methods can be found in [ 18 ].

The Landau notation is undoubtedly known to everybody with any background in computer science. It can be found in almost every textbook, see, for example, [ 15 ].

Most if not all textbooks on randomized algorithms cover Markov’s inequality and Chernoff bounds; the presentation here is close to the one by Motwani and Raghavan [ 88 ]. The book by Mitzenmacher and Upfal [ 87 ] is another very useful source.

Good sources for martingales are the textbooks by Williams [ 133 ] and Bremaud [ 13 ]. The very useful (and, of course, classical) results about the gambler’s ruin problem are usually proved without the help of martingales (see, for example, [ 39 ]). However, using the optional stopping theorem the proof becomes considerably simpler.

References

1.

E. Aarts, J. Korst, Simulated Annealing and Boltzmann Machines: A Stochastic Approach to Combinatorial Optimization and Neural Computing (Wiley, New York, 1998)

2.

E. Aarts, J.K. Lenstra (eds.), Local Search in Combinatorial Optimization (Princeton University Press, Princeton, 2003)

3.

A. Agapie, Genetic algorithms: minimal conditions for convergence, in Third European Conference on Artificial Evolution , Nimes. Lecture Notes in Computer Science, vol. 1363 (Springer, Berlin, 1997), pp. 183–206

4.

G. Anil, R.P. Wiegand, Black-box search by elimination of fitness functions, in 10th ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA 2009) , Orlando, ed. by I. Garibay, T. Jansen, R.P. Wiegand, A.S. Wu (ACM, New York, 2009), pp. 67–78

5.

D. Ashlock, Evolutionary Computation for Modeling and Optimization (Springer, Berlin, 2006)

6.

A. Auger, B. Doerr (eds.), Theory of Randomized Search Heuristics (World Scientific, Singapore, 2011)

7.

A. Auger, O. Teytaud, Continuous lunches are free! in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007) , London (ACM, New York, 2007), pp. 916–922

8.

T. Bäck, An overview of parameter control methods by self-adaptation in evolutionary algorithms. Fundam. Inform. 35 (1–4), 51–66 (1998)

9.

T. Bäck, D.B. Fogel, Z. Michalewicz (eds.), Handbook of Evolutionary Computation (IOP Publishing/Oxford University Press, Bristol/Oxford, 1997)

10.

T. Bartz-Beielstein, Experimental Research in Evolutionary Computation: The New Experimentalism (Springer, Berlin, 2006)

11.

H.-G. Beyer, H.-P. Schwefel, I. Wegener, How to analyse evolutionary algorithms. Theor. Comput. Sci. 287 , 101–130 (2002)

12.

S. Böttcher, B. Doerr, F. Neumann, Optimal fixed and adaptive mutation rates for the LeadingOnes problem, in Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN XI) , ed. by R. Schaefer, C. Cotta, J. Kołodziej, G. Rudolph. Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, 2010), pp. 1–10

13.

P. Bremaud, Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (Springer, Berlin, 1999)

14.

D. Brockhoff, T. Friedrich, N. Hebbinghaus, C. Klein, F. Neumann, E. Zitzler, On the effects of adding objectives to plateau functions. IEEE Trans. Evol. Comput. 13 (3), 591–603 (2009)

15.

T.H. Cormen, C.E. Leiserson, R.L. Rivest, C. Stein, Introduction to Algorithms , 2nd edn. (MIT, Cambridge, 2001)

16.

K.A. De Jong, Evolutionary Computation: A Unified Approach (MIT, Cambridge, 2006)

17.

M. Dietzfelbinger, B. Naudts, C.V. Hoyweghen, I. Wegener, The analysis of a recombinative hill-climber on H-IFF. IEEE Trans. Evol. Comput. 7 (5), 417–423 (2003)

18.

B. Doerr, Analyzing randomized search heuristics: tools from probability theory, in Theory of Randomized Search Heuristics , ed. by A. Auger, B. Doerr (World Scientific, Singapore, 2011), pp. 1–20

19.

B. Doerr, L.A. Goldberg, Adaptive drift analysis, in Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN XI) , Kraków, ed. by R. Schaefer, C. Cotta, J. Kołodziej, G. Rudolph. Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, 2010), pp. 32–41

20.

B. Doerr, L.A. Goldberg, Drift analysis with tail bounds, in Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN XI) , Kraków, ed. by R. Schaefer, C. Cotta, J. Kołodziej, G. Rudolph. Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, 2010), pp. 174–183

21.

B. Doerr, C. Winzen, Playing Mastermind with constant-size memory, in 29th International Symposium on Theoretical Aspects of Computer Science (STACS 2012) , Paris, ed. by C. Dürr, T. Wilke. Leibniz International Proceedings in Informatics, vol. 14 (Dagstuhl Publishing, Saarbrücken, 2012), pp. 441–452

22.

B. Doerr, C. Winzen, Reducing the arity in unbiased black-box complexity. in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2012) , Philadelphia (ACM, New York, 2012), pp. 1309–1316

23.

B. Doerr, D. Johannsen, C. Winzen, Drift analysis and linear functions revisited, in IEEE Congress on Evolutionary Computation (CEC 2010) , Barcelona (IEEE, Piscataway, 2010), pp. 1–8

24.

B. Doerr, D. Johannsen, C. Winzen, Multiplicative drift analysis, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2010) , Portland (ACM, New York, 2010), pp. 1449–1456

25.

B. Doerr, J. Lengler, T. Kötzing, C. Winzen, Black-box complexity of combinatorial problems, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2011) , Dublin (ACM, New York, 2011), pp. 981–988

26.

B. Doerr, D. Johannsen, T. Kötzing, P.K. Lehre, M. Wagner, C. Winzen, Faster black-box algorithms through higher arity operators, in 11th ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA 2011) , Schwarzenberg, ed. by H.-G. Beyer, W.B. Langdon (ACM, New York, 2011), pp. 163–172

27.

B. Doerr, F. Neumann, D. Sudholt, C. Witt, Runtime analysis of the 1-ANT ant colony optimizer. Theor. Comput. Sci. 412 (17), 1629–1644 (2011)

28.

B. Doerr, T. Jansen, D. Sudholt, C. Winzen, C. Zarges, Mutation rate matters even when optimizing monotonic functions. Evol. Comput. (2013). doi:10.1162/EVCO_a_00055

29.

S. Droste, D. Wiesmann, On the design of problem-specific evolutionary algorithms, in Advances in Evolutionary Computing , ed. by A. Ghosh, S. Tsutsui (Springer, Berlin, 2003), pp. 153–173

30.

S. Droste, T. Jansen, I. Wegener, A rigorous complexity analysis of the (1 + 1) evolutionary algorithm for linear functions with Boolean inputs, in Proceedings of the IEEE International Conference on Evolutionary Computation (ICEC 1998) , Anchorage, ed. by D. Fogel, H.-P. Schwefel, T. Bäck, X. Yao (IEEE, Piscataway, 1998), pp. 499–504

31.

S. Droste, T. Jansen, I. Wegener, On the optimization of unimodal functions with the (1 + 1) evolutionary algorithm, in Proceedings of the 5th International Conference on Parallel Problem Solving from Nature (PPSN 1998) , Amsterdam, ed. by A. Eiben, T. Bäck, M. Schoenauer, H.-P. Schwefel. Lecture Notes in Computer Science, vol. 1498 (Springer, Berlin, 1998), pp. 13–22

32.

S. Droste, T. Jansen, I. Wegener, Perhaps not a free lunch but at least a free appetizer, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 1999) , Orlando (Springer, Berlin, 1999), pp. 833–839

33.

S. Droste, T. Jansen, I. Wegener, Optimization with randomized search heuristics – the (A)NFL theorem, realistic scenarios, and difficult functions. Theor. Comput. Sci. 287 (1), 131–144 (2002)

34.

S. Droste, T. Jansen, K. Tinnefeld, I. Wegener, A new framework for the valuation of algorithms for black-box optimization, in Foundations of Genetic Algorithms 7 (FOGA) , Torremolinos, ed. by K.A. De Jong, R. Poli, J.E. Rowe (Morgan Kaufmann, San Francisco, 2003), pp. 253–270

35.

S. Droste, T. Jansen, I. Wegener, Upper and lower bounds for randomized search heuristics in black-box optimization. Theory Comput. Syst. 39 (4), 525–544 (2006)

36.

E.A. Duéñez-Guzán, M.D. Vose, No free lunch and benchmarks. Evol. Comput. (2013). doi:10.1162/EVCO_a_00077

37.

A. Eiben, J. Smith, Introduction to Evolutionary Computing (Springer, Berlin, 2007). Corrected 2nd printing

38.

A. Eiben, R. Hinterding, Z. Michalewicz, Parameter control in evolutionary algorithms. IEEE Trans. Evol. Comput. 3 (2), 124–141 (1999)

39.

W. Feller, An Introduction to Probability Theory and Its Applications. Volume I (Wiley, New York, 1957)

40.

D.B. Fogel, Evolutionary Computation: The Fossil Record (IEEE, Piscataway, 1998)

41.

L.J. Fogel, A.J. Owens, M.J. Walsh, Artificial Intelligence Through Simulated Evolution (Wiley, New York, 1966)

42.

S. Forrest, M. Mitchell, Relative building block fitness and the building block hypotheses, in Foundations of Genetic Algorithms 2 (FOGA 1993) , Vail, ed. by L.D. Whitley (Morgan Kaufmann, San Francisco, 1993), pp. 198–226

43.

S. Forrest, J.H. Holland, M. Mitchell, When will a genetic algorithm outperform hill climbing? in Advances in Neural Information Processing Systems , ed. by J.D. Cowan, G. Tesauro, J. Alspector (Morgan Kaufmann, San Francisco, 1994), pp. 51–58

44.

T. Friedrich, P.S. Oliveto, D. Sudholt, C. Witt, Analysis of diversity-preserving mechanisms for global exploration. Evol. Comput. 17 (4), 455–476 (2009)

45.

J. Garnier, L. Kallel, Statistical distribution of the convergence time of evolutionary algorithms for long-path problems. IEEE Trans. Evol. Comput. 4 (1), 16–30 (2000)

46.

D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning (Addison-Wesley, Reading, 1989)

47.

J. He, X. Yao, Drift analysis and average time complexity of evolutionary algorithms. Artif. Intell. 127 (1), 57–85 (2001)

48.

J. He, X. Yao, A study of drift analysis for estimating computation time of evolutionary algorithms. Nat. Comput. 3 (1), 21–35 (2004)

49.

D.W. Hillis, Co-evolving parasites improve simulated evolution in an optimization procedure. Physica D 42 , 228–234 (1990)

50.

J. Holland, Adaptation in Natural and Artificial Systems (University of Michigan Press, Ann Arbor, 1975)

51.

J. Horn, D. Goldberg, K. Deb, Long path problems, in Proceedings of the 4th International Conference on Parallel Problem Solving from Nature (PPSN IV) , Jerusalem. Lecture Notes in Computer Science, vol. 866 (Springer, Berlin, 1994), pp. 149–158

52.

C. Igel, M. Toussaint, On classes of functions for which no free lunch results hold. Inf. Process. Lett. 86 (6), 317–321 (2003)

53.

C. Igel, M. Toussaint, A no-free-lunch theorem for non-uniform distributions of target functions. J. Math. Model. Algorithms 3 (4), 313–322 (2004)

54.

J. Jägersküpper, Algorithmic analysis of a basic evolutionary algorithm for continuous optimization. Theor. Comput. Sci. 379 (3), 329–347 (2007)

55.

T. Jansen, Simulated annealing, in Theory of Randomized Search Heuristics , ed. by A. Auger, B. Doerr (World Scientific, Singapore, 2011), pp. 171–196

56.

T. Jansen, I. Wegener, On the analysis of evolutionary algorithms – a proof that crossover really can help, in Proceedings of the 7th Annual European Symposium on Algorithms (ESA 1999) , Prague, ed. by J. Nesetril. Lecture Notes in Computer Science, vol. 1643 (Springer, Berlin, 1999), pp. 184–193

57.

T. Jansen, I. Wegener, On the choice of the mutation probability for the (1 + 1) EA, in Proceedings of the 6th International Conference on Parallel Problem Solving from Nature (PPSN 2000) , Paris, ed. by M. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J.J. Merelo-Guervos, H.-P. Schwefel. Lecture Notes in Computer Science, vol. 1917 (Springer, Berlin, 2000), pp. 89–98

58.

T. Jansen, I. Wegener, Real royal road functions – where crossover provably is essential, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001) , San Francisco, ed. by L. Spector, E. Goodman, A. Wu, W. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. Garzon, E. Burke (Morgan Kaufmann, San Francisco, 2001), pp. 1034–1041

59.

T. Jansen, I. Wegener, Evolutionary algorithms – how to cope with plateaus of constant fitness and when to reject strings of the same fitness. IEEE Trans. Evol. Comput. 5 (6), 589–599 (2002)

60.

T. Jansen, I. Wegener, On the analysis of evolutionary algorithms – a proof that crossover really can help. Algorithmica 34 (1), 47–66 (2002)

61.

T. Jansen, I. Wegener, Real royal road functions – where crossover provably is essential. Discret. Appl. Math. 149 , 111–125 (2005)

62.

T. Jansen, I. Wegener, On the analysis of a dynamic evolutionary algorithm. J. Discret. Algorithms 4 (1), 181–199 (2006)

63.

T. Jansen, I. Wegener, A comparison of simulated annealing with simple evolutionary algorithms on pseudo-Boolean functions of unitation. Theor. Comput. Sci. 386 , 73–93 (2007)

64.

T. Jansen, R.P. Wiegand, Exploring the explorative advantage of the cooperative coevolutionary (1 + 1) EA, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2003) , Chicago, ed. by I.E. Cantu-Paz, J. Foster, K. Deb, L. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. Potter, A. Schultz, K.A. Dowsland, N. Jonoska, J. Miller. Lecture Notes in Computer Science, vol. 2723 (Springer, Berlin, 2003), pp. 310–321

65.

T. Jansen, R.P. Wiegand, Sequential versus parallel cooperative coevolutionary (1 + 1) EAs, in Proceedings of the IEEE International Congress on Evolutionary Computation (CEC 2003) , Canberra (IEEE, Piscataway, 2003), pp. 30–37

66.

T. Jansen, R.P. Wiegand, The cooperative coevolutionary (1 + 1) EA. Evol. Comput. 12 (4), 405–434 (2004)

67.

T. Jansen, D. Sudholt, Analysis of an asymmetric mutation operator. Evol. Comput. 18 (1), 1–26 (2010)

68.

T. Jansen, C. Zarges, Analysis of evolutionary algorithms: from computational complexity analysis to algorithm engineering, in 11th ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA 2011) , Schwarzenberg, ed. by H.-G. Beyer, W.B. Langdon (ACM, New York, 2011), pp. 1–14

69.

T. Jansen, C. Zarges, Analyzing different variants of immune inspired somatic contiguous hypermutations. Theor. Comput. Sci. 412 (6), 517–533 (2011)

70.

T. Jansen, K.A. De Jong, I. Wegener, On the choice of the offspring population size in evolutionary algorithms. Evol. Comput. 13 (4), 413–440 (2005)

71.

T. Jansen, P.S. Oliveto, C. Zarges, On the analysis of the immune-inspired B-cell algorithm for the vertex cover problem, in Proceedings of the 10th International Conference on Artificial Immune Systems (ICARIS 2011) , Cambridge. Lecture Notes in Computer Science, vol. 6825 (Springer, Berlin, 2011), pp. 117–131

72.

S. Kirkpatrick, C.D. Gelatt, M.P. Vecchi, Optimization by simulated annealing. Science 220 (4598), 671–680 (1983)

73.

J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (MIT, Cambridge, 1992)

74.

J.R. Koza, Genetic Programming II: Automatic Discovery of Resuable Programs (MIT, Cambridge, 1994)

75.

J.R. Koza, F.H. Bennett III, D. Andre, M.A. Keane, Genetic Programming III: Darwinian Invention and Problem Solving (Morgan Kaufmann, San Francisco, 1999)

76.

J.R. Koza, M.A. Keane, M.J. Streeter, W. Mydlowec, J. Yu, G. Lanza, Genetic Programming IV: Routine Human-Competitive Machine Intelligence (Kluwer Academic, New York, 2003)

77.

W.B. Langdon, R. Poli, Foundations of Genetic Programming (Springer, Berlin, 2002)

78.

P.K. Lehre, Fitness-levels for non-elitist populations, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2011) , Dublin (ACM, New York, 2011), pp. 2075–2082

79.

P.K. Lehre, C. Witt, Black box search by unbiased variation, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2010) , Portland (ACM, New York, 2010), pp. 1441–1448

80.

P.K. Lehre, C. Witt, Black box search by unbiased variation. Algorithmica 64 (4), 623–642 (2012)

81.

P.K. Lehre, X. Yao, On the impact of the mutation-selection balance on the runtime of evolutionary algorithms. IEEE Trans. Evol. Comput. 16 (2), 225–241 (2012)

82.

N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, Equation of state calculations by fast computing machines. J. Chem. Phys. 21 , 1087–1092 (1953)

83.

W. Michiels, E. Aarts, J. Korst, Theoretical Aspects of Local Search (Springer, Berlin, 2007)

84.

M. Mitchell, An Introduction to Genetic Algorithms (MIT, Cambridge, 1995)

85.

M. Mitchell, S. Forrest, Royal road functions, in Handbook of Evolutionary Computation , ed. by T. Bäck, D.B. Fogel, Z. Michalewicz (IOP Publishing/Oxford University Press, Bristol/Oxford, 1997), pp. B2.7:20–B.2.7:25

86.

M. Mitchell, S. Forrest, J.H. Holland, The royal road function for genetic algorithms: fitness landscapes and GA performance, in Proceedings of the 1st European Conference on Artificial Life , Paris, ed. by F.J. Varela, P. Bourgine (MIT, Cambridge, 1992), pp. 245–254

87.

M. Mitzenmacher, E. Upfal, Probability and Computing: Randomized Algorithms and Probabilistic Analysis (Cambridge University Press, Cambridge, 2005)

88.

R. Motwani, P. Raghavan, Randomized Algorithms (Cambridge University Press, Cambridge, 1995)

89.

H. Mühlenbein, Parallel genetic algorithms, population genetics, and combinatorial optimization, in Parallelism, Learning, Evolution , Neubiberg, ed. by J. Becker, I. Eisele, F. Mündemann. Lecture Notes in Computer Science, vol. 565 (Springer, Berlin, 1991), pp. 398–406

90.

H. Mühlenbein, How genetic algorithms really work: mutation and hillclimbing, in Proceedings of the 2nd International Conference on Parallel Problem Solving from Nature (PPSN II) , Brussels (Elsevier, Amsterdam, 1992), pp. 15–26

91.

H. Mühlenbein, D. Schlierkamp-Voosen, The science of breeding and its application to the breeder genetic algorithm (BGA). Evol. Comput. 1 (4), 335–360 (1993)

92.

F. Neumann, I. Wegener, Randomized local search, evolutionary algorithms, and the minimum spanning tree problem, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2004) , Seattle. Lecture Notes in Computer Science, vol. 3102 (Springer, Berlin, 2004), pp. 713–724

93.

F. Neumann, C. Witt, Runtime analysis of a simple ant colony optimization algorithm. Algorithmica 54 (2), 243–255 (2009)

94.

F. Neumann, C. Witt, Bioinspired Computation in Combinatorial Optimization: Algorithms and Their Computational Complexity (Springer, Berlin, 2010)

95.

A.E. Nix, M.D. Vose, Modelling genetic algorithms with Markov chains. Ann. Math. Artif. Intell. 5 (1), 77–88 (1992)

96.

P.S. Oliveto, C. Witt, Simplified drift analysis for proving lower bounds in evolutionary computation. Algorithmica 59 (3), 369–386 (2011)

97.

R. Poli, Exact schema theory for genetic programming and variable-length genetic algorithms with one-point crossover. Genet. Program. Evolvable Mach. 2 (2), 123–163 (2001)

98.

R. Poli, Tournament selection, iterated coupon-collection problem, and backward-chaining evolutionary algorithms, in Foundations of Genetic Algorithms (FOGA 2005) , Aizu-Wakamatsu, ed. by A.H. Wright, M.D. Vose, K.A. De Jong, L.M. Schmitt. Lecture Notes in Computer Science, vol. 3469 (Springer, Berlin, 2005), pp. 132–155

99.

R. Poli, N.F. McPhee, General schema theory for genetic programming with subtree-swapping crossover: part I. Evol. Comput. 11 (1), 53–66 (2003)

100.

R. Poli, N.F. McPhee, General schema theory for genetic programming with subtree-swapping crossover: part II. Evol. Comput. 11 (2), 169–206 (2003)

101.

R. Poli, N.F. McPhee, J.E. Rowe, Exact schema theory and Markov chain models for genetic programming and variable-length genetic algorithms with homologous crossover. Genet. Program. Evolvable Mach. 5 (1), 31–70 (2004)

102.

R. Poli, W.B. Langdon, N.F. McPhee, A Field Guide to Genetic Programming (Lulu, 2008). http://www.gp-field-guide.org.uk

103.

M. Potter, K.A. De Jong, The coevolution of antibodies for concept learning, in Proceedings of the 5th International Conference on Parallel Problem Solving from Nature (PPSN 1998) , Amsterdam, ed. by A. Eiben, T. Bäck, M. Schoenauer, H.-P. Schwefel. Lecture Notes in Computer Science, vol. 1498 (Springer, Berlin, 1998), pp. 530–539

104.

M. Potter, K.A. De Jong, Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evol. Comput. 8 (1), 1–29 (2000)

105.

A. Prügel-Bennett, A. Rogers, Modelling genetic algorithms dynamics, in Theoretical Aspects of Evolutionary Computing , ed. by L. Kallel, B. Naudts, A. Rogers (Springer, Berlin, 2001), pp. 59–85

106.

Y. Rabani, Y. Rabinovich, A. Sinclair, A computational view of population genetics. Random Struct. Algorithms 12 (4), 313–334 (1998)

107.

I. Rechenberg, Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der Evolution (Frommann-Holzboog, Stuttgart-Bad Cannstatt, 1973)

108.

F. Rothlauf, Representations for Genetic and Evolutionary Algorithms (Springer, Berlin, 2006)

109.

J.E. Rowe, M.D. Vose, A.H. Wright, Reinterpreting the no free lunch. Evol. Comput. 17 (1), 117–129 (2009)

110.

G. Rozenberg, T. Bäck, J.N. Kok (eds.), Handbook of Natural Computing (Springer, Berlin, 2012)

111.

G. Rudolph, How mutation and selection solve long path problems in polynomial expected time. Evol. Comput. 4 (2), 195–205 (1996)

112.

G. Rudolph, Convergence Properties of Evolutionary Algorithms (Kovac, Hamburg, 1997)

113.

C. Schumacher, M.D. Vose, L.D. Whitley, The no free lunch and problem description length, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001) , San Francisco (Morgan Kaufmann, San Francisco, 2001), pp. 565–570

114.

H.-P. Schwefel, Evolution and Optimum Seeking (Wiley, New York, 1995)

115.

J.L. Shapiro, Statistical mechanics theory of genetic algorithms, in Theoretical Aspects of Evolutionary Computing , ed. by L. Kallel, B. Naudts, A. Rogers (Springer, Berlin, 2001), pp. 87–108

116.

A. Sinclair, Algorithms for Random Generation and Counting: A Markov Chain Approach (Birkhäuser, Boston, 1996)

117.

C.R. Stephens, H. Waelbroeck, Schemata evolution and building blocks. Evol. Comput. 7 (2), 109–125 (1999)

118.

D. Sudholt, Computational complexity of evolutionary algorithms, hybridizations, and swarm intelligence. Ph.D. thesis, Technische Universität Dortmund, 2008

119.

D. Sudholt, General lower bounds for the running time of evolutionary algorithms, in Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN XI) , Kraków, ed. by R. Schaefer, C. Cotta, J. Kołodziej, G. Rudolph. Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, 2010), pp. 124–133

120.

D. Sudholt, A new method for lower bounds on the running time of evolutionary algorithms. Technical report abs/1109.1504v2, CoRR, 2011. http://arxiv.org/abs/1109.1504

121.

D. Sudholt, C. Thyssen, Running time analysis of ant colony optimization for shortest path problems. J. Discret. Algorithms 10 , 165–180 (2012)

122.

D. Sudholt, C. Witt, Runtime analysis of a binary particle swarm optimizer. Theor. Comput. Sci. 411 (21), 2084–2100 (2010)

123.

A. Valsecchi, L. Vanneschi, G. Mauri, Optimisation speed and fair sets of functions, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2010) , Portland (ACM, New York, 2010), pp. 1475–1476

124.

P.M.B. Vitanyi, A discipline of evolutionary programming. Theor. Comput. Sci. 241 (1–2), 3–23 (2000)

125.

M.D. Vose, The Simple Genetic Algorithm: Foundations and Theory (MIT, Cambridge, 1999)

126.

R.A. Watson, G. Hornby, J.B. Pollack, Modeling building-block interdependency, in Proceedings of the 5th International Conference on Parallel Problem Solving from Nature (PPSN 1998) , Amsterdam, ed. by A. Eiben, T. Bäck, M. Schoenauer, H.-P. Schwefel. Lecture Notes in Computer Science, vol. 1498 (Springer, Berlin, 1998), pp. 97–108

127.

I. Wegener, Theoretical aspects of evolutionary algorithms, in International Colloquium on Automata, Languages and Programming (ICALP 2001) , Heraklion (Springer, Berlin, 2001), pp. 64–78

128.

I. Wegener, Methods for the analysis of evolutionary algorithms on pseudo-Boolean functions, in Evolutionary Optimization , ed. by R. Sarker, X. Yao, M. Mohammadian (Kluwer Academic, New York, 2002), pp. 349–369

129.

I. Wegener, Towards a theory of randomized search heuristics, in International Symposium on Mathematical Foundations of Computer Science (MFCS 2003) , Bratislava (Springer, Berlin, 2003), pp. 125–141

130.

I. Wegener, Complexity Theory: Exploring the Limits of Efficient Algorithms (Springer, Berlin, 2005)

131.

I. Wegener, Simulated annealing beats Metropolis in combinatorial optimization, in Automata, Languages and Programming, 32nd International Colloquium (ICALP 2005) , Lisbon, ed. by L. Caires, G. Italiano, L. Monteiro, C. Palamidessi, M. Yung. Lecture Notes in Computer Science, vol. 3580 (Springer, Berlin, 2005), pp. 589–601

132.

I. Wegener, J. Scharnow, K. Tinnefeld, The analysis of evolutionary algorithms on sorting and shortest paths problems. J. Math. Model. Algorithms 3 , 349–366 (2004)

133.

D. Williams, Probability with Martingales (Cambridge University Press, Cambridge, 1991)

134.

C. Witt, Runtime analysis of (μ + 1) EA on simple pseudo-Boolean functions. Evol. Comput. 14 (1), 65–86 (2006)

135.

C. Witt, Optimizing linear functions with randomized search heuristics – the robustness of mutation, in 29th International Symposium on Theoretical Aspects of Computer Science (STACS 2012) , Paris, ed. by C. Dürr, T. Wilke. Leibniz International Proceedings in Informatics, vol. 14 (Dagstuhl Publishing, Saarbrücken, 2012), pp. 420–431

136.

D.H. Wolpert, W.G. Macready, No free lunch theorems for search. Technical report SFI-TR-9502-010, Santa Fe Institute, 1995

137.

D.H. Wolpert, W.G. Macready, No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1 (1), 67–82 (1997)

138.

C. Zarges, On the utility of the population size for inversely fitness proportional mutation rates, in 10th ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA 2009) , Orlando, ed. by I. Garibay, T. Jansen, R.P. Wiegand, A.S. Wu (ACM, New York, 2009), pp. 39–46

Index

Analytical methods
delay sequences
drift analysis
fitness-based partitions
trivial f -based partition
random family trees
typical events
typical runs

Annealing schedule

Arithmetic crossover. See Crossover

Asymmetric mutation. See Mutation
b-bit mutation. See Mutation

BINVAL. See Fitness function

Black-box algorithm

Black-box complexity

Black-box optimization

Boltzmann selection. See Selection

Building block

Building block hypothesis

Canonical partition

Chernoff bounds

Comma-selection. See Selection

Coupon collector’s problem

Crossover
arithmetic crossover
gene pool crossover
intermediate recombination
k-point crossover
probability
uniform crossover

Delay sequences. See Analytical methods

Drift analysis. See Analytical methods

Dynamic (1C1) EA

Evolutionary cycle

Fitness function
BINVAL
function of unitation
H-IFF
JUMPk
LEADINGONES
linear function
LONGPATHk
NEEDLE
ONEMAX
PLATEAU
RIDGE
unimodal function

Fitness-based partitions. See Analytical
methods

Fitness-proportional selection. See Selection

Function of unitation. See Fitness function

Gambler’s ruin problem

Gene pool crossover. See Crossover

Generation

Genetic programming

Genotype–phenotype mapping

H

Hamming distance

Hamming neighbor

H-IFF. See Fitness function

Individual

Initialization

Intermediate recombination. See Crossover

Iterated local search

JUMPk . See Fitness function

k-point crossover. See Crossover

Landau notation

LEADINGONES. See Fitness function

Linear function. See Fitness function

Long k-path

Longest common subsequence

LONGPATHk. See Fitness function

Markov chain

Markov’s inequality

Martingale

Metric

Metropolis algorithm

Minimum spanning tree

Mutation
asymmetric mutation
b-bit mutation
Gaussian mutation
probability
standard bit mutation

NEEDLE. See Fitness function

No free lunch theorem

Offspring

Offspring population size

ONEMAX. See Fitness function

Optimization time

Optional stopping theorem

Parameter settings

Parents

Plus-selection. See Selection

Population

Population size

Pure random search

Random family trees. See Analytical methods

Random local search

RIDGE. See Fitness function

Schema

Search space

Selection
Boltzmann selection
comma-selection
fitness-proportional selection
plus-selection
rank selection
selection for replacement
selection for reproduction
tournament selection
truncation selection
uniform selection

Simple GA

Simulated annealing

Standard bit mutation. See Mutation

Termination criterion

Tournament selection. See Selection

Transition probability

Truncation selection. See Selection

Typical events. See Analytical methods

Typical runs. See Analytical methods

Uniform crossover. See Crossover

Uniform selection. See Selection

Unimodal
function ( see fitness function)
weakly unimodal

Union bound

Variation

Weakly unimodal. See Unimodal

Yao’s minimax principle