PURSUIT DIFFERENTIAL GAMES WITH STATE CONSTRAINTS

We prove the existence of a value for pursuit games with state constraints. We also prove that this value is lower semicontinuous.

In this paper, we intend to prove that two-players differential games with state-constraints have a value.
We investigate a differential game where the first player, called Ursula, playing with u, controls a first system y (t) = g(y(t), u(t)), u(t) ∈ U (1) and has to ensure the state constraint y(t) ∈ K U to be fulfilled, while the second player, called Victor, playing with v, controls a second system and has to ensure the state constraint z(t) ∈ K V .The first player aims the state of the full system (y(•), z(•)) at avoiding a target as long as possible while the second player aims the state of the system at reaching this target in minimal time.
This game is known as the pursuit game.Most of the examples and results of the early theory for differential games are concerned with this problem (for several examples and for methods of explicit resolution, see Isaacs [25], Flynn [19], Breakwell [9], Bernhard [8]).
As usually in differential game theory, one can define two value functions for the game, the upper one and the lower one.The purpose of this paper is to give some conditions on the system under which the pursuit game has a value, i.e., that the upper value function is equal to the lower value function.We have to face two difficulties: the presence of state-constraints, and the fact that the value functions can be discontinuous.Let us point out that in most examples studied in Isaacs' book [25] one has to face at least one of the difficulties, and often both.
More recently, the techniques of viscosity solutions allow Evans & Souganidis [17] to simplify the proofs of existence of a value and to characterize this value as unique solution of some Hamilton-Jacobi equation.Pursuit games without state constraints and for continuous value functions are studied by Soravia in [34].Alziary [1] uses these methods for a particular continuous pursuit game with state-constraints (the Lion & Man game).
In [32], Rozyev & Subbotin prove the existence of a value for differential game without continuity and with some state-constraints for one player (Victor).However, these results couldn't directly be extended to differential games with separate dynamics and with state constraints on both dynamics.Actually, the basic idea of this method -the so-called "extremal aiming" (which gives the strategy) -is not applicable to situations where there are state constraints for both players.Let us also point out that their approach is devoted to games in a context of strategies slightly different from the one used in this paper.This method is adapted to the kind of strategies we use, the non-anticipative strategies, and to the viscosity solution approach, by Bardi, Bottacin and Falcone in [5].Our Definition of value function is (partially) borrowed from this paper.
When this paper was complete, we received a preprint of Bardi, Koike & Soravia [6] establishing the existence of a value for pursuit games with state constraints when this value is continuous.Let us point out that the constraints in [6] are more general than ours.However, the method developed in [6] heavily relies on the fact that the value function is continuous.For getting this continuity, the authors assume that one of the value functions vanishes at the boundary of the target and is continuous on this boundary, and that each control system (1) and ( 2) is locally controllable.
In this paper, we do not make any controllability condition at the boundary of the target, so that the value of the game is in general not continuous.However, following Soner [33], we make some restriction on the dynamics at the boundary of the state constraints.Under this restriction, we prove that the pursuit-evasion game has a value.Let us point out that most examples given in the "classical theory" of differential games are concerned by our result.
We follow here the method described in [13] for characterizing the value functions of differential game: Namely, we reduce the study of the pursuit game to a qualitative differential game called "approach-evasion game".The idea of reducing a quantitative game to a qualitative one comes back to Isaacs [25].
Here we use an idea of Frankowska for control problems, which amounts to characterize the epigraph of the value function (see [20] for instance, or [13] for further references on the subject).So, in a first step, we study this qualitative game (or game of kind, in Isaacs terminology), we give an "alternative result" for that game and characterize the victory domains in a geometric way.Then, in a second step, we interpret the pursuit game as a qualitative pursuit evasion game in order to prove that there is a value to the problem.

Existence of a value for the pursuit game
In this section, we state the main result of this paper, namely that pursuit games have a value.For doing so, we first introduce some notations and assumptions.

Notations and assumptions
The dynamics of the system are: The first player (Ursula), controlling u, has to ensure that y(t) ∈ K U , while the second player (Victor), playing with v, has to ensure that z(t) ∈ K V .
The sets K U and K V have smooth boundaries.Moreover, we assume some transversality conditions of the vector fields at the boundary of the state con-straints similar as that of Soner [33].Namely, U and V are compact subsets of some finite dimensional spaces ii) f For any time-measurable controls u(•) and v(•), we denote by y[y 0 , u(•)], z[z 0 , v(•)] and x[x 0 , u(•), v(•)] the solutions (in the Caratheodory sense) to equations (1), (2) and to equation starting respectively from y 0 , z 0 and x 0 := (y 0 , z 0 ).The sets of time measurable controls u(•) : I R + → U and v(•) : I R + → V are denoted respectively U and V, while the sets of admissible controls are denoted by U(y 0 ) and V(z 0 ): Under condition (3), it is well known (see [3]) that the sets U(y 0 ) and V(z 0 ) are not empty for any y 0 ∈ K U and z 0 ∈ K V .Moreover, according to Arisawa-Lions [2] (see also [27]), the sets U(y 0 ) and V(z 0 ) are Lipschitz continuous with respect to y 0 and z 0 .Namely (for instance for y) Lemma 1.1 Under assumption (3), for any positive constants Q and T , there is some positive λ = λ(Q, T, ) such that, for any y 0 , y 1 belonging to K U , with y 0 ≤ Q and y 1 ≤ Q, for any admissible control u 0 (•) ∈ U(y 0 ), there is some

Remarks on assumptions (3):
1.The regularity assumptions on the domains K U and K V can also be weakened, by using an extension of Lemma 1.1 recently obtained by Frankowska-Rampazzo in [21] for non smooth domains.The transversality condition has to be extended in a suitable way.The results of the present paper also hold true (with the exactly the same proof) under Frankowska-Rampazzo assumption provided that the sets K U and K V are sleek in the sense of [3].
2. Following also [21], the convexity assumption (iii) can be avoided.We have preferred not to do so for simplicity.However, if one omits this assumption, one has to modify the transversality assumption in a suitable way as well as the Definition of ϑ C (see [21]).
The non-anticipative strategies β for the second player Victor are defined symmetrically and we denote by S V (x 0 ) the set of such strategies.
Throughout this paper, B denotes the closed unit ball of I R N (endowed with the Euclidean norm) and d S (x) denotes the distance from a point x to a set S. Moreover, if S is a subset of I R N and ε is positive, we denote by S + εB the set

The main Theorem
∈ C for every t ≥ 0, then we set θ C (x(•)) := +∞.In the pursuit game, Ursula wants to maximize θ C while Victor wants to minimize it.

Definition 1.2 (Value functions)
The lower optimal hitting time function is the map ϑ C : K U × K V → I R + ∪ {+∞} defined, for any x 0 := (y 0 , z 0 ), by The upper optimal hitting time function is the map ϑ

Remarks :
1. Let us point out that the limit in the definition of ϑ C exists because the quantity sup is non decreasing with respect to ε.Such a Definition is used for instance in [5].The meaning of such a definition is the following: Whatever strategy α is played, the second player can ensure the state of the system to go as close as he wants to the target before ϑ C (x 0 ) (but the state of the system need not reach the target).
2. The following Definition of the upper value function has been used in several papers: However, without controllability assumptions on the boundary of the target, we cannot hope to have a value with this Definition of upper value function.For instance, if one considers the unconstrained pursuit game where the dynamics is and the target is Proof : We first prove the last equality: Let us notice that β(u(•))(t) = u(t) is an optimal strategy for the second player.Hence ϑ C (0, 0, 0) = 1.Equality ϑ C (0, 0, 0) = ϑ C (0, 0, 0) comes from Theorem 1.3 below.
We now prove the first equality.We define the non-anticipative strategy α in the following way.For any control v(•), let z 1 (•) be the solution to Let us point out that such a map α is a non-anticipative strategy.We claim that, for any control v(•), the solution We consider two cases.If on the one hand, c > −1, then there is some τ ∈ (0, 1) such that Therefore, for any t, z 1 (t) > y(t), so that x(t) / ∈ C.
If, on another hand, c = −1, there is a sequence Hence there is some n such that t n ∈ (0, 1) and Therefore, for any t ≥ t n , z 1 (t) < y(t), so that x(t) / ∈ C.
3. A natural question is: What happens if one modifies the lower value function in the same way ?We show below that 4. A last remark, which is not really interesting for differential games, but maybe is interesting on a p.d.e. point of view.In the theory of viscosity solutions, one often characterizes the solution through its lower semicontinuous and upper semi-continuous envelope.In the above example we can notice that the lower semi-continuous envelope of ϑ C is not equal to ϑ C .
Theorem 1.3 Assume that conditions (3) are fulfilled.Then the game has a value: This Theorem is proved in section 4 by reducing the pursuit game to a qualitative differential game called the pursuit evasion game.We deduce from the "alternative Theorem" for this qualitative game (see Theorem 2.6 below) the existence of a value for the pursuit game.
Moreover the proof gives a geometric characterization of the value function.This characterization can be formulated as a Hamilton-Jacobi-Isaacs equation (see [13]).We shall not do so for sake of shortness.As indicated in [13], we can also derive from this characterization numerical schemes for computing the value function.

An Alternative Theorem for a qualitative differential game with state constraints
In this section, we study the differential game in which the first player Ursula, controlling system (1), aims the state of the full system at reaching an open target O while the other player, Victor, controlling system (2), aims the state of the system at avoiding O and -if possible -at reaching some given evasion set E. This game is very close to the approach-evasion game of Krasovskii-Subbotin [26].

Statement of the qualitative problem
The victory domains of the players are defined as follows: • Victor's victory domain is the set of initial positions x 0 := (y 0 , z 0 ) belonging to K U × K V for which there is an admissible non-anticipative strategy β : U(y 0 ) → V(z 0 ) such that, for any admissible control u(•) ∈ U(y 0 ), the solution x[x 0 , u(•), β(u(•))] avoids O as long as it does not reach E (or avoids O on [0, +∞) if it never reaches E).
• Ursula's victory domain is the set of initial positions x 0 := (y 0 , z 0 ) belonging to K U × K V for which there are T ≥ 0, ε > 0 and an admissible non-anticipative strategy α : V(z 0 ) → U(y 0 ) such that, for any admissible control In this section, we prove the following alternative result: if x belongs to K U × K V , then x belongs to one and only one victory domain.Moreover, we give a geometric characterization of the victory domains.

The discriminating domains
For x := (y, z) ∈ I R N , we set where T K U (y) is the usual tangent half-space to the set with smooth boundary K U at y. Let us notice that, under assumptions (3), the set-valued map y → f (y, U (y)) is lower semi-continuous with convex compact values (see [4]).
Let us introduce the Hamiltonian of our system: where x := (y, z) and where where N P D (x) denotes the set of proximal normal to D at x, i.e., the set of p ∈ I R N such that the distance of x + p to D is equal to p .
For the original definition of discriminating domains, see Aubin [3].
Discriminating domains can be characterized in two different ways: Remark : This result was proved independently and in the same time by Plaskacz [29] and by the first author [10] when K U = I R l and E = ∅.For timemeasurable dynamics, see also [14].Theorem 2.3 (in a more general form) was announced in [12].
Theorem 2.4 Suppose that assumptions (3) are fulfilled.A closed set D ⊂ K U ×K V is a discriminating domain for H if and only if, for any initial position x 0 := (y 0 , z 0 ) ∈ D, for any admissible non-anticipative strategy α : V(z 0 ) → U(y 0 ), for any T ≥ 0 and any ε > 0, there is an admissible control \O is a discriminating domain, then D is a subset of Victor's victory domain (according to Theorem 2.3) and has an empty intersection with Ursula's victory domain (according to Theorem 2.4).
The proof of Theorems 2.3 and 2.4, being rather technical, are given in appendix.

The Alternative Theorem
We now characterize Victor's and Ursula's victory domains.For that purpose, we first recall the definition of the discriminating kernel.
Proposition 2.5 ( [11]) Let H : I R N × I R N → I R be a lower semi-continuous map.If K is a subset of I R N , then K contains a largest (for the inclusion) closed discriminating domain for H.This set is called the discriminating kernel of K for H and is denoted Disc H (K).
Theorem 2.6 (Alternative Theorem) Let H be defined by ( 5) and assume that (3) is fulfilled.Then, • Victor's victory domain is equal to Disc H (K).
• Ursula's victory domain is equal to In particular, any point of (K U × K V ) belongs either to Victor's victory domain or to Ursula's one.When K U = K V = I R N , this characterization can be found in [10].
The proof is given in appendix.
3 Proof of the existence of a value for the pursuit game We come back to the problem of the existence of a value (see section 1), i.e., to the equality between ϑ C and ϑ C .We are going to prove that their epigraph are equal.Let us recall that the epigraph of ϑ C (for instance) is a subset of I R N +1 defined by In the sequel, we always denote by (x, ρ) any point of I R N +1 , where x ∈ I R N and ρ ∈ I R.
Theorem 3.1 Assume that conditions (3) are fulfilled.Then we have: where K := K U × K V × I R + and where the Hamiltonian Remarks : • This result proves Theorem 1.3 since the functions ϑ C and ϑ C , having the same hypograph, are equal.
• We can deduce from this result that the map ϑ C = ϑ C is lower semicontinuous, since its epigraph is closed (the set Disc H (K) being closed from Proposition 2.5).
• We can also derive from the proof of Theorem 3.1 the existence of an optimal strategy for pursuer (Victor).
Proof of Theorem 3.1 : Proof of the first equality of ( 6): Let us introduce the following dynamic: Note that the Hamiltonian H defined above is actually of the form of the Hamiltonian H defined by (5) for the dynamics f and for the closed evasion set defined by E := C × I R.
Since the discriminating kernel is a closed set, we have proved that Proof of the second equality of (6) : Let (y 0 , z 0 , w 0 ) belong to Disc H (K). Fix ε > 0 and let α : V(z 0 ) → U(y 0 ) be such that From Theorem 2.6, for this non-anticipative strategy α, for this ε > 0 and for T := w 0 + ε, there is a control v(•) ∈ V(z 0 ) such that the solution to remains in K + εB on [0, T ] as long as it does not reach (C × I R) + εB.
We can also use the previous Theorem in order to prove some stability results.Namely Proposition 3.2 Under the assumptions of Theorem 3.1 Proof : Let us denote by ϑ C+ B and H the value and the Hamiltonian associated with the target C + B: According to Theorem 3.1, we have: where Since H ≤ H, the following inequality is obvious: Conversely, since the lower semi-continuous Hamiltonians H converge in the sense of Proposition 1.2 of [11], this Proposition states that the decreasing limit of Disc H (K) is a discriminating domain for H. Since this limit is contained in K, because so are Disc H (K), it is contained in Disc H (K).So we have proved that which is equivalent with saying that

Appendix
We now prove Theorems 2.3, 2.4 and 2.6.The proof of these results has the same framework as the proof of Theorems 2.1, 2.2, 2.3 and 2.4 of [10].However, the key points of the proofs essentially differ because of the presence of the constraints.Hence, for sake of shortness, we refer to [10] for the framework of the proofs and we only give the key points.

Proof of Theorem 2.3
The condition is sufficient : Assume that D is a discriminating domain.
The crucial point of the proof is the following Lemma: Lemma 4.1 Suppose that assumptions of Theorem 2.3 are fulfilled and assume that D is a discriminating domain for H. Then for any The sequel of the proof runs as in Theorem 2.1 of [10] or as in [14].
Proof of Lemma 4.1 : Let us assume that D is a discriminating domain for H. Let x 0 ∈ D and u 0 (•) ∈ U(y 0 ).It is enough to prove that there is a measurable control v(•) ∈ V and a time T > 0 such that x(•) : then the proof is obvious.We now assume x 0 / ∈ E. We divide the proof in two steps: in the first step we assume that y 0 belongs to the interior of K U and in the second step that y 0 belongs to ∂K U .

First step :
We assume that there is some T > 0 such that the solution y(•) := y[y 0 , u 0 (•)] remains in Int(K U ) on [0, T ].Then we are going to prove that there is a control v Let us introduce the following open set: Let us define the set-valued map: The main point of the proof is to check the assumptions for applying the measurable viability theorem of [20] for the set W ∩ D and F . the Measurable Selection Theorem states that there exists a control v(•) ∈ V(z 0 ) such that x(•) = x[x 0 , u 0 (•), v(•)] as long as this solution has not reached E. After the solution has reached E, we can set v(t) = v where v is any element of V .
The condition is necessary : Assume that the closed set D satisfies the property given in Theorem 2.3.Let x = (ȳ, z) ∈ D\E, p ∈ N P D (x) and ū ∈ U (ȳ).We have to prove that sup Since the set-valued map y → g(y, U (y)) is lower semi-continuous with compact convex values, Michael Selection Theorem (see [4]) yields the existence of a continuous selection w : K U → U of this set-valued map, i.e., w(y) ∈ g(y, U (y)) such that w(ȳ) = g(ȳ, ū).Let y(•) be any solution to the differential equation Then from the Measurable Selection Theorem, there is some measurable control Let β be a non-anticipative strategy as in Theorem 2.3.Then x(•) := x[x, u(•), β(u(•))] remains in D and thus in I R N \(x + p + p B). Then from standard arguments, there is a sequence

Proof of Theorem 2.4
The condition is sufficient: The proof of the sufficient condition follows the proof of Theorem 2.3 in [10] with Lemma 4.2 below instead of Lemma 4.4 of [10].In Lemma 4.4 of [10], we use a kind of "extremal aiming" method.Extremal aiming [26] amounts to associate with any point x / ∈ D some projection x of x onto D, and to play, for Ursula some u ∈ U such that inf v < f (x, u, v), x − x > is maximum, and for Victor some v ∈ V such that sup u < f (x, u, v), x− x > is minimum.Unfortunately, this method fails here since the players have to play admissible strategies and the strategies given by the extremal aiming method have no reason to be admissible.
For stating Lemma 4.2, let us fix some notations.Since we are only working on the bounded interval [0, T ] and with solutions starting from initial position x 0 , we denote by Q a radius such that any solution starting from x 0 remains in QB on [0, T ].We denote by M a upper bound of f on QB.Thanks to Lemma 1.1, there is some control ū(•) ∈ U(ȳ) such that From Lemma 4.1, there is some admissible control v(•) ∈ V(z) such that the solution x[x, ū(•), v(•)] remains in D. Thanks to Lemma 1.1, there is some control for some constants a and b, since from the very construction of u(•).Therefore which is the desired result since p = d D (x).
Necessary condition : let us assume that D is not a discriminating domain for H.We are going to prove the existence of some point x 0 and of a nonanticipative strategy α : V(z 0 ) → U(y 0 ), of positive ε and T such that, for any v(•) ∈ V(z 0 ), the solution x[x 0 , α(v(•)), v(•)] leaves D + εB before T and avoids E + εB on [0, T ].
Let x 0 = (y 0 , z 0 ) ∈ ∂D be such that the normal condition is not satisfied.Clearly x 0 / ∈ E and there is some p = (p y , p z ) ∈ N P D (x 0 ) and some γ > 0 such that inf < g(y 0 , u), p y > ≥ γ .
For τ > 0 sufficiently small, for any admissible control v(•) ∈ V(z 0 ), we have Let v(•) ∈ V(z 0 ) be any admissible control.Then, since p is a proximal normal to D at x 0 , for any t ∈ (0, τ ), which is positive for any t ∈ (0, τ ) if τ > 0 is sufficiently small.
In conclusion, the desired non-anticipative strategy α :

Proof of Theorem 2.6
We first prove that Disc H (K) is equal to Victor's victory domain.This runs as in the proof of Theorem 2.2 of [10] where we use the following sequence K n instead of the sequence K n of [10]: For the characterization of Ursula's victory domain, we introduce the complement of Ursula's victory domain: Thanks to Theorem 2.4, Disc H (K) ⊂ L. So we have to prove the converse inclusion.
For doing this, the key argument of [10] (Lemma 5.1) is replaced by the following In the proof of Lemma 5.1 of [10], the same strategy α 0 (•) gave the desired result for any x ∈ (x 0 + ηB) -provided that η was sufficiently small.The situation is more complicated here because α 0 is not necessarily an admissible strategy for x ∈ (x 0 + ηB) ∩ K because of the constraints.For solving this difficulty, we use the following Lemma proved below.Although we shall apply this Lemma indifferently to y and to z, we only formulate it for y.Lemma 4.4 Let Q and T be fixed positive constants.There is some λ > 0 (depending on the constants of the problem on Q and on T ) such that, for any y and y 0 belonging to K U , with y ≤ Q and y 0 ≤ Q, there is a non-anticipative strategy σ : U(y) → U(y 0 ) with, for any u(•) ∈ U(y), for t ∈ [0, T ], It is easy to check that Σ is non-anticipative in the sense of [14], so that, from Plaskacz Lemma (Lemma 2.7 of [14]), it enjoys a non-anticipative selection σ, i.e., σ is non-anticipative and σ(u 0 (•)) ∈ Σ(u 0 (•)) for any u 0 ∈ U(y 0 ).

Theorem 2 . 3
Suppose that assumptions (3) are fulfilled.A closed subset D of K U × K V is a discriminating domain for H if and only if, for any initial position x 0 = (y 0 , z 0 ) ∈ D, there is a non-anticipative strategy β ∈ S V (x 0 ), such that, for any u(•) ∈ U(y 0 ), the solution x[x 0 , u(•), β(u(•))] remains in D until it reaches E (or remains in D on [0, +∞) if it never reaches E).