数学物理学报, 2026, 46(5): 1800-1824

一类混合阶非线性椭圆系统的变号解与多重解

王启红,, 钟延生,*

福建师范大学数学与统计学院 福州 350117

The Sign-Changing and Multiple Solutions for Mixed Order Nonlinear Elliptic System

Wang Qihong,, Zhong Yansheng,*

School of Mathematics and Statistics, Fujian Normal University, Fuzhou 350117

通讯作者: * 钟延生, E-mail:zys08@fjnu.edu.cn

收稿日期: 2024-09-29   修回日期: 2025-10-21  

基金资助: 国家自然科学基金(11671085)
福建省自然科学基金(2024J01479)

Received: 2024-09-29   Revised: 2025-10-21  

Fund supported: NSFC(11671085)
Science foundation of Fujian province(2024J01479)

作者简介 About authors

王启红,E-mail:18308301427@163.com

摘要

该文研究了以下混合阶非线性椭圆系统的变号解与多重解的存在性

$\begin{equation} \left\{ \begin{aligned} {(-\Delta)}^s{u}&=|u|^{3s-1}u+\beta v^2|u|^{\alpha-1}u, && x\in \Omega,\\ -\Delta {v}&=|v|^{2}v+\frac{2\beta}{\alpha+1} |u|^{\alpha }uv, && x\in \Omega,\\ u=v&=0, && x\in\partial\Omega,\nonumber \end{aligned} \right. \end{equation}$

其中 $\Omega\subset\mathbb{R}^{n}(2\leq n\leq3)$ 是一个光滑的有界域, 参数 $ \beta<0,\ \alpha>0$. $(-\Delta)^{s}$ 是分数阶 Laplacian 算子, $ -\Delta $ 是 Laplacian 算子. 设 $s\in[\frac{3}{4},1)$, $\alpha\in(0,\frac{6s-n}{n-2s})$, 则以上系统存在正解、负解和变号解. 进而, 若 $\alpha=\frac{n_{1}}{m_{1}},s=\frac{n_{2}}{m_{2}}$, 其中 $m_{1},m_{2},n_{1},n_{2}$ 是奇数, 则以上系统存在一个无界的变号解序列 $ \{(u_m,v_m)\},\ m\in\mathbb{N}$ 且 $ u_m,\ v_m$ 最多有 $ m+1$ 个变号域.

关键词： 混合阶; 分数阶; 非线性椭圆系统; 变号解

Abstract

In this article, we study the existence of sign-changing and multiple solutions for the following mixed order nonlinear elliptic system

$\begin{equation} \left\{ \begin{aligned} {(-\Delta)}^s{u}&=|u|^{3s-1}u+\beta v^2|u|^{\alpha-1}u, && {\rm in}\ \Omega,\\ -\Delta {v}&=|v|^{2}v+\frac{2\beta}{\alpha+1} |u|^{\alpha }uv, && {\rm in}\ \Omega,\\ u=v&=0, && {\rm on}\ \partial\Omega,\nonumber \end{aligned} \right. \end{equation}$

where $\Omega\subset \mathbb{R}^n(2\leq n \leq3 )$ is a smooth bounded domain, the parameters $ \beta<0,\ \alpha>0$. $(-\Delta )^s$ is a fractional Laplacian operator, and $-\Delta $ is a Laplacian operator. If $s\in[\frac{3}{4},\ 1)$ and $\alpha\in(0,\frac{6s-n}{n-2s})$, we prove the existence of positive, negative, and sign-changing solutions for the above system. Moreover, if $\alpha=\frac{n_{1}}{m_{1}}, s=\frac{n_{2}}{m_{2}}, m_{1}, n_{1}, m_{2}, n_{2}$ are odd integers, then there exists an unbounded sequence of sign-changing solutions $\{(u_m,v_m)\}, m\in\mathbb{N}$, and $u_m, v_m$ both have at most $m+1$ sign-changing domains.

Keywords： mixed order; fractional Laplacian; nonlinear elliptic system; sign-changing solution

PDF (694KB) 元数据 多维度评价 相关文章 导出 EndNote| Ris| Bibtex  收藏本文

本文引用格式

王启红, 钟延生. 一类混合阶非线性椭圆系统的变号解与多重解[J]. 数学物理学报, 2026, 46(5): 1800-1824

Wang Qihong, Zhong Yansheng. The Sign-Changing and Multiple Solutions for Mixed Order Nonlinear Elliptic System[J]. Acta Mathematica Scientia, 2026, 46(5): 1800-1824

1 引言

本文主要考虑以下混合阶非线性椭圆系统

$\begin{equation}\label{1.1} \left\{ \begin{aligned} {(-\Delta)}^s{u}&=|u|^{3s-1}u+\beta v^2|u|^{\alpha-1}u, && x\in \Omega,\\ -\Delta {v}&=|v|^{2}v+\frac{2\beta}{\alpha+1} |u|^{\alpha }uv, && x\in \Omega,\\ u=v&=0, && x\in\partial\Omega, \end{aligned} \right. \end{equation}$

其中$\Omega\subset\mathbb{R}^{n}(2\leq n\leq3)$ 是一个有界域且有光滑的边界$\partial\Omega$. $ -\Delta $ 是 Laplacian 算子, $(-\Delta)^{s}$ 是分数阶Laplacian 算子, 定义为

$\begin{equation} \begin{aligned} (-\Delta)^{s}u(x)=C_{n,2s}P.V.\int_{\mathbb{R}^{n}}\frac{u(x)-u(z)}{|x-z|^{n+2s}} {\rm d}z,\nonumber \end{aligned} \end{equation}$

其中$s\in[\frac{3}{4},1),\ P.V.$ 表示Cauchy 主值积分且

$\begin{equation} \begin{aligned} C_{n,2s}=\frac{4^{s}\Gamma(\frac{n+2s}{2})}{\pi^{\frac{n}{2}}\Gamma(-s)}.\nonumber \end{aligned} \end{equation}$

近年来, 关于分数阶Laplacian方程的研究取得了许多成就. 在文献 [1–7]中, Caffarelli 等研究了分数阶Laplacian 的自由边界问题, 包括正则性、非退化性. 特别地, Caffarelli 和Silvestre 在文献[8]中应用$s$-调和延拓的方法定义了分数阶Laplacian. 进而 Kwa$\acute{\rm s}$nicki 等在文献[9,10]中利用分数阶Laplacian 算子$s$-调和延拓的表示得到更广泛的应用.

同时, 关于Schrödinger 系统解的性质研究引起了学者们的广泛关注. Lin-Wei 在文献[11] 中研究了以下非线性奇异摄动Schrödinger 系统最小能量解的存在性和渐近行为

$\begin{equation} \left\{ \begin{aligned} \varepsilon^{2}\Delta u-\lambda_{1}u+\mu_{1}u^{3}+\beta uv^{2}&=0,&&x\in\Omega,\\ \varepsilon^{2}\Delta v-\lambda_{2}v+\mu_{2}v^{3}+\beta u^{2}v &=0,&&x\in\Omega,\\ u,v&>0, &&x\in\Omega,\\ u=v&=0,&&x\in\partial\Omega, \nonumber \end{aligned} \right. \end{equation}$

其中 $\Omega\subset \mathbb{R}^{n}(n\leq3)$ 是一个光滑有界域, 参数$\varepsilon,\ \lambda_{1},\ \lambda_{2},\ \mu_{1},\ \mu_{2}>0,$$\beta\neq0$ 是一个耦合常数. Bartsch-Wang 等[12]研究了以下Dirichlet 问题的Morse 指标和变号解

$\begin{equation} \left\{ \begin{aligned} -\Delta u&=f(u),&&x\in\Omega,\\ u&=0,&&x\in\partial\Omega, \nonumber \end{aligned} \right. \end{equation}$

其中 $\Omega\subset\mathbb{R}^{N}$ 是一个 Lipschitz 有界域. 此外, Zhang-Xu 在文献[13] 中研究了以下Schrödinger-Poisson 系统的极小解的存在性

$\begin{equation} \left\{ \begin{aligned} & -\Delta u+V(x)u+\phi u=f(x,u),\ && x\in\mathbb{ R}^{3},\\ &-\Delta\phi=u^{2},\quad&& \lim_{|x|\rightarrow\infty}\phi(x)=0. \nonumber \end{aligned} \right. \end{equation}$

更多的椭圆系统和Schrödinger-Poisson 系统正解和变号解的存在性参见文献 [14-18]. 而对于分数阶的情况, 在文献 [19-22]中, Chen 等研究了分数阶Schrödinger-Poisson 系统在不同条件下的正解、变号解和非平凡解的存在性. 相应的下降流不变集的方法可参见文献 [23-25].

由方程 (1.1), 记

$\begin{equation}\label{defb1.1} \left\{ \begin{aligned} f_{1}(u,v)&=|u|^{3s-1}u+\beta v^2|u|^{\alpha-1}u,\\ f_{2}(u,v)&=|v|^{2}v+\frac{2\beta}{\alpha+1} |u|^{\alpha }uv. \end{aligned} \right. \end{equation} $

令$\alpha=\frac{n_{1}}{m_{1}}, s=\frac{n_{2}}{m_{2}}$, 其中$m_{1},m_{2},n_{1},n_{2}$ 是奇数, 以及$s\in[\frac{3}{4},1),\alpha\in(0,\frac{6s-n}{n-2s})(2\leq n\leq3)$. 显然, $f_{1}$ 和$f_{2}$ 满足以下性质.

${\bf(I)}$$f_{i}\in C(\mathbb{R}\times \mathbb{R}, \mathbb{R})$, 以及$f_{i}(0,0)=0(i=1,2)$;

$ {\bf(II)}$ 存在$C>0$ 和$p\in[2,\frac{2n}{n-2s})$, 对于任意$\bar{\omega},\bar{\nu}\in \mathbb{R}$, 有

$\begin{equation} \begin{aligned} |f_{i}(\bar{\omega},\bar{\nu})|\leq C(1+|\bar{\omega}|^{p-1}+|\bar{\nu}|^{p-1}), i=1,2,\nonumber \end{aligned} \end{equation}$

且当$\bar{\omega},\bar{\nu}\rightarrow0$ 时, $f_{i}(\bar{\omega},\bar{\nu})\rightarrow0(i=1,2)$;

${\bf(III)}$ 存在$M>0$ 和$\mu>2$, 使得

$\begin{equation} \begin{aligned} 0\leq F(\bar{\omega},\bar{\nu})\leq\frac{\bar{\omega}}{\mu}f_{1}(\bar{\omega},\bar{\nu})+\frac{\bar{ \nu}}{\mu}f_{2}(\bar{\omega},\bar{\nu}), \forall |\bar{\omega}|,|\bar{\nu}|\geq M,\nonumber \end{aligned} \end{equation}$

其中

$\begin{equation} \begin{aligned} F(\bar{\omega},\bar{\nu})&=\int_{0}^{\bar{\omega}}f_{1}(\gamma,\bar{\nu}){\rm d}\gamma+\frac{1}{4}|\bar{\nu}|^{4}=\int_{0}^{\bar{\nu}}f_{2}(\bar{\omega},\gamma){\rm d}\gamma+\frac{1}{3s+1}|\bar{\omega}|^{3s+1}\\ &=\int_{0}^{\bar{\omega}}f_{1}(\gamma,\bar{\nu}){\rm d}\gamma+\int_{0}^{\bar{\nu}}f_{2}(\bar{\omega},\gamma){\rm d}\gamma-\frac{\beta}{\alpha+1}|\bar{\nu}|^{2}|\bar{\omega}|^{\alpha+1} ;\nonumber \end{aligned} \end{equation}$

${\bf(IV)}$$f_{1}(-\bar{\omega},\bar{\nu})=-f_{1}(\bar{\omega},\bar{\nu}),f_{2}(\bar{\omega},-\bar{\nu})=-f_{2}(\bar{\omega},\bar{\nu}),\forall \bar{\omega},\bar{\nu}\in \mathbb{R}$;

${\bf(V)} $$\forall \bar{\omega},\bar{\nu}\in \mathbb{R},\frac{f_{1}(\bar{\omega}, \bar{\nu})}{|\bar{\omega}|}$ 和$\frac{f_{2}(\bar{\omega}, \bar{\nu})}{|\bar{\nu}|}$ 是不减的.

结合上面性质, 对于Laplacian 算子$-\Delta$, 结合$H_{0}^{1}(\Omega)$ 和$H^{-1}(\Omega)$ 之间的同构, 而对于非局部算子$(-\Delta)^{s}$, 应用$s$-调和延拓的方法将非局部问题由$\Omega$ 转换到$\mathcal{C}\triangleq \Omega\times(0,+\infty)$ 上的局部问题. 由此, 在空间$(H_{0}^{s}(\Omega)\cap H_{0,L}^{s}(\mathcal{C}))\times H_{0}^{1}(\Omega)$ 中通过构造下降流不变集, 讨论混合阶椭圆系统 (1.1) 解的存在性, 进而应用亏格和相对亏格 [25,26]得到解的多重性. 本文的主要结果如下

定理 1.1 假设$s\in[\frac{3}{4},1),\alpha\in(0,\frac{6s-n}{n-2s})(2\leq n\leq 3)$, 则系统 (1.1) 至少有三个非平凡解, 即正解、负解和变号解且变号解只含两个变号域.

定理 1.2 基于定理1.1的条件, 若$\alpha=\frac{n_{1}}{m_{1}},s=\frac{n_{2}}{m_{2}}$, 其中$m_{1},m_{2},n_{1},n_{2}$ 是奇数, 则系统 (1.1) 有一列无界的变号解序列且变号解至多有$m+1$ 个变号域.

本文的结构如下: 在第 2 节中将给出一些准备工作. 第3 节介绍系统 (1.1) 的解的一些性质. 第4 节将给出算子$((-\Delta)^{s})^{-1}$ 和$(-\Delta)^{-1}$ 的性质. 第5 节应用了下降流不变集给出了定理1.1的证明. 最后, 第 6 节中应用亏格和相对亏格证明了定理1.2.

在本文中, $L^{p}(\Omega)(1<p\leq\infty)$ 与 $C^{k}(k\in \mathbb{N})$ 的范数分别定义为$\|\cdot\|_{p}$ 与 $\|\cdot\|_{C^{k}}$; $C_{0}(\bar{\Omega})$ 是$\bar{\Omega}$ 中的连续实值函数空间且在边界$\partial\Omega$ 上取值为零.

2 准备工作

记

$\begin{equation} \begin{aligned} \mathbb{R}_{+}^{n+1}\triangleq\{(x,y):x\in\mathbb{R}^{n},y>0\},\nonumber \end{aligned} \end{equation}$

$\mathcal{C}\triangleq\Omega\times(0,+\infty)$ 表示光滑有界域$\Omega\subset \mathbb{R}^{n}(2\leq n\leq3)$ 的上半圆柱, 其侧面边界为$\partial_{L}\mathcal{C}\triangleq\partial\Omega\times[0,+\infty)$.

由 $-\Delta$ 算子在$\Omega$ 中满足零 Dirichlet 边界条件, 可取$L^{2}(\Omega)$ 中的标准正交基 $\{\varphi_{k}\}$, $\lambda_{k}^{s}$ 和$\lambda_{k}$ 分别是$(-\Delta)^{s}$ 和$-\Delta$ 的特征值, 即有

$\begin{equation} \left\{ \begin{aligned} (-\Delta)^{s}\varphi_{k}&=\lambda_{k}^{s}\varphi_{k},&&x\in\Omega,\\ \varphi_{k}&=0,&&x\in\partial\Omega\nonumber \end{aligned} \right. \end{equation}$

和

$\begin{equation} \left\{ \begin{aligned} -\Delta\varphi_{k}&=\lambda_{k}\varphi_{k},&&x\in\Omega,\\ \varphi_{k}&=0,&&x\in\partial\Omega.\nonumber \end{aligned} \right. \end{equation}$

令

$\begin{equation} \begin{aligned} H_{0}^{s}(\Omega)&=\{u=\sum_{k=0}^{\infty}a_{k}\varphi_{k}\in L^{2}(\Omega):\|u\|_{H_{0}^{s}(\Omega)}=(\sum_{k=0}^{\infty}a_{k}^{2}\lambda_{k}^{s})^{\frac{1}{2}}<\infty\},\\ H_{0}^{1}(\Omega)&=\{v=\sum_{k=0}^{\infty}b_{k}\varphi_{k}\in L^{2}(\Omega):\|v\|_{H_{0}^{1}(\Omega)}=(\sum_{k=0}^{\infty}b_{k}^{2}\lambda_{k})^{\frac{1}{2}}<\infty\}.\nonumber \end{aligned} \end{equation}$

$H^{-s}(\Omega)$ 是 $H^{s}(\Omega)$ 的对偶空间, $H^{-1}(\Omega)$ 是$H^{1}(\Omega)$ 的对偶空间. 对于$u\in H^{s}_{0}(\Omega),$ 有$u=\sum\limits_{k=0}^{\infty}a_{k}\varphi_{k}$, 其中$a_{k}=\int _{\Omega}u \varphi_{k} {\rm d}x$. 同样地, 对于$v\in H^{1}_{0}(\Omega),$ 有$v=\sum\limits_{k=0}^{\infty}b_{k}\varphi_{k},$ 其中$ b_{k}=\int _{\Omega}v\varphi_{k} {\rm d}x.$ 此时, 在Dirichlet 条件下算子$(-\Delta)^{s}$ 和$-\Delta$ 也可定义为

$\begin{equation} \begin{aligned} (-\Delta)^{s}u=\sum_{k=0}^{\infty}a_{k}\lambda_{k}^{s}\varphi_{k} \in H^{-s}(\Omega),\ -\Delta v=\sum_{k=0}^{\infty}b_{k}\lambda_{k}\varphi_{k}\in H^{-1}(\Omega).\nonumber \end{aligned} \end{equation}$

同时, 定义$H_{0}^{s}(\Omega)$ 和$H_{0}^{1}(\Omega)$ 的内积为

$\begin{equation} \begin{aligned} (u_{1},u_{2})_{H_{0}^{s}(\Omega)}&=\int_{\Omega}(-\Delta)^{\frac{s}{2}}u_{1}(-\Delta)^{\frac{s}{2}}u_{2} {\rm d}x{\rm d}y,\\ (v_{1},v_{2})_{H_{0}^{1}(\Omega)}&=\int_{\Omega} \nabla v_{1}\cdot\nabla v_{2} {\rm d}x{\rm d}y. \nonumber \end{aligned} \end{equation}$

且系统 (1.1) 对应的能量泛函定义为 $E(u,v):\ H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega) \rightarrow\mathbb{R}$

$\begin{equation} \begin{aligned} E(u,v)=&\frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}u|^{2}\!+\!|\nabla v|^{2} {\rm d}x\!-\!\frac{1}{3s+1}\int_{\Omega}|u|^{3s+1}{\rm d}x\!-\!\frac{1}{4}\int_{\Omega}|v|^{4} {\rm d}x \!-\!\frac{\beta}{\alpha+1}\int_{\Omega}|v|^{2} |u|^{\alpha+1}{\rm d}x.\nonumber \end{aligned} \end{equation}$

记$E_{1}'(u,v)=\partial_{u}E(u,v),\ E_{2}'(u,v)=\partial_{v}E(u,v).$

定义 2.1 如果对于任意的$ \phi_{1}\in H_{0}^{s}(\Omega),\ \phi_{2}\in H_{0}^{1}(\Omega)$, 满足

$\begin{equation} \begin{aligned} \int_{\Omega}(-\Delta)^{\frac{s}{2}}u(-\Delta)^{\frac{s}{2}}\phi_{1}{\rm d}x&=\int_{\Omega}f_{1}(u,v)\phi_{1}{\rm d}x,\\ \int_{\Omega} \nabla v\cdot\nabla \phi_{2}{\rm d}x&=\int_{\Omega}f_{2}(u,v)\phi_{2}{\rm d}x,\nonumber \end{aligned} \end{equation}$

则称$(u,v)\in H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$ 是系统 (1.1) 的一个弱解.

根据 (I), (II) 和Sobolev 嵌入$H_{0}^{s}(\Omega)\hookrightarrow L^{\frac{2n}{n-2s}}(\Omega),\ H_{0}^{1}(\Omega)\hookrightarrow L^{\frac{2n}{n-2 }}(\Omega)$, 则$f_{1} ( u,v)\in L^{\frac{2n}{n+2s}}(\Omega),\ f_{2} (u,v)\in L^{\frac{2n}{n+2 }}(\Omega) $ 和$\phi_{1}\in L^{\frac{2n}{n-2s}}(\Omega),\phi_{2}\in L^{\frac{2n}{n-2}}(\Omega)$. 显然, $E\in C^{1}(H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega),\ \mathbb{R})$ 且$E(u,v)$ 的临界点是系统 (1.1) 的弱解.

对于非局部算子${(-\Delta)}^s$, 首先通过$s$-调和延拓和算子${(-\Delta)}^s$ 关于谱分解的等价定义, 将非局部问题转化为局部问题, 具体参见文献[8,27]. 对于正则函数$u$, 定义它到$\mathcal{C} $ 的$s$-调和延拓为$\omega=E_{s}(u)$. 同样定义$\xi=E_{1}(v)$, $E_{1}(\cdot)$ 是从$H_{0}^{1}(\Omega)$ 到$H^{-1}(\Omega)$ 的同构映射. 由于$H_{0}^{s}(\Omega)$ 紧嵌入到$L^{r_{1}},\ r_{1}\in[2,\frac{2n}{n-2s})$; $H_{0}^{1}(\Omega)$ 紧嵌入到$L^{r_{2}},\ r_{2}\in[2,\frac{2n}{n-2})$. 则方程 (1.1) 可转化为

$\begin{equation}\label{2.1} \left\{ \begin{aligned} -\mathrm{div}(y^{1-2s}\nabla \omega)&=0,\ && (x,y)\in\mathcal{C},\\ \omega&=0,&& (x,y)\in \partial_{L}\mathcal{C},\\ \omega( x,0 )&=u,&& \quad\ \, x\in \Omega,\\ E_{1}^{-1}(\xi)&=v,&&\quad\ \, x\in \Omega. \end{aligned} \right. \end{equation} $

而在文献[8] 中, 可以通过Dirichlet 到Neumann 的映射来定义分数阶Laplacian 算子.

定义 2.2 对于任意正则函数$u\in H_{0}^{s}(\Omega)$, 有

$\begin{equation} \begin{aligned} (-\Delta)^{s}u(x)&=-\frac{1}{k_{s}}\underset{y\rightarrow0^{+}}{\lim}y^{1-2s}\frac{\partial\omega}{\partial y}(x,y),&&\forall x\in\Omega,\nonumber \end{aligned} \end{equation}$

其中$E_{s}(u)=\omega,\ k_{s}=\frac{2^{1-2s}\Gamma(1-s)}{\Gamma(s)}.$ 且$H_{0,L}^{s}(\mathcal{C})$ 和$ H^{-1}(\Omega)$ 的范数定义为

$\begin{equation} \begin{aligned} \|\omega\|_{H_{0,L}^{s}(\mathcal{C})}=(k_{s}\int_{\Omega}y^{1-2s}|\nabla\omega|^{2} {\rm d}x{\rm d}y)^{\frac{1}{2}},\quad\|\xi\|_{H^{-1}(\Omega)}=\underset{\|v\|_{H_{0}^{1}(\Omega)}\leq1}{\sup} (\xi,v ).\nonumber \end{aligned} \end{equation}$

引理 2.1 以下性质成立

(i)

$\begin{equation} \begin{aligned} \|(-\Delta)^{s}u\|_{H^{-s}(\Omega)}&=\|u\|_{H_{0}^{s}(\Omega)}=\|E_{s}(u)\|_{H_{0,L}^{s}(\mathcal{C})},\\ \| -\Delta v\|_{H^{-1}(\Omega)}&=\|v\|_{H_{0}^{1}(\Omega)}=\|E_{1}(v)\|_{H^{-1}(\Omega)};\nonumber \end{aligned} \end{equation}$

(ii) 对于任意的$ \omega\in H_{0,L}^{s}(\mathcal{C}),\ \xi\in H^{-1}(\Omega)$, 存在一个独立于$\omega$ 和$\xi$ 的常数$C$, 使得

$\begin{equation} \begin{aligned} &\|tr_{\Omega}\omega\|_{L^{r_{1}}(\Omega)}\leq C\|\omega\|_{H_{0,L}^{s}(\mathcal{C})},\ \|E_{1}^{-1}(\xi)\|_{L^{r_{2}}(\Omega)}\leq C\|\xi\|_{H^{-1}(\Omega)},\\ & \|(u,v)\|_{L^{r_{1}}(\Omega)\times L^{r_{2}}(\Omega)}\leq C\|(\omega,\xi)\|_{H_{0,L}^{s}(\mathcal{C})\times H^{-1}(\Omega) },\nonumber \end{aligned} \end{equation}$

其中$r_{1}\in[2,\frac{2n}{n-2s}),\ r_{2}\in[2,\frac{2n}{n-2 }),\enspace H_{0,L}^{s}(\mathcal{C})$, $H^{-1}(\Omega)$ 分别紧嵌入到$L^{r_{1}}(\Omega)$ 和 $L^{r_{2}}(\Omega)$ 中.

因此, 问题 (1.1) 可转化为以下局部问题

$\begin{equation}\label{2.2} \left\{ \begin{aligned} -\mathrm{div}(y^{1-2s}\nabla \omega)&=0,&& (x,y)\in\mathcal{C},\\ \partial_{\nu}^{s}\omega&=f_{1}(u,v),&& \quad\ \, x \in\Omega,\\ -\Delta v&=f_{2}(u,v),&& \quad\ \, x\in \Omega,\\ \omega&=0,&& (x,y)\in \partial_{L}\mathcal{C}, \end{aligned} \right. \end{equation}$

其中$\partial_{\nu}^{s}\omega\triangleq-\frac{1}{k_{s}}\underset{y\rightarrow0^{+}}{\lim}y^{1-2s}\frac{\partial\omega}{\partial y}(x,y).$ 不失一般性, 全文假设 $k_{s}=1$.

定义 2.3 记$u=tr_{\Omega}\omega,\ v=E_{1}^{-1}(\xi),\ \omega\in H_{0,L}^{s}(\mathcal{C}),\ \xi\in H^{-1}(\Omega),$ 取 $\varphi_{1}\in H_{0,L}^{s}(\mathcal{C}),\ \varphi_{2}\in H^{-1}(\Omega), $ 若有

$\begin{equation} \begin{aligned} \int_{\mathcal{C}}y^{1-2s}(\nabla\omega,\nabla\varphi_{1}) {\rm d}x{\rm d}y&=\int_{\Omega}f_{1}( tr_{\Omega}\omega,v)tr_{\Omega}\varphi_{1}{\rm d}x,\\ \int_{\Omega} \nabla \xi\cdot \nabla \varphi_{2}{\rm d}x&=\int_{\Omega}f_{2}( u,E_{1}^{-1}(\xi))E_{1}^{-1}(\varphi_{2}){\rm d}x, \nonumber \end{aligned} \end{equation}$

则称$(u,v)\ (u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega))$ 是系统 (1.1) 的一个能量解.

同样地, 定义能量泛函$J(\omega,\xi):\ H_{0,L}^{s}(\mathcal{C})\times H^{-1}(\Omega)\rightarrow\mathbb{R} $

$\begin{equation} \begin{aligned} J(\omega,\xi)=&\frac{1}{2}\int_{\mathcal{C}}y^{1-2s}|\nabla\omega|^{2} {\rm d}x{\rm d}y+\frac{1}{2}\int_{\Omega}| \nabla \xi|^{2}{\rm d}x-\frac{1}{3s+1}\int_{\Omega}|u|^{3s+1}{\rm d}x-\frac{1}{4}\int_{\Omega}|v|^{4} {\rm d}x\\ &-\frac{ \beta}{\alpha+1}\int_{\Omega}|v|^{2}|u|^{\alpha+1}{\rm d}x.\nonumber \end{aligned} \end{equation}$

由引理 2.1(ii), 条件 (I), (II) 和文献[26], 可得$J\in C^{1}(H_{0,L}^{s}(\mathcal{C})\times H^{-1}(\Omega),\ \mathbb{R}).$ 若$(\omega,\xi)\ (\omega\in H_{0,L}^{s}(\mathcal{C}),\ \xi\in H^{{-1}}(\Omega)$ 是$J$ 的一个临界点, 则$(u,v)=(tr_{\Omega}\omega, E_{1}^{-1}(\xi))\in H_{0}^{s}\times \in H_{0}^{1}(\Omega))$ 是$E$ 的一个临界点, 也是系统 (1.1) 的一个弱解. 反之也成立.

引理 2.2 设条件(I)-(III) 成立, 则$E(u,v)$ 满足$\mathrm{(PS)}$ 条件.

证 记$\mathrm{(PS)}$ 序列$\{(u_{k},v_{k})\}\subset H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega),$ 即当$k\rightarrow\infty $ 时有

$\begin{equation} \begin{aligned} E(u_{k},v_{k})\rightarrow c\in\mathbb{R},\ E_{1}'(u_{k},v_{k})\rightarrow0,\ E_{2}'(u_{k},v_{k})\rightarrow0.\nonumber \end{aligned} \end{equation}$

根据条件(III), 可得

$\begin{equation} \begin{aligned} &\mu c+o(1)(1+\|u_{k}\|_{H_{0}^{s}(\Omega)}+\|v_{k}\|_{H_{0}^{1}(\Omega)})\\ =&\mu E(u_{k},v_{k})-(E_{1}'(u_{k},v_{k}),u_{k})_{H^{-s}(\Omega),{H_{0}^{s}(\Omega)}}-(E_{2}'(u_{k},v_{k}),v_{k})_{H^{-1}(\Omega),{H_{0}^{1}(\Omega)}}\\ =&(\frac{\mu}{2}-1)\|u_{k}\|^{2}_{H_{0}^{s}(\Omega)}+(\frac{\mu}{2}-1)\|v_{k}\|^{2}_{H_{0}^{1}(\Omega)}\\ &-\int_{\Omega}(\mu F( u_{k},v_{k}) -u_{k}f_{1}( u_{k},v_{k})-v_{k}f_{2}( u_{k},v_{k})){\rm d}x.\\ \geq&(\frac{\mu}{2}-1)(\|u_{k}\|^{2}_{H_{0}^{s}(\Omega)}+ \|v_{k}\|^{2}_{H_{0}^{1}(\Omega)}). \nonumber \end{aligned} \end{equation}$

因此, $\{{u_{k}}\}\subset H_{0}^{s}(\Omega)$, $\{{v_{k}}\}\subset H_{0}^{1}(\Omega)$ 是有界的, 故存在$u_{0}\in H_{0}^{s}(\Omega),\ v_{0}\in H_{0}^{1}(\Omega),$ 使得$ u_{k}\rightharpoonup u_{0},\ v_{k}\rightharpoonup v_{0}.$ 由引理 2.1(ii), 在$L^{r_{1}}(\Omega)$ 中有$u_{k}\rightarrow u_{0}$, 在$L^{r_{2}}(\Omega)$ 中有$v_{k}\rightarrow v_{0}$, 其中 $r_{1}\in [2,\frac{2n}{n-2s})$, $r_{2}\in [2,\frac{2n}{n-2 })$, 即对于 a.e. $x\in\Omega$, 当$k\rightarrow\infty $ 时有

$\begin{equation} \begin{aligned} u_{k}(x)\rightarrow u_{0}(x),~v_{k}(x)\rightarrow v_{0}(x),\nonumber \end{aligned} \end{equation}$

以及

$\begin{equation} \begin{aligned} (E_{1}'(u_{k},v_{k}),u_{k}-u_{0})\rightarrow0,~(E_{2}'(u_{k},v_{k}),v_{k}-v_{0})\rightarrow 0.\nonumber \end{aligned} \end{equation}$

由条件(II), 当$ k\rightarrow\infty$ 时, 可得

$\begin{equation} \begin{aligned} &\|u_{k}-u_{0}\|^{2}_{H_{0}^{s}(\Omega)}\\ =&\int_{\Omega}(-\Delta)^{\frac{s}{2}}u_{k}(-\Delta)^{\frac{s}{2}}(u_{k}-u_{0}){\rm d}x-\int_{\Omega}(-\Delta)^{\frac{s}{2}}u_{0}(-\Delta)^{\frac{s}{2}}(u_{k}-u_{0}){\rm d}x\\ =&\int_{\Omega}f_{1}( u_{k},v_{k})(u_{k}-u_{0}){\rm d}x+o(1)\\ \leq& C\int_{\Omega}|u_{k}-u_{0}|{\rm d}x+C\int_{\Omega}|u_{k}|^{p-1}|u_{k}-u_{0}|{\rm d}x+C\int_{\Omega}|v_{k}|^{p-1}|u_{k}-u_{0}|{\rm d}x+o(1)\\ \leq& C|\Omega|^{\frac{1}{2}}\|u_{k}-u_{0}\|_{2}+C\|u_{k}\|_{p}^{p-1}\|u_{k}-u_{0}\|_{p}+C\|v_{k}\|_{p}^{p-1}\|u_{k}-u_{0}\|_{p}+o(1)\\ \rightarrow &0, \nonumber \end{aligned} \end{equation}$
$\begin{equation} \begin{aligned} &\|v_{k}-v_{0}\|^{2}_{H_{0}^{1}(\Omega)}\\ =&\int_{\Omega} \nabla v_{k}\cdot\nabla(v_{k}-v_{0}){\rm d}x-\int_{\Omega}\nabla v_{0}\cdot\nabla(v_{k}-v_{0}){\rm d}x\\ =&\int_{\Omega}f_{2}( u_{k},v_{k})(v_{k}-v_{0}){\rm d}x+o(1)\\ \leq &C\int_{\Omega}|v_{k}-v_{0}|{\rm d}x+C\int_{\Omega}|u_{k}|^{p-1}|v_{k}-v_{0}|{\rm d}x+C\int_{\Omega}|v_{k}|^{p-1}|v_{k}-v_{0}|{\rm d}x+o(1)\\ \leq &C|\Omega|^{\frac{1}{2}}\|v_{k}-v_{0}\|_{2}+C\|u_{k}\|_{p}^{p-1}\|v_{k}-v_{0}\|_{p}+C\|v_{k}\|_{p}^{p-1}\|v_{k}-v_{0}\|_{p}+o(1)\\ \rightarrow& 0.\nonumber \end{aligned} \end{equation}$

故存在$u_{0}\in H_{0}^{s}(\Omega),\ v_{0}\in H_{0}^{1}(\Omega)$, 使得$u_{k}\rightarrow u_{0},\ v_{k}\rightarrow v_{0}$ 成立.

设$\Omega$ 是$\mathbb{R}^{n}$$( 2\leq n\leq3)$ 中的一个光滑有界域, 记$v^{+}= \max\{v,0\}$, 有

引理 2.3(强极大值原理)[28,29] 设$c(x)\in L^{\infty}(\Omega).$ 如果$\omega\in C^{2}(\bar{\mathcal{C}})\cap C_{0}(\bar{\mathcal{C}})$, 满足

$\begin{equation}\label{2.3} \left\{ \begin{aligned} -\mathrm{div}(y^{1-2s}\nabla \omega)&=0,&& (x,y)\in\mathcal{C},\\ \omega&\geq0,&& (x,y)\in\mathcal{C},\\ \omega&=0,&& (x,y)\in \partial_{L}\mathcal{C},\\ \partial_{\nu}^{s}\omega+c(x)\omega&\geq0,&& \quad\ \, x\in \Omega,\\ -\Delta v-\frac{2\beta}{\alpha+1}|u|^{\alpha }uv&=(v^{+})^{3}, \ && \quad\ \, x\in \Omega,\\ v &= 0,&& \quad\ \, x\in\partial\Omega. \end{aligned} \right. \end{equation} $

则在$\mathcal{C}$ 中 $\omega>0$, 或$\omega\equiv0$. 特别地, 在$\Omega$ 中$u>0$, 或$u\equiv0$; 以及$v>0$, 或$v\equiv0$.

为了估计变号域的个数, 需以下引理, 其证明过程可参见文献[30,31].

引理 2.4 设$\Omega\subset\mathbb{R}^{n}$$( 2\leq n\leq3 )$ 是一个有界开集, 如果$u \in C(\bar{\Omega})\cap H^{s}(\Omega),\ v \in C(\bar{\Omega})\cap H^{1}(\Omega)$, 以及在边界$\partial\Omega$ 上$u=v=0$, 则$u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega).$

推论 2.1 设$\Omega\subset\mathbb{R}^{n}$$(2\leq n\leq3 )$ 是一个有界开集, 以及$u\in C(\bar{\Omega})\cap H_{0}^{s}(\Omega),\ v\in C(\bar{\Omega})\cap H_{0}^{1}(\Omega)$. 取 $\Omega$ 的一个子集$\Omega_{1}$, 使得对于$x\in\partial\Omega_{1}\cap\Omega$, 有$u(x)=v(x)=0$ 且$u$ 限制在$\Omega_{1}$ 上为$u_{\Omega_{1}}\in H_{0}^{s}(\Omega_1)$; 同样地, $v$ 限制在$\Omega_{1}$ 为$v_{\Omega_{1}}\in H_{0}^{1}(\Omega_1)$, 定义

$\begin{equation} z_{1}(x) = \begin{cases} u(x) & x \in \Omega,\\ 0 & x \in \Omega\setminus\Omega_{1} \nonumber \end{cases} \end{equation}$

和

$\begin{equation} z_{2}(x) = \begin{cases} v(x) & x\in\Omega,\\ 0 & x\in \Omega\setminus\Omega_{1},\nonumber \end{cases} \end{equation}$

则$z_{1}(x)\in H_{0}^{s}(\Omega),\ z_{2}(x)\in H_{0}^{1}(\Omega).$

证 易得 $u_{\Omega_{1}}$, $v_{\Omega_{1}}$ 在$\bar{\Omega}_{1}$ 上是连续的以及$u_{\Omega_{1}}$ 和$v_{\Omega_{1}}$ 在$\partial\Omega_{1}\cap\Omega$ 上都为零. 根据$u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega)$ 以及$\Omega$ 的有界性, 可得$u_{\Omega_{1}}\in H^{s}(\Omega),\ v_{\Omega_{1}}\in H^{1}(\Omega)$. 结合引理 2.4, 可得$u_{\Omega_{1}}\in H_{0}^{s}(\Omega_{1}),\ v_{\Omega_{1}}\in H_{0}^{1}(\Omega_{1})$. 此外, $z_{1}\in C(\bar{\Omega})\cap H^{s}(\Omega)$, $ z_{2}\in C(\bar{\Omega})\cap H^{1}(\Omega)$ 以及对于$x\in\partial\Omega$ 有$z_{1}(x)=z_{2}(x)=0$. 再由引理 2.4 可得$z_{1}(x)\in H_{0}^{s}(\Omega),\ z_{2}(x)\in H_{0}^{1}(\Omega).$

3 解的正则性

在本节中, 将讨论系统(1.1) 弱解的正则性.

定理 3.1 设条件(I)-(V) 成立, 若$(u,v)\in H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$ 是系统(1.1) 的一个弱解, 则对于$\sigma\in(0,1)$, 有$u,\ v\in C^{\sigma}(\bar{\Omega})\cap C_{0}(\bar{\Omega}) $. 此外, 若$f_{1}$ 和$ f_{2}$ 都是$C^{1,\sigma}$ 函数, 则$u,\ v\in C^{2,\sigma}(\bar{\Omega})\cap C_{0}(\bar{\Omega})$.

为了证明定理 3.1, 首先考虑以下问题

$\begin{equation}\label{3.1} \left\{ \begin{aligned} -\mathrm{div}(y^{1-2s}\nabla \omega)&=0, && (x,y)\in\mathcal{C},\\ \omega&=0,&& (x,y)\in \partial_{L}\mathcal{C},\\ \partial_{\nu}^{s}\omega&=g_{1}(u,v),&&\quad\ \, x\in\Omega,\\ (-\Delta) E_{1}^{-1}(\xi)&=g_{2}(u,v),&&\quad\ \, x\in \Omega.\\ \end{aligned} \right. \end{equation}$

基于文献[32], 以下结论成立.

引理 3.1 设$\Omega$ 是$\mathbb{R}^{n}\ (2\leq n\leq3)$ 上的一个有界$C^{2,\sigma}$ 区域, 其中 $\sigma\in(0,1)$, 以及$g_{1}\in H^{-s}(\Omega),\ g_{2}\in H^{-1}(\Omega)$. 如果$(\omega,\xi)\ \in H_{0,L}^{s}(\mathcal{C})\times H^{-1}(\Omega))$ 是系统(3.1) 的一个能量解, 令$u=tr_{\Omega}\omega,\ v=E_{1}^{-1}(\xi)$, 则

(i) 若$r_{1}>\frac{n}{2s}$, 且$ g_{1}\in L^{r_{1} (\Omega)},\ g_{2}\in L^{\infty}(\Omega)$, 则$\omega\in L^{\infty}(\mathcal{C});\ u,\ v \in L^{\infty}(\Omega)$;

(ii) 若$g_{1},\ g_{2}\in L^{\infty}(\Omega)$, 则$\omega\in C^{\sigma}(\bar{\mathcal{C}});\ u,\ v \in C^{\sigma}(\bar{\Omega})$;

(iii) 若$g_{1},\ g_{2}\in C^{\sigma}(\bar{\Omega})$, 且$g_{1}|_{\partial\Omega}\equiv0,\ g_{2}|_{\partial\Omega}\equiv0$, 则$\omega\in C^{1,\sigma}(\bar{\mathcal{C}});\ u,\ v \in C^{1,\sigma}(\bar{\Omega})$;

(iv) 若$g_{1},\ g_{2}\in C^{1,\sigma}(\bar{\Omega})$, 且$g_{1}|_{\partial\Omega}\equiv0,\ g_{2}|_{\partial\Omega}\equiv0$, 则$\omega\in C^{2,\sigma}(\bar{\mathcal{C}});\ u,\ v \in C^{2,\sigma}(\bar{\Omega})$.

下面将讨论以下问题

$\begin{equation}\label{3.2} \left\{ \begin{aligned} -\mathrm{div}(y^{1-2s}\nabla \omega)&=0, \ && (x,y)\in\mathcal{C},\\ \omega&=0, && (x,y)\in \partial_{L}\mathcal{C},\\ \partial_{\nu}^{s}\omega&=f_{1}( tr\omega,v), &&\quad\ \, x\in\Omega,\\ -\Delta E_{1}^{-1}(\xi)&=f_{2}( u,E_{1}^{-1}(\xi)), &&\quad\ \, x\in \Omega.\\ \end{aligned} \right. \end{equation}$

以下引理主要讨论正解的正则性.

引理 3.2 设条件(II) 成立, 如果$(\omega,\xi)\in H_{0,L}^{s}(\mathcal{C})\times H^{-1}(\Omega))$ 是系统 (3.2) 的一个能量解, 令$u=tr_{\Omega}\omega,\ v=E_{1}^{-1}(\xi)$, 则存在 $\sigma\in(0,1),$ 使得$\omega\in C^{\sigma}(\bar{\mathcal{C}}), \ u,\ v \in C^{\sigma}(\bar{\Omega})$.

证 类似于文献[33], 对给定$T\geq1,$ 定义$\omega_{T}= \min\{|\omega|,\ T\},\ \tilde{u}_{T}=tr_{\Omega}\omega_{T},\ \tilde{u}=tr_{\Omega}\omega $. 对于$ \hat{\beta}>0,$ 取$\varphi=\omega\omega_{T}^{2\hat{\beta}}\in H_{0,L}^{s}(\mathcal{C})$ 是系统 (3.2) 的一个试验函数. 根据文献[30], 对于$r_{1}>\frac{n}{2s}$ 可得$f_{1}( u,v)\in L^{r_{1}}(\Omega)$, 从而$\omega\in L^{\infty}(\bar{\mathcal{C}})$, $u\in C^{\sigma}(\bar{\Omega})$. 又由$ v\in H_{0}^{1}(\Omega)$, 可得$ v\in L^{\infty}(\Omega)$, 则$ v\in C^{\sigma}(\bar{\Omega})$.

定理 3.1 的证明 设$(u,v)\in H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$ 是系统 (1.1) 的一个弱解, $(\omega,\xi)\in H_{0,L}^{s}(\mathcal{C})\times H^{-1}(\Omega)$ 是系统 (3.2) 的一个能量解. 根据引理 3.2, 可得$\omega\in C^{\sigma}(\bar{\mathcal{C}})$ 且 $u,\ v \in C^{\sigma}(\bar{\Omega})$, 其中$\sigma\in(0,1)$. 根据定义, 可得$f_{1}(u,v) \in C^{1,\sigma}(\bar{\Omega})$, 以及$f_{1}|_{\partial\Omega } =0$. 再由引理 3.1(iii), 结合条件$f_{1}\in C^{1,\sigma}(\bar{\Omega}) $, 则有$u\in C^{2,\sigma}(\bar{\Omega})\cap C_{0}(\bar{\Omega})$. 类似结合$f_{2}(u,v) \in C^{1,\sigma}(\bar{\Omega}) $ 与$f_{2}|_{\partial\Omega} =0$, 可得$v\in C^{2,\sigma}(\bar{\Omega})\cap C_{0}(\bar{\Omega}).$

4 算子$((-\Delta)^{s})^{-1}$, $(-\Delta)^{-1}$ 的性质

本节将定义$(-\Delta)^{s}$ 的逆算子$((-\Delta)^{s})^{-1}$, $(-\Delta) $ 的逆算子$(-\Delta)^{-1}$且讨论这两个算子的性质.

定义 4.1 定义算子$T_{s}:H^{-s}(\Omega)\rightarrow H_{0}^{s}(\Omega),\ T_{1}:H^{-1}(\Omega)\rightarrow H_{0}^{1}(\Omega)$, 即对于$ \tilde{g }\in H^{-s}(\Omega),\ g \in H^{-1}(\Omega)$ 有$T_{s}(\tilde{g })=tr_{\Omega}\omega,\ T_{1}(g )=E_{1}^{-1}(\xi)$, 其中$\omega\in H_{0,L}^{s}(\mathcal{C}),\ \xi\in H^{-1}(\Omega).$

关于逆算子的性质研究可以转化为以下系统的讨论

$\begin{equation}\label{4.1} \left\{ \begin{aligned} -\mathrm{div}(y^{1-2s}\nabla \omega)&=0, && (x,y)\in\mathcal{C},\\ \omega&=0, && (x,y)\in\partial_{L}\mathcal{C},\\ \partial_{\nu}^{s}\omega=(-\Delta)^{s}u&=\tilde{g }(u,v),&&\quad\ \, x\in\Omega,\\ -\Delta v&=g (u,v),&&\quad\ \, x\in \Omega. \end{aligned} \right. \end{equation}$

定义[27,32]

$\begin{equation} \begin{aligned} T_{s}\circ (-\Delta)^{s}&=id|_{H_{0}^{s}(\Omega)},\ (-\Delta)^{s}\circ T_{s}=id|_{H^{-s}(\Omega)},\\ T_{1}\circ(-\Delta)&=id|_{H_{0}^{1}(\Omega)},\ (-\Delta)\circ T_{1}=id|_{H^{-1}(\Omega)}.\nonumber \end{aligned} \end{equation}$

记$T_{s}=((-\Delta)^{s})^{-1},\ T_{1}=(-\Delta)^{-1}.$ 显然, $((-\Delta)^{s})^{-1}=(-\Delta)^{-s}$.

命题 4.1 算子$(-\Delta)^{-s}$ 和$(-\Delta)^{-1}$ 满足以下性质.

(i) $(-\Delta)^{-s},\ (-\Delta)^{-1}$ 是正的自伴紧算子;

(ii) 对于所有的$ r_{1}>\frac{n}{2s}$, $(-\Delta)^{-s} $ 从$L^{r_{1}}(\Omega)$ 映射到$L^{\infty}(\Omega)$, $ (-\Delta)^{-1}$ 从$L^{\infty}(\Omega)$ 映射到$L^{\infty}(\Omega)$;

(iii) 对于所有的$\sigma\in (0,1)$, $(-\Delta)^{-s},\ (-\Delta)^{-1}$ 从$L^{\infty}(\Omega)$ 映射到$C^{\sigma}(\Omega)$.

证 (ii) 和(iii) 的证明可参见引理 3.1 中 (i), (ii). 只需证明性质(i). 首先证明$(-\Delta)^{-s}$ 和$(-\Delta)^{-1}$ 是自伴的. 对于$\tilde{g}_{1},\ \tilde{g}_{2}\in H^{-s}(\Omega),\ g_{1},\ g_{2}\in H^{-1}(\Omega)$, 存在$\omega_{1},\ \omega_{2}\in H_{0,L}^{s}(\mathcal{C}),\ \xi_{1},\ \xi_{2}\in H^{-1}(\Omega)$, 使得对于任意的$ \hat{\varphi}_{1}\in H_{0,L}^{s}(\mathcal{C}),\ \hat{\varphi}_{2}\in H^{-1}(\Omega)$, 以下式子成立

$\begin{equation} \begin{aligned} \int_{\mathcal{C}}y^{1-2s}(\nabla\omega_{1},\nabla\hat{\varphi}_{1}) {\rm d}x{\rm d}y&=\int_{\Omega}\tilde{g}_{1}( tr_{\Omega}\omega_{1},v)tr_{\Omega}\hat{\varphi}_{1}{\rm d}x,\\ \int_{\mathcal{C}}y^{1-2s}(\nabla\omega_{2},\nabla\hat{\varphi}_{1}) {\rm d}x{\rm d}y&=\int_{\Omega}\tilde{g}_{2}( tr_{\Omega}\omega_{2},v)tr_{\Omega}\hat{\varphi}_{1}{\rm d}x, \nonumber \end{aligned} \end{equation}$

及

$\begin{equation} \begin{aligned} \int_{\Omega}\nabla\xi_{1}\cdot\nabla\hat{\varphi}_{2}{\rm d}x&=\int_{\Omega}g_{1}( u,E_{1}^{-1}(\xi_{1}))E_{1}^{-1}(\hat{\varphi}_{2}) {\rm d}x,\\ \int_{\Omega}\nabla\xi_{2}\cdot\nabla\hat{\varphi}_{2}{\rm d}x&=\int_{\Omega}g_{2}( u,E_{1}^{-1}(\xi_{2}))E_{1}^{-1}(\hat{\varphi}_{2}){\rm d}x. \nonumber \end{aligned} \end{equation}$

即有

$\begin{equation} \begin{aligned} ((-\Delta)^{-s}\tilde{g}_{1},\ \tilde{g}_{2})&=\int_{\Omega}\tilde{g}_{2}( tr_{\Omega}\omega_{2},v)tr_{\Omega}\omega_{1}{\rm d}x\\ &=\int_{\mathcal{C}}y^{1-2s}(\nabla\omega_{2},\nabla\omega_{1}) {\rm d}x{\rm d}y\\ &=\int_{\mathcal{C}}y^{1-2s}(\nabla\omega_{1},\nabla\omega_{2}) {\rm d}x{\rm d}y\\ &=\int_{\Omega}\tilde{g}_{1}( tr_{\Omega}\omega_{1},v)tr_{\Omega}\omega_{2}{\rm d}x\\ &=((-\Delta)^{-s}\tilde{g}_{2},\tilde{g}_{1}),\nonumber \end{aligned} \end{equation}$

及

$\begin{equation} \begin{aligned} ((-\Delta)^{-1}g_{1},\ g_{2})&=\int_{\Omega}\nabla\xi_{1}\cdot\nabla\xi_{2}{\rm d}x\\ &=\int_{\Omega}g_{1}( u,E_{1}^{-1}(\xi_{1}))E_{1}^{-1}(\xi_{2}){\rm d}x\\ &=((-\Delta)^{-1}g_{2},\ g_{1}).\nonumber \end{aligned} \end{equation}$

因此, $(-\Delta)^{-s}$ 和$(-\Delta)^{-1}$ 是自伴的.

其次, 验证$(-\Delta)^{-s}$ 和$(-\Delta)^{-1}$ 是正算子. 对于$\forall \tilde{g}=\sum\limits_{k=0}^{\infty}\tilde{g}_{k} \varphi_{k}\in H ^{-s}(\Omega)$ 和$\sum\limits_{k=0}^{\infty}\tilde{g}_{k}^{2}\varphi_{k}^{-s}<\infty$, 可得$(-\Delta)^{-s}\tilde{g}=\sum\limits_{k=0}^{\infty}\tilde{g}_{k}\lambda_{k}^{-s}\varphi_{k}$. 对于$\forall g=\sum\limits_{k=0}^{\infty}g_{k}\varphi_{k}\in H ^{-1}(\Omega) $ 和$\sum\limits_{k=0}^{\infty}g_{k}^{2}\varphi_{k}^{-1}<\infty$, 可知$(-\Delta)^{-1}g=\sum\limits_{k=0}^{\infty}g_{k}\lambda_{k}^{-1}\varphi_{k}$. 因此

$\begin{equation} \begin{aligned} ((-\Delta)^{-s}\tilde{g},\ \tilde{g})&=\int_{\Omega}\tilde{g}\cdot tr_{\Omega}(\tilde{T}(\tilde{g} )){\rm d}x = \sum\limits_{k=0}^{\infty}\tilde{g}_{k}^{2}\lambda_{k}^{-s}\in[0,\infty),\\ ((-\Delta)^{-1}g,\ g)&=\int_{\Omega}\nabla \xi\cdot\nabla \xi {\rm d}x= \sum\limits_{k=0}^{\infty}g_{k}^{2}\lambda_{k}^{-1}\in[0,\infty). \nonumber \end{aligned} \end{equation}$

显然, 对于任意的 $ \tilde{g}\in H^{-s}(\Omega)\setminus\{0\},\ g\in H^{-1}(\Omega)\setminus\{0\}$, 有$((-\Delta)^{-s}\tilde{g},\ \tilde{g})>0,\ ((-\Delta)^{-1}g,\ g)>0$, 故 $(-\Delta)^{-s}$ 和 $(-\Delta)^{-1}$ 是正的算子. 而关于$(-\Delta)^{-s}$ 和$(-\Delta)^{-1}$ 紧性的证明可参见文献[28,30].

5. 定理 1.1 的证明

在本节中, 主要应用下降流不变集的方法证明定理 1.1.

定义算子$A_{1}:H_{0}^{s}(\Omega)\rightarrow H_{0}^{s}(\Omega),\ A_{2}:H_{0}^{1}(\Omega)\rightarrow H_{0}^{1}(\Omega)$, 其中

$\begin{equation} \begin{aligned} A_{1}(u)=(-\Delta)^{-s}\circ h_{1}(u),\ A_{2}(v)=(-\Delta)^{-1}\circ h_{2}(v), \nonumber \end{aligned} \end{equation}$

$h_{1}(u)=f_{1}(u,v),\ h_{2}(v)=f_{2}(u,v)$, 以及$A = \left[ {\begin{array}{cc} A_{1} & 0 \\ 0 & A_{2} \\ \end{array} } \right]$ 是一个 Nemytskii 算子, $(-\Delta)^{-s}$ 和$(-\Delta)^{-1}$ 都是紧的. 结合 (II) 以及紧嵌入$H _{0}^{s}(\Omega)\times H _{0}^{1}(\Omega)\hookrightarrow L^{r_{1}}(\Omega)\times L^{r_{2}}(\Omega),\ r_{1}\in [2,\frac{2n}{n-2s}),\ r_{2}\in [2,\frac{2n}{n-2 })$, 故有以下引理.

引理 5.1 算子$A:H _{0}^{s}(\Omega)\times H _{0}^{1}(\Omega)\rightarrow H _{0}^{s}(\Omega)\times H _{0}^{1}(\Omega)$ 是紧的.

注意到

$\begin{align*} &(E_{1}^{'}(u,v),\phi _{1})_{H^{-s}(\Omega)\times H_{0}^{s}(\Omega)}+(E_{2}^{'}(u,v),\phi _{2})_{H^{-1}(\Omega)\times H_{0}^{1}(\Omega)}\\ =&\int_{\Omega}(-\Delta)^{\frac{s}{2}}u(-\Delta)^{\frac{s}{2}}\phi_{1}{\rm d}x-\int_{\Omega}f_{1}(u,v)\phi_{1}{\rm d}x +\int_{\Omega}\nabla v\cdot\nabla \phi_{2}{\rm d}x-\int_{\Omega}f_{2}(u,v)\phi_{2}{\rm d}x\\ =&\int_{\Omega}(-\Delta)^{\frac{s}{2}}u(-\Delta)^{\frac{s}{2}}\phi_{1}{\rm d}x-\int_{\Omega}(-\Delta)^{s}A_{1}(u)\phi_{1}{\rm d}x +\int_{\Omega}\nabla v\cdot\nabla \phi_{2}{\rm d}x-\int_{\Omega} -\Delta A_{2}(v)\phi_{2}{\rm d}x\\ =&\int_{\Omega}(-\Delta)^{\frac{s}{2}}u(-\Delta)^{\frac{s}{2}}\phi_{1}{\rm d}x-\int_{\Omega}(-\Delta)^{\frac{s}{2}}A_{1}(u)(-\Delta)^{\frac{s}{2}}\phi_{1}{\rm d}x +\int_{\Omega}\nabla v\cdot\nabla \phi_{2}{\rm d}x-\int_{\Omega}\nabla A_{2}(v)\cdot\nabla\phi_{2}{\rm d}x\\ =&(u-A_{1}(u),\phi_{1})_{H_{0}^{s}(\Omega)}+(v-A_{2}(v),\phi_{2})_{H_{0}^{1}(\Omega)}\quad\forall u,\ \phi_{1}\in H_{0}^{s}(\Omega);\ v,\ \phi_{2}\in H_{0}^{1}(\Omega),\nonumber \end{align*}$

则

$\begin{equation} \begin{aligned} E_{1}^{'}(u,v)=u-A_{1}(u),\ E_{2}^{'}(u,v)=v-A_{2}(v),\quad\forall u\in H_{0}^{s}(\Omega),~v\in H_{0}^{1}(\Omega).\nonumber \end{aligned} \end{equation}$

考虑凸集

$\begin{equation} \begin{aligned} P&=\{M_{1}=(u,v): u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega),\ u\geq0,\ v\geq0.~~a.e.\ x\in \Omega\},\\ -P&=\{N_{1}=(u,v): u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega),\ u\leq0,\ v\leq0.~~a.e.\ x\in \Omega\}. \nonumber \end{aligned} \end{equation}$

对于$\epsilon>0$, 令

$\begin{equation} \begin{aligned} D_{\epsilon}^{+}&=\{M=(u,v): u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega),\ \mathrm{dist} (M,M_{1})<\epsilon\},\\ D_{\epsilon}^{-}&=\{N=(u,v): u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega),\ \mathrm{dist} (N,N_{1})<\epsilon\}. \nonumber \end{aligned} \end{equation}$

显然, 对于任意的$\epsilon>0 $, $D_{\epsilon}^{+}$ 和$D_{\epsilon}^{-}$ 是$H _{0}^{s}(\Omega)\times H _{0}^{1}(\Omega)$ 中的两个开凸集. 让$O_{\epsilon}\triangleq D_{\epsilon}^{+}\cap D_{\epsilon}^{-},\ D_{\epsilon}\triangleq \bar{D}_{\epsilon}^{+}\cup \bar{D}_{\epsilon}^{-}$, 则$D_{\epsilon}$ 是$H _{0}^{s}(\Omega)\times H _{0}^{1}(\Omega)$ 中的一个对称闭子集.

让

$\begin{equation} \begin{aligned} K_{1}&=\{u:u \in H_{0}^{s}(\Omega),\ E_{1}^{'}(u,v)=0\},\\ K_{2}&=\{v:v \in H_{0}^{1}(\Omega),\ E_{2}^{'}(u,v)=0\}, \nonumber \end{aligned} \end{equation} $
$ K=K_{1}\times K_{2}. $

对于常数$c$, 记

$\begin{equation} \begin{aligned} K_{c}\!=\!\{(u,v):u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega),\ E_{1}^{'}(u,v)\!=\!E_{2}^{'}(u,v)\!=\!0,\ E(u,v)\!=\!c\}, \ K_{c}^{*}(\epsilon)\!=\!K_{c}\setminus D_{\epsilon}. \nonumber \end{aligned} \end{equation}$

引理 5.2 设条件 (I)-(III) 成立, 则存在$\epsilon_{0}>0$, 使得

$A(\partial D_{\epsilon}^{+})\subset D_{\epsilon}^{+},\ A(\partial D_{\epsilon}^{-})\subset D_{\epsilon}^{-},\ \forall\epsilon\in(0,\epsilon_{0}].$

进而, 若$(u,v)\in D_{\epsilon}^{+}~(\text{或}\ D_{\epsilon}^{-})$ 是系统 (1.1) 的解, 则$(u,v)\in P\ (\text{或} -P).$

证 显然只需证明$A(\partial D_{\epsilon}^{+})\subset D_{\epsilon}^{+}$, 其它结论类似可得. 对于$M\in\partial D_{\epsilon}^{+}$, 记$M=(u,v)$, $\theta=(\theta_{1},\theta_{2})=A(M)$, $ u=u^{+}+u^{-},$ 其中$u^{+}=\max \{u,\ 0\},\ u^{-}=\min \{u,\ 0\}$, 则

$\begin{equation}\label{5.1ab} \begin{aligned} \mathrm{dist}(\theta,P)=\underset{(\phi_{1},\phi_{2})\in P}{\inf }\|(\theta_{1},\theta_{2})-(\phi_{1},\phi_{2})\|_{H _{0}^{s}(\Omega)\times H _{0}^{1}(\Omega)}\leq\|(\theta_{1}^{-},\theta_{2}^{-})\|_{H _{0}^{s}(\Omega)\times H _{0}^{1}(\Omega)}. \nonumber \end{aligned} \end{equation}$

根据引理 2.1(ii), 存在常数$C$, 对于任意的$r_{1}\in[2,\ \frac{2n}{n-2s}),\ r_{2}\in[2,\ \frac{2n}{n-2 }),$ 有

$\begin{equation}\label{5.1a} \begin{aligned} \|(u^{-},v^{-})\|_{L^{r_{1}}(\Omega)\times L^{r_{2}}(\Omega)}&=\underset{(\phi_{1},\phi_{2})\in P}{\inf }\|(u,v)-(\phi_{1},\phi_{2})\|_{L^{r_{1}}(\Omega)\times L^{r_{2}}(\Omega)}\\ &\leq C \underset{(\phi_{1},\phi_{2})\in P}{\inf }\|(E_{s}(u-\phi_{1}),E_{1}(v-\phi_{2}))\|_{H_{0,L}^{s}(\Omega)\times H^{-1}(\Omega)}. \end{aligned} \end{equation} $

由 (I)-(III), 存在常数$C_{\epsilon_{1}}>0$, 使得对于任意的$\epsilon_{1} >0,\ \bar{\omega},\ \bar{\nu}\in\mathbb{R},$ 有

$\begin{align*}\label{5.2} |f_{1}(\bar{\omega},\bar{\nu})|+|f_{2}(\bar{\omega},\bar{\nu})|&\leq\epsilon_{1}|(\bar{\omega},\bar{\nu})|+C_{\epsilon_{1}}|(\bar{\omega},\bar{\nu})|^{p-1},\nonumber\\ |f_{1}(\bar{\omega},\bar{\nu})|&\leq\epsilon_{1} |\bar{\omega}|+C_{\epsilon_{1} }|\bar{\omega}|^{p-1}+C_{\epsilon_{1} }|\bar{\nu}|^{p-1},\\ |f_{2}(\bar{\omega},\bar{\nu})|&\leq\epsilon_{1} |\bar{\nu}|+C_{\epsilon_{1} }|\bar{\omega}|^{p-1}+C_{\epsilon_{1} }|\bar{\nu}|^{p-1},\nonumber \end{align*}$

其中$|(\bar{\omega},\bar{\nu})|=\sqrt{\bar{\omega}^{2}+\bar{\nu}^{2}},\ |\bar{\omega}|,\ |\bar{\nu}|\leq|(\bar{\omega},\bar{\nu})|.$ 由引理 2.1(i), 可得

$\begin{equation} \begin{aligned} \|(E_{s}(\theta_{1}^{-}),E_{1}(\theta_{2}^{-}))\|_{H_{0,L}^{s}(\Omega)\times H^{-1}(\Omega)}^{2}&=\|E_{s}(\theta_{1}^{-})\|^{2}_{H_{0,L}^{s}(\Omega)}+\|E_{1}(\theta_{2}^{-})\|^{2}_{H^{-1}(\Omega)}\\ &=\int_{\Omega}(-\Delta)^{s}A_{1}(u)\theta_{1}^{-}{\rm d}x+\int_{\Omega} -\Delta A_{2}(v)\theta_{2}^{-}{\rm d}x.\nonumber \end{aligned} \end{equation}$

则有

$\begin{equation} \begin{aligned} &\|(E_{s}(\theta_{1}^{-}),E_{1}(\theta_{2}^{-}))\|_{H_{0,L}^{s}(\Omega)\times H^{-1}(\Omega)}^{2}\\ =&\int_{\Omega}f_{1}(u,v)\theta_{1}^{-}{\rm d}x+\int_{\Omega}f_{2}(u,v)\theta_{2}^{-}{\rm d}x\\ \leq&\int_{\Omega}f_{1}( u^{-},v^{-})\theta_{1}^{-}{\rm d}x+\int_{\Omega}f_{2}( u^{-},v^{-})\theta_{2}^{-}{\rm d}x\\ \leq&\int_{\Omega}[\epsilon_{1}|(u^{-},v^{-})|+C_{\epsilon_{1}}|(u^{-},v^{-})|^{p-1}]|(\theta_{1}^{-},\theta_{2}^{-})|{\rm d}x\\ \leq&\epsilon_{1}\|(u^{-},v^{-})\|_{2}\|(\theta_{1}^{-},\theta_{2}^{-})\|_{2}+C_{\epsilon_{1}}\|(u^{-},v^{-})\|_{p}^{p-1}\|\|(\theta_{1}^{-},\theta_{2}^{-})\|_{p}\\ \leq&\big[\epsilon_{1}\tilde{C}\underset{(\phi_{1},\phi_{2})\in P}{\inf }\|(E_{s}(u-\phi_{1}),E_{1}(v-\phi_{2}))\|_{H_{0,L}^{s}(\Omega)\times H^{-1}(\Omega)}\\ &+C_{\epsilon_{1}}\tilde{C}^{p-1}\underset{(\phi_{1},\phi_{2})\in P}{\inf }\|(E_{s}(u-\phi_{1}),E_{1}(v-\phi_{2}))\|_{H_{0,L}^{s}(\Omega)\times H^{-1}(\Omega)}^{p-1}\big]\\ &\cdot\|(E_{s}(\theta_{1}^{-}),E_{1}(\theta_{2}^{-}))\|_{H_{0,L}^{s}(\Omega)\times H^{-1}(\Omega)},\nonumber \end{aligned} \end{equation}$

其中$\tilde{C}>0,$ 结合引理 2.1(i) 和 (5.1) 式, 取$\epsilon_{1}=\frac{1}{4\tilde{C}}$, 则

$\begin{equation} \begin{aligned} \mathrm{dist}(\theta,P)\leq\frac{1}{4}\mathrm{dist}(M,P)+\tilde{M}\mathrm{dist}(M,P)^{p-1},\nonumber \end{aligned} \end{equation}$

其中$\tilde{M}=C_{\epsilon}\tilde{C}^{p-1}$. 让$\epsilon_{0}\in(0,\frac{1}{(4\tilde{M})^{\frac{1}{p-2}}})$, 如果$\mathrm{dist}(M,P)\leq\epsilon\leq\epsilon_{0}$, 以下成立

$\begin{equation}\label{5.3} \begin{aligned} \mathrm{dist}(\theta,P)\leq\frac{1}{2}\mathrm{dist}(M,P)<\epsilon. \end{aligned} \end{equation}$

则有$\theta=A(M)\in D_{\epsilon}^{+}, $ 即$A(\partial D_{\epsilon}^{+})\subset D_{\epsilon}^{+}$.

由于 $M\in D_{\epsilon}^{+},\ 0<\epsilon\leq\epsilon_{0}$, 以及$M$ 是系统 (1.1) 的一个非平凡解, 则$M=A(M)$. 根据(5.3) 式, 可以推出$\mathrm{dist}(M,P)\!=\!0$, 即$M\!\in\! P$. 类似可得$M\!\in\! D_{\epsilon}^{-}\!\cap\!(K\setminus\{(0,0)\})$, 即$M\!\in\! -P$.

引理 5.3 对于任意的$ c\in\mathbb{R},$ 存在常数$\rho_{1}=\rho_{1}(c)>0$, 使得

$\begin{equation} \begin{aligned} &\|u\|_{H_{0}^{s}(\Omega)}+\|v\|_{H_{0}^{1}(\Omega)}+\|A_{1}(u)\|_{H_{0}^{s}(\Omega)}+\|A_{2}(v)\|_{H_{0}^{1}(\Omega)}\\ \leq&\rho_{1}(1+\|u-A_{1}(u)\|_{H_{0}^{s}(\Omega)}+\|v-A_{2}(v)\|_{H_{0}^{1}(\Omega)}),\nonumber \end{aligned} \end{equation}$

其中$(u,v)\in E^{c}=\{(u,v):u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega),\ E(u,v)\leq c\}.$

证 对于$u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega)$, 根据 (III), 可推出

$\begin{equation} \begin{aligned} &E(u,v)-\frac{1}{\mu}(E_{1}^{'}(u,v),u)_{H^{-s}(\Omega),H _{0}^{s}(\Omega)}-\frac{1}{\mu}(E_{2}^{'}(u,v),v)_{H^{-1}(\Omega),H _{0}^{1}(\Omega)}\\ =&(\frac{1}{2}-\frac{1}{\mu})\int_{\Omega}|(-\Delta)^{\frac{s}{2}}u|^{2}{\rm d}x+(\frac{1}{2}-\frac{1}{\mu})\int_{\Omega}|\nabla v|^{2}{\rm d}x -\int_{\Omega}[F(u,v)-\frac{u}{\mu}f_{1}(u,v)-\frac{v}{\mu}f_{2}(u,v)] {\rm d}x\\ \geq&(\frac{1}{2}-\frac{1}{\mu})\int_{\Omega}|(-\Delta)^{\frac{s}{2}}u|^{2}{\rm d}x+(\frac{1}{2}-\frac{1}{\mu})\int_{\Omega}|\nabla v|^{2}{\rm d}x.\nonumber \end{aligned} \end{equation}$

若$(u,v)\in E^{c}$, 则存在$ M_{1}>0$, 使得

$\begin{equation} \begin{aligned} \|u\|_{H_{0}^{s}(\Omega)}^{2}+\|v\|_{H_{0}^{1}(\Omega)}^{2}\leq M_{1}(1+|(E_{1}^{'}(u,v),u)|_{H^{-s}(\Omega)\times H _{0}^{s}(\Omega)}+|(E_{2}^{'}(u,v),v)|_{H^{-1}(\Omega)\times H _{0}^{1}(\Omega)}).\nonumber \end{aligned} \end{equation}$

由Young 不等式, 存在常数$ M_{2}>0,$ 以下成立

$\begin{equation} \begin{aligned} \|u\|_{H_{0}^{s}(\Omega)}+\|v\|_{H_{0}^{1}(\Omega)}\leq M_{2}(1+\|u-A_{1}(u)\|_{H _{0}^{s}(\Omega)}+\|v-A_{2}(v)\|_{H _{0}^{1}(\Omega)}).\nonumber \end{aligned} \end{equation}$

则有

$\begin{equation} \begin{aligned} &\|u\|_{H_{0}^{s}(\Omega)}+\|v\|_{H_{0}^{1}(\Omega)}+\|A_{1}(u)\|_{H_{0}^{s}(\Omega)}+\|A_{2}(v)\|_{H_{0}^{1}(\Omega)}\\ \leq&2\|u\|_{H_{0}^{s}(\Omega)}+2\|v\|_{H_{0}^{1}(\Omega)}+\|u-A_{1}(u)\|_{H _{0}^{s}(\Omega)}+\|v-A_{2}(v)\|_{ H _{0}^{1}(\Omega)}\\ =&2(\|u\|_{H_{0}^{s}(\Omega)}+\|v\|_{H_{0}^{1}(\Omega)})+\|u-A_{1}(u)\|_{H _{0}^{s}(\Omega)}+\|v-A_{2}(v)\|_{H _{0}^{1}(\Omega)}\\ \leq&2M_{2}(1+\|u-A_{1}(u)\|_{H _{0}^{s}(\Omega)}+\|v-A_{2}(v)\|_{ H _{0}^{1}(\Omega)})\\ &+\|u-A_{1}(u)\|_{H _{0}^{s}(\Omega)}+\|v-A_{2}(v)\|_{H _{0}^{1}(\Omega)}\\ \leq&(2M_{2}+1)(1+\|u-A_{1}(u)\|_{H _{0}^{s}(\Omega)}+\|v-A_{2}(v)\|_{H _{0}^{1}(\Omega)}).\nonumber \end{aligned} \end{equation}$

让$\rho_{1}=2M_{2}+1$, 可得

$\begin{equation} \begin{aligned} &\|u\|_{H_{0}^{s}(\Omega)}+\|v\|_{H_{0}^{1}(\Omega)}+\|A_{1}(u)\|_{H_{0}^{s}(\Omega)}+\|A_{2}(v)\|_{H_{0}^{1}(\Omega)}\\ \leq &\rho_{1}(1+\|u-A_{1}(u)\|_{H_{0}^{s}(\Omega)}+\|v-A_{2}(v)\|_{H_{0}^{1}(\Omega)}).\nonumber \end{aligned} \end{equation}$

记

$\begin{equation} \begin{aligned} P_{1}&=\{u:u \in H_{0}^{s}(\Omega),\ u\geq0.\ a.e.\ x\in \Omega\},\ P_{2}=\{v:v \in H_{0}^{1}(\Omega),\ v\geq0.\ a.e.\ x\in \Omega\},\\ -P_{1}&=\{u:u \in H_{0}^{s}(\Omega),\ u\leq0.\ a.e.\ x\in \Omega\},\ -P_{2}=\{v:v \in H_{0}^{1}(\Omega),\ v\leq0.\ a.e.\ x\in \Omega\}; \nonumber \end{aligned} \end{equation} $
$\begin{equation} \begin{aligned} D_{1\epsilon}^{+}&=\{u:u \in H_{0}^{s}(\Omega),\ \mathrm{dist}(u,P_{1})<\epsilon\},&&D_{2\epsilon}^{+} =\{v:v \in H_{0}^{1}(\Omega),\ \mathrm{dist}(v,P_{2})<\epsilon\},\\ D_{1\epsilon}^{-}&=\{u:u \in H_{0}^{s}(\Omega),\ \mathrm{dist}(u,-P_{1})<\epsilon\},&& D_{2\epsilon}^{-} =\{v:v \in H_{0}^{1}(\Omega),\ \mathrm{dist}(v,-P_{2})<\epsilon\}. \nonumber \end{aligned} \end{equation}$

结合文献[23] 和引理 5.2, 有以下引理.

引理 5.4 存在两个局部连续的Lipschitz 算子

$\begin{equation} \begin{aligned} B_{1}:H_{0}^{s}(\Omega)\setminus K_{1}\longrightarrow H_{0}^{s}(\Omega); \quad B_{2}:H_{0}^{1}(\Omega)\setminus K_{2}\longrightarrow H_{0}^{1}(\Omega). \nonumber \end{aligned} \end{equation}$

它们有以下性质

(i) $B_{1}(\bar{D}_{1\epsilon}^{\pm})\subset \bar{D}_{1\epsilon}^{\pm},\ B_{2}(\bar{D}_{2\epsilon}^{\pm})\subset \bar{D}_{2\epsilon}^{\pm},\ \forall\epsilon\in(0,\epsilon_{0}].$

(ii) $\begin{equation} \left\{ \begin{aligned} &\frac{1}{2}\|u-B_{1}(u)\|_{H_{0}^{s}(\Omega)}\leq\|u-A_{1}(u)\|_{H_{0}^{s}(\Omega)}\leq2\|u-B_{1}(u)\|_{H_{0}^{s}(\Omega)},\ &&\forall u\in H_{0}^{s}(\Omega)\setminus K_{1},\\ &\frac{1}{2}\|v-B_{2}(v)\|_{H_{0}^{1}(\Omega)}\leq\|v-A_{2}(v)\|_{H_{0}^{1}(\Omega)}\leq2\|v-B_{2}(v)\|_{H_{0}^{1}(\Omega)},\ &&\forall v\in H_{0}^{1}(\Omega)\setminus K_{2}.\nonumber \end{aligned} \right. \end{equation}$

(iii) $\begin{equation} \left\{ \begin{aligned} &(E_{1}^{'}(u,v),u-B_{1}(u))_{H^{-s}(\Omega),H_{0}^{s}(\Omega)}\geq\frac{1}{2}\|u-A_{1}(u)\|_{H_{0}^{s}(\Omega)}^{2},\ &&\forall u\in H_{0}^{s}(\Omega)\setminus K_{1},\\ &(E_{2}^{'}(u,v),v-B_{2}(v))_{H^{-1}(\Omega),H_{0}^{1}(\Omega)}\geq\frac{1}{2}\|v-A_{2}(v)\|_{H_{0}^{1}(\Omega)}^{2},\ &&\forall v\in H_{0}^{1}(\Omega)\setminus K_{2}.\nonumber \end{aligned} \right. \end{equation}$

(iv) 设条件 (IV) 成立, 则$B_{1}$ 和$B_{2}$ 是奇算子.

记

$\begin{equation} \begin{aligned} B = \left[ {\begin{array}{cc} B_{1} & 0 \\ 0 & B_{2} \\ \end{array} } \right]. \nonumber \end{aligned} \end{equation}$

对于$M_{0}=(u_{0},v_{0})\in (H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega))\setminus K$, 有

$\begin{equation}\label{5.4} \left\{ \begin{aligned} \frac{{\rm d}(M)}{{\rm d}t}&=-M+B(M),\\ (u(0),v(0))&=(u_{0},v_{0}), \end{aligned} \right. \end{equation}$

即

$\begin{equation} \left\{ \begin{aligned} (\frac{{\rm d} u }{{\rm d}t},\frac{{\rm d} v }{{\rm d}t})&=(-u+B_{1}(u),-v+B_{2}(v)),\\ (u(0),v(0))&=(u_{0},v_{0}).\nonumber \end{aligned} \right. \end{equation}$

根据Banach 空间上的广义积分, 存在最大区间$[0,\tau(M_{0}))$, 使得 (5.4)式有唯一解$\varphi^{t}(M_{0})$ 且满足

$\begin{equation} \begin{aligned} \varphi^{t}(M_{0})&=e^{-t}(M_{0}+\int_{0}^{t}e^{s}B(\varphi^{s}(M_{0})) {\rm d}s),\nonumber \end{aligned} \end{equation}$

其中$t\in[0,\tau(M_{0}))$. 结合引理 5.4(iii), 可得

$\begin{equation} \begin{aligned} \frac{\rm d}{{\rm d}t}E(\varphi^{t}(M_{0}))&=(E_{1}^{'}(u,v),-u+B_{1}(u))_{H^{-s}(\Omega),H_{0}^{s}(\Omega)}+(E_{2}^{'}(u,v),-v+B_{2}(v))_{H^{-1}(\Omega),H_{0}^{1}(\Omega)}\\ &\leq-\frac{1}{2}\|u-A_{1}(u)\|_{H_{0}^{s}(\Omega)}^{2}-\frac{1}{2}\|v-A_{2}(v)\|_{H_{0}^{1}(\Omega)}^{2}.\nonumber \end{aligned} \end{equation}$

因此, $E(\varphi^{t}(M_{0}))$ 在$[0,\tau(M_{0})) $ 上是一个递减函数. 根据文献[25], 有以下结论.

引理 5.5 对于任意的$\epsilon\in(0,\epsilon_{0}],\ M_{0}\in \bar{D}_{\epsilon}^{\pm}\setminus K~(\bar{D}^{\pm}_{\epsilon}=\bar{D}^{\pm}_{1\epsilon}\times \bar{D}^{\pm}_{2\epsilon} ),$ 可得$\varphi (t,M_{0})\in D_{\epsilon}^{\pm}$, 其中$t\in (0,\tau(M_{0}))$ 且 $\epsilon_{0}$ 见引理 5.2.

给定$\epsilon\in (0,\epsilon_{0}]$, 定义

$\begin{equation} \begin{aligned} C_{X}(O_{\epsilon})=O_{\epsilon}\cup\{M_{0}\in X_{0}:\exists t\in [0,\tau(M_{0})),\ s.t.\ \varphi (t,M_{0})\in O_{\epsilon}\}. \nonumber \end{aligned} \end{equation}$

记$\partial C_{X}(O_{\epsilon})$ 是$H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$ 中 $C_{X}(O_{\epsilon})$ 的边界.

为得到定理 1.1 的第一部分结论, 需要以下定理.

定理 5.1[25] 记$X$ 是一个Hilbert 空间, $ \Phi\in C^{1}(X,\mathbb{R} )$. 设$\Phi$ 在$X $ 上满足$\mathrm{(PS)}$ 条件, 并且对于$M\in X $, 满足$\Phi'(M)=M-A(M)$ 且设 $D^{+}$, $D^{-}$ 是$X $ 的开凸子集, $D^{+}\cap D^{-}\neq\emptyset,\ A(\partial D^{\pm})\subset D^{\pm}.$ 如果存在一条路径$h$: $[0,1]\times[0,1] \rightarrow X,$ 使得

$\begin{equation} \begin{aligned} h(0,0 )\in D^{+}\setminus\bar{D}^{-},\quad\quad h(1,1 )\in D^{-}\setminus\bar{D}^{+}. \nonumber \end{aligned} \end{equation}$

以及

$\begin{equation} \begin{aligned} \underset{(t,s )\in[0,1]\times[0,1]}{\max }\Phi(h(t,s ))< \underset{M\in\bar{D}^{+}\cap\bar{D}^{-}}{\inf}\Phi(M), \nonumber \end{aligned} \end{equation}$

则在$\partial C_{X}(D^{+}\cap D^{-})\setminus (\bar{D}^{+}\cup\bar{D}^{-}),\ \partial C_{X}(D^{+}\cap D^{-})\cap D^{+},\ \partial C_{X}(D^{+}\cap D^{-})\cap D^{-} $ 和$ D^{+}\cap D^{-} $ 中至少各存在一个临界点.

定理 1.1 的证明 根据 (5.2) 式和(III), 可得

$\begin{equation} \begin{aligned} E(u,v)&=\frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}u|^{2}+|\nabla v|^{2}{\rm d}x-\int_{\Omega}[\int_{0}^{u}f_{1}( t,v)\ dt+\frac{1}{4}|v|^{4}\ ]{\rm d}x\\ &\geq\frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}u|^{2}+|\nabla v|^{2}{\rm d}x-\int_{\Omega}[\frac{\epsilon_{1}}{2}|u|^{2}+\frac{C_{\epsilon_{1}}}{p}|u|^{p}+C_{\epsilon_{1}}|v|^{p-1}|u|+\frac{1}{4}|v|^{4}] {\rm d}x\\ &\geq-\frac{\epsilon_{1}}{2}|\Omega|^{\frac{p-2}{p}}\|u\|_{p}^{2}-\frac{C_{\epsilon_{1}}}{p}|\Omega|^{\frac{p-1}{p}}\|u\|_{p}^{p}-C_{\epsilon_{1}}\|u\|_{p}\|v\|_{p}^{p-1}-\frac{1}{4}|\Omega|^{\frac{p-4}{p}}\|v\|_{p}^{4}\\ &\geq-\frac{\epsilon_{1}}{2}|\Omega|^{\frac{p-2}{p}}\|(u,v)\|_{p}^{2}-(\frac{|\Omega|^{\frac{p-1}{p}}}{p}+1)C_{\epsilon_{1}}\|(u,v)\|_{p}^{p}-\frac{1}{4}|\Omega|^{\frac{p-4}{p}}\|(u,v)\|_{p}^{4}.\nonumber \end{aligned} \end{equation}$

若对于$\epsilon\in [0,\epsilon_{0}]$, 有 $(u,v)\in \bar{D}_{\epsilon}^{+}\cap \bar{D}_{\epsilon}^{-}$. 结合 (5.1) 式和引理 2.1(i), 存在常数$C >0$ 使得 $\forall r_{1}\in [2,\frac{2n}{n-2s}),\ r_{2}\in [2,\frac{2n}{n-2 })$ 有

$\begin{equation} \begin{aligned} \|(u^{\pm},v^{\pm})\|_{L^{r_{1}}(\Omega)\times L^{r_{2}}(\Omega)}&\leq C \underset{(\phi_{1},\phi_{2})\in P}{\inf }\|(E_{s}(u -\phi_{1}),E_{1}(v -\phi_{2})\|_{H_{0,L}^{s}(\Omega)\times H^{-1}(\Omega)}\\ &=C \mathrm{dist}(M,\pm P)\\ &\leq C \epsilon. \nonumber \end{aligned} \end{equation}$

故存在常数 $\alpha_{\epsilon}>-\infty,$ 使得

$\begin{equation} \begin{aligned} \underset{{(u,v)\in \bar{D}_{\epsilon}^{+}}\cap {\bar{D}_{\epsilon}^{-}}}{\inf}E(u,v)=\alpha_{\epsilon}. \nonumber \end{aligned} \end{equation}$

根据 (III), 对于$ \bar{\omega},\bar{\nu}\in \mathbb{R}$, 存在 $b_{1},\ b_{2},\ b_{3}>0$ 使得

$\begin{equation} \begin{aligned} F(\bar{\omega},\bar{\nu})\geq b_{1}|\bar{\omega}|^{\mu}+b_{2}|\bar{\nu}|^{\mu}-b_{3}. \nonumber \end{aligned} \end{equation}$

因此, 对于$ u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega),$ 以下式子成立

$\begin{equation} \begin{aligned} E(u,v)&=\frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}u|^{2}+|\nabla v|^{2}{\rm d}x-\int_{\Omega}F(u,v){\rm d}x\\ &\leq\frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}u|^{2}+|\nabla v|^{2}{\rm d}x-b_{1}\int_{\Omega}|u|^{\mu}{\rm d}x-b_{2}\int_{\Omega}|v|^{\mu}{\rm d}x+b_{3}|\Omega|.\nonumber \end{aligned} \end{equation}$

记$\{\varphi_{k}\}$ 为$(-\Delta)^{s}$ 和$-\Delta$ 的特征向量, 它们对应的特征值分别是$\lambda_{k}^{s}$ 和$ \lambda_{k}$. 令$h (t_{0})= R \varphi_{1}\sin (\pi t_{0})+ R \varphi_{2}\cos (\pi t_{0}),\ h (s_{0})=R \varphi_{1}\sin (\pi s_{0})+R\varphi_{2}\cos (\pi s_{0}),\ t_{0},\ s_{0}\in[0,1].$ 那么

$\begin{align*} &E(h (t_{0}),h (s_{0}))\\ \leq&\frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}h(t_{0})|^{2}+|\nabla h (s_{0})|^{2}{\rm d}x -b_{1}\int_{\Omega}|\ h (t_{0}) |^{\mu}{\rm d}x-b_{2}\int_{\Omega}|h (s_{0})|^{\mu}{\rm d}x+b_{3}|\Omega|\\ =&\frac{1}{2} R ^{2}\|\gamma(t_{0})\|_{H_{0}^{s}(\Omega)}^{2}+\frac{1}{2} R^{2}\|\gamma(s_{0})\|_{H_{0}^{1}(\Omega)}^{2}-b_{1} R ^{\mu}\|\gamma(t_{0})\|_{\mu}^{\mu}-b_{2}R^{\mu}\|\gamma(s_{0})\|_{\mu}^{\mu}+b_{3}|\Omega|, \nonumber \end{align*}$

其中$\gamma(t_{0})=\varphi_{1}\sin(\pi t_{0})+\varphi_{2}\cos(\pi t_{0}),\ \gamma(s_{0})=\varphi_{1}\sin(\pi s_{0})+\varphi_{2}\cos(\pi s_{0}).$ 因为$\mu>2$, 则存在一个充分大的$R_{\epsilon}>0$, 使得$E(h (t_{0}),h (s_{0}))<\bar{\alpha} \triangleq \underset{\epsilon\in[0,\epsilon_{0}]}{\inf}\alpha_{\epsilon}.$ 由定理 5.1, $E(u,v)$ 至少有 4 个临界点, 即$(u_{1},v_{1})\in\partial C_{X}(O_{\epsilon})\setminus D\epsilon,\ (u_{2},v_{2})\in\partial C_{X}(O_{\epsilon})\setminus \bar{D}_{\epsilon}^{+},\ (u_{3},v_{3})\in\partial C_{X}(O_{\epsilon})\setminus \bar{D}_{\epsilon}^{-},\ (u_{4},v_{4})\in O_{\epsilon}$. 根据引理 3.2, 可得$u_{i},\ v_{i}\in C^{\sigma}(\bar{\Omega}),\ (\sigma\in(0,1),\ i=1,2,3,4)$. 显然, $(u_{1},v_{1})$ 是一个变号解, 并且$(u_{4},v_{4})=(0,0)$. 根据引理 5.2, 可以推出$(u_{2},v_{2})\in P,\ (u_{3},v_{3})\in -P$. 应用强极大值原理, 在$ \Omega $ 上, $u_{2},\ v_{2}>0,\ u_{3},\ v_{3}<0$.

下面将要证明定理 1.1 的第二部分.

记

$\begin{equation} \begin{aligned} \mathcal{N}\triangleq \{&(u,v)\in H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega):(u^{+},v^{+}) \neq(0,0),\ (u^{-},v^{-})\neq(0,0),\\ &E_{1}'(u,v)u^{+}+E_{2}'(u,v)v^{+} =E_{1}'(u,v)u^{-}+E_{2}'(u,v)v^{-}=0\}. \nonumber \end{aligned} \end{equation}$

显然, $\mathcal{N}$ 包含了系统 (1.1) 所有的变号解. 定义

$\begin{equation} \begin{aligned} \tilde{\xi}=\underset{(u,v)\in \mathcal{N}}{\inf}E(u,v). \nonumber \end{aligned} \end{equation}$

根据 (III), 可得$E$ 在$\mathcal{N}$ 上是强制的, 则$\tilde{\xi}>-\infty$. 取$\{(u_{k},v_{k})\}$ 是$\mathcal{N}$ 中的极小值序列, 让$ u_{k}=u_{k}^{+}+u_{k}^{-},\ v_{k}=v_{k}^{+}+v_{k}^{-}$, 其中$u_{k}^{+}=\max\{u_{k},0\}$, $u_{k}^{-}=\min\{u_{k},0\}$ 和 $v_{k}^{+}=\max\{v_{k},0\}$, $v_{k}^{-}=\min\{v_{k},0\}$. 令

$\begin{equation}\label{5.5a} \begin{aligned} E(u_{k}^{\pm},v_{k}^{\pm})=\underset {(\hat{t},\hat{s})\in [0,\infty)\times[0,\infty)}{\max}E(\hat{t}u_{k}^{\pm},\hat{s}v_{k}^{\pm}). \end{aligned} \end{equation}$

接下来, 考虑$E(u_{k}^{+},v_{k}^{+})$, $E(u_{k}^{-},v_{k}^{-})$ 类似可得. 对于任意的$ k\in \mathbb{N},$ 定义

$\begin{equation} \begin{aligned} \rho_{k}^{+}(\hat{t},\hat{s})=E(\hat{t}u_{k}^{+},\hat{s}v_{k}^{+}),~~\forall \hat{t},\ \hat{s}\geq0,\nonumber \end{aligned} \end{equation}$

显然, 可得$\rho_{k}^{+}(0,0)=0.$ 另外,

$\begin{equation} \begin{aligned} \frac{\partial}{\partial\hat{t}}\rho_{k}^{+}(\hat{t},\hat{s})&=(E_{1}'(\hat{t}u_{k}^{+},\hat{s}v_{k}^{+}),u_{k}^{+})_{H^{-s}(\Omega),H_{0}^{s}(\Omega)}=\hat{t}[\|u_{k}^{+}\|_{H_{0}^{s}(\Omega)}^{2} -\int_{\Omega}\frac{f_{1}( \hat{t}u_{k}^{+},\hat{s}v_{k}^{+})}{\hat{t}u_{k}^{+}}(u_{k}^{+})^{2}{\rm d}x],\\ \frac{\partial}{\partial\hat{s}}\rho_{k}^{+}(\hat{t},\hat{s})&=(E_{2}'(\hat{t}u_{k}^{+},\hat{s}v_{k}^{+}),v_{k}^{+})_{H^{-1}(\Omega),H_{0}^{1}(\Omega)}=\hat{s}[\|v_{k}^{+}\|_{H_{0}^{1}(\Omega)}^{2} -\int_{\Omega}\frac{f_{2}( \hat{t}u_{k}^{+},\hat{s}v_{k}^{+})}{\hat{s}u_{k}^{+}}(v_{k}^{+})^{2}{\rm d}x]. \nonumber \end{aligned} \end{equation}$

根据 (V), 可得$\frac{1}{\hat{t}}\rho_{k}^{+}(\hat{t},\hat{s}) $ 是$(0,\infty)$ 上关于$\hat{t}$ 的一个不增函数, 以及$ \frac{1}{\hat{s}}\rho_{k}^{+}(\hat{t},\hat{s})$ 是$(0,\infty)$ 上关于$\hat{s}$ 的不增函数. 记$\Theta_{k}^{+}\triangleq \{(\hat{t},\hat{s}):\hat{t},\ \hat{s}>0,\ \frac{\partial}{\partial\hat{t}}\rho_{k}^{+}(\hat{t},\hat{s})=\frac{\partial}{\partial\hat{s}}\rho_{k}^{+}(\hat{t},\hat{s})=0\}$, 显然$\Theta_{k}^{+}$ 是 $(0,\infty)\times(0,\infty)$ 上的一个闭子区间且$(1,1)\in\Theta_{k}^{+}$, 则有

$\begin{equation} \begin{aligned} \rho_{k}^{+}(1,1)=\underset{(\hat{t},\hat{s})\in \Theta_{k}^{+}}{\max}\rho_{k}^{+}(\hat{t},\hat{s})=\underset{(\hat{t},\hat{s})\in [0,\infty)\times[0,\infty)}{\max}\rho_{k}^{+}(\hat{t},\hat{s}). \nonumber \end{aligned} \end{equation}$

同样地, 记$\Theta_{k}^{-}\triangleq\{(\hat{t},\hat{s}):\hat{t},\ \hat{s}>0,\ \frac{\partial}{\partial\hat{t}}\rho_{k}^{-}(\hat{t},\hat{s})=\frac{\partial}{\partial\hat{s}}\rho_{k}^{-}(\hat{t},\hat{s})=0\}$, 则

$\begin{equation} \begin{aligned} \rho_{k}^{-}(1,1)=\underset{(\hat{t},\hat{s})\in \Theta_{k}^{-}}{\max}\rho_{k}^{-}(\hat{t},\hat{s})=\underset{(\hat{t},\hat{s})\in [0,\infty)\times[0,\infty)}{\max}\rho_{k}^{-}(\hat{t},\hat{s}). \nonumber \end{aligned} \end{equation}$

对于任意的$k\in\mathbb{ N}$, 记

$\begin{equation} \begin{aligned} \pi_{k}\triangleq\{(t_{1}u_{k}^{+}+t_{2}u_{k}^{-},\ s_{1}v_{k}^{+}+s_{2}v_{k}^{-}):t_{1},\ t_{2},\ s_{1},\ s_{2}\geq 0\}, \nonumber \end{aligned} \end{equation}$

则

$\begin{equation} \begin{aligned} \sup~E(\pi_{k})&=\underset{(t_{1},s_{1})\in[0,\infty)\times[0,\infty)}{\max}E(t_{1}u_{k}^{+},s_{1}v_{k}^{+})+\underset{(t_{2},s_{2})\in[0,\infty)\times[0,\infty)}{\max}E(t_{2}u_{k}^{-},s_{2}v_{k}^{-})\\ &=E(u_{k}^{+},v_{k}^{+})+E(u_{k}^{-},v_{k}^{-}).\nonumber \end{aligned} \end{equation}$

对于任意的$k$, $(u_{k}^{+},\ v_{k}^{+}),(u_{k}^{-},v_{k}^{-})$ 是线性无关的, 以及$\pi_{k}\subset \mathrm{span}\{(u_{k}^{+},v_{k}^{+}),\ (u_{k}^{-},v_{k}^{-})\}.$ 显然, 存在一个$R_{k}>0$, 使得对所有的$(u,v)\in\pi_{k}\setminus B_{R_{k}},$ 有$E(u,v)\leq-1$, 其中$B_{R_{k}}\triangleq\{(u,v)\in\pi_{k}:\|(u,v)\|_{H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)}<R_{k}\}$.

定义$h_{k} : H= [0,1]\times[0,1]\rightarrow H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$, 满足

$\begin{equation} \begin{aligned} h_{k}(t_{1},s_{1})&\!=\!(R_{k}t_{1}\frac{u_{k}^{+}}{\|u_{k}^{+}\|_{H_{0}^{s}(\Omega)}}\!+\!R_{k}(1-t_{1})\frac{u_{k}^{-}}{\|u_{k}^{-}\|_{H_{0}^{s}(\Omega)}},\ R_{k}s_{1}\frac{v_{k}^{+}}{\|v_{k}^{+}\|_{H_{0}^{1}(\Omega)}}\!+\!R_{k}(1-s_{1})\frac{v_{k}^{-}}{\|v_{k}^{-}\|_{H_{0}^{1}(\Omega)}})\\ &=(h_{1},h_{2}).\nonumber \end{aligned} \end{equation}$

显然, 对于任意的$ \epsilon\in(0,\epsilon_{0}]$, $h_{k}(0,0 )\in(\pi_{k}\setminus B_{R_{k}})\cap C_{X}(D_{\epsilon}^{-}),\ h_{k}(1,1 )\in(\pi_{k}\setminus B_{R_{k}})\cap C_{X}(D_{\epsilon}^{+})$. 定义同伦映射$H_{k} : H\times H\rightarrow H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$, 满足

$\begin{equation} \begin{aligned} H_{k}(t_{1},s_{1},t_{2},s_{2} )= (t_{2}h_{1},s_{2}h_{2} ),~~\forall(t_{1},s_{1},t_{2},s_{2} )\in H\times H.\nonumber \end{aligned} \end{equation}$

由于$O_{\epsilon}$ 是$H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$ 中的一个有界开集, 因此$C_{X}(O_{\epsilon})$ 在$H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega) $ 中是一个有界开集, 故$\mathcal{O\epsilon}\triangleq H_{k}^{-1}(C_{X}(O_{\epsilon}))$ 和$\mathcal{O\epsilon}^{\pm}\triangleq H_{k}^{-1}(C_{X}(D_{\epsilon}^{\pm}))$ 是$H\times H$ 中的开子集. 由于对于所有的$(t,s)\in H,$ 有$H_{k}(t,s, 0,0)=(0,0)\in C_{X}(O_{\epsilon})$, 则$\{(t,s,0,0 ):(t,s)\in H\}\subset \mathcal{O}_{k}.$ 此外, 由 (III), 可推得

$\begin{equation} \begin{aligned} &E(H_{k}(t_{1},s_{1}, 1,1))\\=&E(h_{k}(t_{1},s_{1} ))=E(h_{1},h_{2})\\ =&\frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}h_{1}|^{2}+|\nabla h_{2}|^{2}{\rm d}x-\int_{\Omega}F( h_{1},h_{2}){\rm d}x\\ \leq& \frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}h_{1}|^{2}+|\nabla h_{2}|^{2}{\rm d}x-b_{1}\int_{\Omega}|h_{1}|^{\mu}{\rm d}x-b_{2}\int_{\Omega}|h_{2}|^{\mu}{\rm d}x+b_{3}|\Omega|\\ =&\frac{1}{2}\|h_{1}\|_{H_{0}^{s}(\Omega)}^{2}+\frac{1}{2}\|h_{2}\|_{H_{0}^{1}(\Omega)}^{2}-b_{1}\|h_{1}\|_{\mu}^{\mu}-b_{2}\|h_{2}\|_{\mu}^{\mu}+b_{3}|\Omega|\\ =&\frac{1}{2}R_{k}^{2}\|\gamma_{1k}(t_{1})\|_{H_{0}^{s}(\Omega)}^{2}+\frac{1}{2}R_{k}^{2}\|\gamma_{2k}(s_{1})\|_{H_{0}^{1}(\Omega)}^{2}-b_{1}R_{k}^{\mu}\|\gamma_{1k}(t_{1})\|_{\mu}^{\mu} -b_{2}R_{k}^{\mu}\|\gamma_{2k}(s_{1})\|_{\mu}^{\mu} +b_{3}|\Omega|,\nonumber \end{aligned} \end{equation}$

其中$\gamma_{1k}(t_{1})=t_{1}\frac{u_{k}^{+}}{\|u_{k}^{+}\|_{H_{0}^{s}(\Omega)}}+(1-t_{1})\frac{u_{k}^{-}}{\|u_{k}^{-}\|_{H_{0}^{s}(\Omega)}},\ \gamma_{2k}(s_{1})=s_{1}\frac{v_{k}^{+}}{\|v_{k}^{+}\|_{H_{0}^{1}(\Omega)}}+(1-s_{1})\frac{v_{k}^{-}}{\|v_{k}^{-}\|_{H_{0}^{1}(\Omega)}}.$ 显然, 存在一个$R_{0}>0$, 如果$R_{k}\geq R_{0}$, 对于所有的$(t_{1},s_{1})\in H$, 有$E(H_{k}(t_{1},s_{1}, 1,1))\leq-1$. 然而, $\underset{M\in C_{X}(O_{\epsilon})}{\inf}E(M)=\underset{M\in O_{\epsilon}}{\inf}E(M)\geq\underset{M\in O_{\epsilon}}{\inf}E(\varphi(t,M))=0,$ 则$\{(t_{1},s_{1}, 1,1):(t_{1},s_{1} )\in H\}\cap \mathcal{O}_{k}=\emptyset.$ 根据文献[25], 存在一个$\partial_{H\times H}\mathcal{O}_{k}$ 中的连通分量$P_{k}$, 其中$\partial_{H\times H}\mathcal{O}_{k}$ 是$\{(0,0,t_{2},s_{2}):(t_{2},s_{2})\in H\}\subset \mathcal{O}^{-}_{k}$ 和$\{(1,1,t_{2},s_{2}):(t_{2},s_{2})\in H\}\subset \mathcal{O}^{+}_{k}$ 的交集. 因为$H_{k}(P_{k})\subset H_{k}(\partial_{H\times H}\mathcal{O}_{k})\subset\partial H_{k}(\mathcal{O}_{k})=\partial C_{X}(O_{\epsilon}),\ H_{k}(\mathcal{O}_{k}^{+}\cap\mathcal{O}_{k}^{-})=H_{k}(\mathcal{O}_{k}^{+})\cap H_{k}(\mathcal{O}_{k}^{-})=C_{X}(D_{\epsilon}^{+})\cap C_{X}(D_{\epsilon}^{-})\subset C_{X}(O_{\epsilon}),$ 则$P_{k}\cap\mathcal{O}_{k}^{+}\cap\mathcal{O}_{k}^{-}=\emptyset.$ 结合$\partial_{H\times H}\mathcal{O}_{k}\cap\mathcal{O}_{k}^{+}\supset P_{k}\cap\mathcal{O}_{k}^{+}\neq\emptyset$ 和$\partial_{H\times H}\mathcal{O}_{k}\cap\mathcal{O}_{k}^{-}\supset P_{k}\cap\mathcal{O}_{k}^{-}\neq\emptyset,$ 取$(t_{1k},s_{1k},t_{2k},s_{2k} )\in P_{k}\setminus(\mathcal{O}_{k}^{+}\cup\mathcal{O}_{k}^{-}),$ 显然有

$\begin{equation} \begin{aligned} (v_{1k}^{*},v_{2k}^{*})\triangleq H_{k}(t_{1k},s_{1k},t_{2k},s_{2k} )\in \mathcal{M}_{\epsilon}\triangleq\partial C_{X}(O_{\epsilon})\setminus(C_{X}(D_{\epsilon}^{+})\cup D_{\epsilon}^{-})). \nonumber \end{aligned} \end{equation}$

容易验证$\partial C_{X}(O_{\epsilon})$ 和$\partial C_{X}(D_{\epsilon}^{\pm})$ 是$\varphi^{t}(v_{1k}^{*},v_{2k}^{*}) $ 的不变集, 由此可得

$\begin{equation}\label{5.6} \begin{aligned} \{\varphi^{t}(v_{1k}^{*},v_{2k}^{*}):0\leq t< \tau(v_{1k}^{*},v_{2k}^{*})\}\subset \mathcal{M}_{\epsilon}. \end{aligned} \end{equation}$

对于任意的$(u,v)\in\bar{D_{\epsilon}^{+}}\cap\bar{D_{\epsilon}^{-}}$, 结合 (5.1) 式和引理 2.1(i), 有

$\begin{equation} \begin{aligned} \|(u^{\pm},v^{\pm})\|_{L^{r_{1}}(\Omega)\times L^{r_{2}}(\Omega)}\leq C \epsilon_{0}, \nonumber \end{aligned} \end{equation}$

其中$r_{1}\in[2,\frac{2n}{n-2s}),\ r_{2}\in[2,\frac{2n}{n-2 })$. 根据 (V) 和引理 5.5, 可得

$\begin{equation} \begin{aligned} \underset{0\leq t<\tau(M^{*})}{\inf}E(\varphi^{t}(M^{*}))\geq \underset{M\in\partial C_{X}(O_{\epsilon})}{\inf}E(M)\geq\underset{M\in C_{X}(O_{\epsilon})}{\inf}E(M)\geq0, \nonumber \end{aligned} \end{equation}$

其中$M^{*}=(v_{1k}^{*},v_{2k}^{*}),\ M=(u,v)$. 对于$0<\tilde{t}_{1}<\tilde{t}_{2}<\tau(M^{*})$, 可得

$\begin{equation} \begin{aligned} \|\varphi^{\tilde{t}_{2}}(M^{*})-\varphi^{\tilde{t}_{1}}(M^{*})\|_{H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)}&\leq\int_{\tilde{t}_{1}}^{\tilde{t}_{2}}\|\varphi^{\gamma}(M^{*})-B(\varphi^{\gamma}(M^{*}))\|_{H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)}{\rm d}\gamma\\ &\leq2\int_{\tilde{t}_{1}}^{\tilde{t}_{2}}\|\varphi^{\gamma}(M^{*})-A(\varphi^{\gamma}(M^{*}))\|_{H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)}{\rm d}\gamma\\ &\leq2(\int_{\tilde{t}_{1}}^{\tilde{t}_{2}}\|\varphi^{\gamma}(M^{*})-A(\varphi^{\gamma}(M^{*}))\|^{2}_{H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)} {\rm d}\gamma)^{\frac{1}{2}}(\tilde{t}_{2}-\tilde{t}_{1})^{\frac{1}{2}}\\ &\leq2(E(\varphi^{\tilde{t}_{1}}(M^{*}))-E(\varphi^{\tilde{t}_{2}}(M^{*})))^{\frac{1}{2}}(\tilde{t}_{2}-\tilde{t}_{1})^{\frac{1}{2}}\\ &\leq2(E(M^{*}))^{\frac{1}{2}}(\tilde{t}_{2}-\tilde{t}_{1})^{\frac{1}{2}}. \nonumber \end{aligned} \end{equation}$

若$\tau(M^{*})<\infty$, 那么

$\begin{equation} \begin{aligned} \underset{\tilde{t}_{1},\tilde{t}_{2}\rightarrow \tau(M^{*})^{-}}{\lim}\|\varphi^{\tilde{t}_{2}}(M^{*})-\varphi^{\tilde{t}_{1}}(M^{*})\|_{H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)}=0. \nonumber \end{aligned} \end{equation}$

即存在一个$ M_{n}^{*} \in H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega),$ 使得

$\begin{equation}\label{5.7} \begin{aligned} \underset{t\rightarrow \tau(M^{*})^{-}}{\lim}\|\varphi^{t}(M^{*})- M_{n}^{*}\|_{H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)}=0. \end{aligned} \end{equation}$

记$\varphi^{t}(M^{*})=\varphi^{t}(v_{1k}^{*},v_{2k}^{*})=(u^{*},v^{*}),$$\varphi^{t_{k}}(M^{*})=\varphi^{t_{k}}(v_{1k}^{*},v_{2k}^{*})=(u_{k}^{*},v_{k}^{*}),$ 如果$\tau(M^{*})=\infty,$ 则存在一个序列$\{t_{k}\}$ 使得

$\begin{equation} \begin{aligned} 0=&\underset{k\rightarrow\infty}{\lim}\frac{\rm d}{{\rm d}t}E(\varphi^{t}((M^{*}))_{t=t_{k}}\\ =&\underset{k\rightarrow\infty}{\lim}(E_{1}'(\varphi^{t_{k}}(M^{*})),-u_{k}^{*}+B_{1}(u_{k}^{*}))_{H^{-s}(\Omega),H_{0}^{-s}(\Omega)}\\ &+\underset{k\rightarrow\infty}{\lim}(E_{2}'(\varphi^{t_{k}}(M^{*})),-v_{k}^{*}+B_{2}(v_{k}^{*}))_{H^{-1}(\Omega),H_{0}^{-1}(\Omega)}\\ \leq&-C_{1}\underset{k\rightarrow\infty}{\lim}\|u_{k}^{*}-A_{1}(u_{k}^{*})\|_{H_{0}^{s}(\Omega)}^{2}-C_{2}\underset{k\rightarrow\infty}{\lim}\|v_{k}^{*}-A_{2}(v_{k}^{*})\|_{H_{0}^{1}(\Omega)}^{2}.\nonumber \end{aligned} \end{equation}$

由引理 5.3, 可推出

$\begin{equation} \begin{aligned} &\underset{k\rightarrow\infty}{\lim}(\|u_{k}^{*}\|_{H_{0}^{s}}(\Omega))+\|v_{k}^{*}\|_{H_{0}^{1}(\Omega)}+\|A_{1}(u_{k}^{*})\|_{H_{0}^{s}(\Omega)}+\|A_{2}(v_{k}^{*})\|_{H_{0}^{1}(\Omega)})\\ \leq &\ C\underset{k\rightarrow\infty}{\lim}(1+\|u_{k}^{*}-A_{1}(u_{k}^{*})\|_{H_{0}^{s}(\Omega)}+\|v_{k}^{*}-A_{2}(v_{k}^{*})\|_{H_{0}^{1}(\Omega)})\\ \leq &\ C.\nonumber \end{aligned} \end{equation}$

因为$A$ 是紧的, 从而存在一个$\tilde{M}^{*}\in H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$, 使得 (5.7) 式成立. 根据 (5.6) 式, 可得$\tilde{M}^{*}\in \mathcal{M}_{\epsilon}.$ 显然, 每一个$\tilde{M}^{*}\in \mathcal{M}_{\epsilon}$ 都是 $E$ 的一个变号的临界点. 由 (5.5) 式, 以下成立

$\begin{equation} \begin{aligned} E(\tilde{M}^{*})&\leq E(M^{*})\\ &\leq\underset{(\hat{a}_{1},\hat{b}_{1}) \in [0,1]\times[0,1],\hat{a}_{2},\hat{b}_{2} \geq0}{\sup}E(k^{*}_{1},k^{*}_{2})\\ &\leq \sup E(\pi_{k})\\ &=E(u_{k}^{+},v_{k}^{+})+E(u_{k}^{-},v_{k}^{-})\\ &=E(u_{k},v_{k})\\ &\rightarrow\tilde{\xi},\nonumber \end{aligned} \end{equation}$

其中$(k^{*}_{1},k^{*}_{2})=(\hat{a}_{2} (\hat{a}_{1} \frac{u_{k}^{+}}{\|u_{k}^{+}\|_{H_{0}^{s}(\Omega)}}+(1-\hat{a}_{1} )\frac{u_{k}^{-}}{\|u_{k}^{-}\|_{H_{0}^{s}(\Omega)}}),\hat{b}_{2} (\hat{b}_{1} \frac{v_{k}^{+}}{\|v_{k}^{+}\|_{H_{0}^{1}(\Omega)}}+(1-\hat{b}_{1} )\frac{v_{k}^{-}}{\|v_{k}^{-}\|_{H_{0}^{1}(\Omega)}})).$ 因此, $\{\tilde{M}^{*}\}$ 是$\mathcal{N}$ 中的一个$\mathrm{(PS)}$ 序列, 由此可得$E$ 的一个临界点$(\bar{u},\bar{v})$ 且$(\bar{u},\bar{v})\in \mathcal{M}_{\epsilon},$ 这是系统 (1.1) 的最小能量变号解.

接下来关于变号域的个数利用反证法. 设$(\bar{u},\bar{v})$ 有三个变号域$\Omega_{1},\ \Omega_{2}$ 和$\Omega_{3}$, 使得在 $\Omega_{1}$ 中$\bar{u}>0,\ \bar{v}>0$; 在$\Omega_{2}$ 中$\bar{u}<0,\ \bar{v}<0$; 以及在$\Omega_{3}$ 中 $\bar{u}>0,\ \bar{v}>0$. 根据推论 2.1, 可知$(\bar{u}_{\mathcal{X}_{\Omega_{1}\cup\Omega_{2}}},\bar{v}_{\mathcal{X}_{\Omega_{1}\cup\Omega_{2}}})\in \mathcal{N}$, 则$E(\bar{u}_{\mathcal{X}_{\Omega_{1}\cup\Omega_{2}}},\bar{v}_{\mathcal{X}_{\Omega_{1}\cup\Omega_{2}}})\geq\tilde{\xi}.$ 由于$\Omega^{*}$ 是$(\bar{u},\bar{v})$ 的一个变号域, 可得

$\begin{equation} \begin{aligned} &(E_{1}'(\bar{u}_{\mathcal{X}_{\Omega^{*}}},\bar{v}_{\mathcal{X}_{\Omega^{*}}}),\bar{u}_{\mathcal{X}_{\Omega^{*}}})_{H^{-s}(\Omega),H_{0}^{s}(\Omega)}+(E_{2}'(\bar{u}_{\mathcal{X}_{\Omega^{*}}},\bar{v}_{\mathcal{X}_{\Omega^{*}}}),\bar{v}_{\mathcal{X}_{\Omega^{*}}})_{H^{-1}(\Omega),H_{0}^{1}(\Omega)}\\ =&(E_{1}'(\bar{u},\bar{v}),\bar{u}_{\mathcal{X}_{\Omega^{*}}})_{H^{-s}(\Omega),H_{0}^{s}(\Omega)}+(E_{2}'(\bar{u},\bar{v}),\bar{v}_{\mathcal{X}_{\Omega^{*}}})_{H^{-1}(\Omega),H_{0}^{1}(\Omega)}\\ =& 0.\nonumber \end{aligned} \end{equation}$

即有

$\begin{equation} \begin{aligned} E(\bar{u}_{\mathcal{X}_{\Omega^{*}}},\bar{v}_{\mathcal{X}_{\Omega^{*}}})\geq(\frac{1}{2}-\frac{1}{\mu})\|\bar{u}_{\mathcal{X}_{\Omega^{*}}}\|_{H_{0}^{s}(\Omega)}^{2}+(\frac{1}{2}-\frac{1}{\mu})\|\bar{v}_{\mathcal{X}_{\Omega^{*}}}\|_{H_{0}^{1}(\Omega)}^{2} >0. \nonumber \end{aligned} \end{equation}$

但是

$\begin{equation} \begin{aligned} \tilde{\xi}\leq E(\bar{u}_{\mathcal{X}_{\Omega_{1}\cup\Omega_{2}}},\bar{v}_{\mathcal{X}_{\Omega_{1}\cup\Omega_{2}}})<E(\bar{u}_{\mathcal{X}_{\Omega_{1}\cup\Omega_{2}}},\bar{v}_{\mathcal{X}_{\Omega_{1}\cup\Omega_{2}}})+E(\bar{u}_{\mathcal{X}_{\Omega_{3}}},\bar{v}_{\mathcal{X}_{\Omega_{3}}})\leq E(\bar{u},\bar{v})=\tilde{\xi}. \nonumber \end{aligned} \end{equation}$

显然矛盾. 故定理 1.1 结论成立.

6 定理 1.2 的证明

本节将应用亏格和相对亏格[24,26]证明定理 1.2.

定义 6.1 记$\sum \triangleq A \subset H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega) $, 其中$ A$ 是闭的, 满足 $A =-A $. 对于$A\subset \sum,$ 定义$\gamma(A)$ 是最小整数$k$, 使得存在一个奇连续映射$h :A\rightarrow \mathbb{R}^{k_{1}+k_{2}} \setminus\{0\},$ 其中$k=k_{1}+k_{2}$. 若不存在这样的$k$, 则$\gamma(A)=\infty$ 且$\gamma(\emptyset)=0.$

命题 6.1[26] 设 $A,\ B\in \sum,$$h$$ :H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)\rightarrow H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$ 是一个奇连续映射, 则

(i) 若$A\subset B$, 则$\gamma(A)\leq\gamma(B)$;

(ii) $\gamma(A\cup B)\leq\gamma(A)+\gamma(B);$

(iii) $\gamma(A)\leq\gamma(h(A));$

(iv) 若$A$ 是紧的, 则存在$N\in\sum$ 满足 $A\subset int(N)$, 使得$\gamma(N)=\gamma(A);$

(v) 设$W$ 是$H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$ 的一个线性子空间, 其中$\mathrm{dim}$$W=k$; 若$ A\subset W$ 是$W$ 中$(0,0)$ 的一个对称有界邻域, 则$\partial A\in\sum$, 以及$\gamma(A)=k.$

定义 6.2[30] 设$X$ 是一个Banach 空间. 对于$X$ 的对称闭子集$A\subset B\subset C$, 定义 $C$ 关于$(A, B)$ 的亏格 $\gamma(C;B,A)$ 为最小整数$k\in \mathbb{N}$, 使得存在对称闭子集$U,\ V\subset X$, 满足

(i) $C\subset U\cup V,\ B\subset U$, $\gamma(V)\leq k;$

(ii) 存在一个奇映射$h:U\rightarrow B$, 使得$h(A)\subset A$, 若不存在这样的$k\in \mathbb{N}$, 则$\gamma(C;B,A)=\infty;$

(iii) $\gamma(B;B,A)=0,$ 以及$\gamma(C;\emptyset,\emptyset)=\gamma(C).$

命题 6.2[24] 设$X$ 是一个Banach 空间, 并且$A\subset B\subset C$ 是$X$ 的对称闭子集.

(i) 若存在$X$ 的对称闭子集$C_{0},\ C_{1}$, 使得$C\subset C_{0} \cup C_{1},\ C_{1}\cap B=\emptyset,$ 则

$\begin{equation} \begin{aligned} \gamma(C;B,A)\leq\gamma(C_{0};B,A)+\gamma(C_{1}); \nonumber \end{aligned} \end{equation}$

(ii) 若$C'$ 是$X$ 的对称闭子集, 使得$B\subset C'$, 并且存在一个奇连续映射$\eta:C'\rightarrow C$, 使得$\eta(B)\subset B$ 与$\eta(A)\subset A$, 则

$\begin{equation} \begin{aligned} \gamma(C';B,A)\leq\gamma(C;B,A). \nonumber \end{aligned} \end{equation}$

因为$E$ 满足$\mathrm{(PS)}$ 条件, 可知$K_{c}$ 和$K_{c}^{*}(\epsilon)$ 是紧的. 因此, 对于所有的$ c\in \mathbb{R}$, 以下成立

$\begin{equation} \begin{aligned} \gamma(K_{c})<\infty, \ \gamma(K_{c}^{*}(\epsilon))<\infty,\ \forall\epsilon\in(0,\epsilon_{0}], \nonumber \end{aligned} \end{equation}$

其中 $\epsilon_{0}$ 见引理 5.2.

引理 6.1 给定 $c\geq0$ 和$\epsilon\in(0,\epsilon_{0}]$. 对于$K_{c}^{*}(\epsilon)$ 的一个对称邻域$N\subset(H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)\setminus D_{\epsilon})$, 存在$\sigma>0$ 和连续的奇映射$\eta:E^{c+\sigma}\cup D_{\epsilon}\setminus N\rightarrow E^{c-\sigma}\cup D_{\epsilon}$, 使得

(i) $\eta(D_{\epsilon})\subset D_{\epsilon};$

(ii) $E(\eta(u,v))\leq E(u,v),\ \forall\ (u,v)\in E^{c+\sigma}\cup D_{\epsilon}\setminus N.$

证 对于任意的$ \delta>0,$ 定义

$\begin{equation} \begin{aligned} (K_{c})_{\delta}&\triangleq\{M_{1}=(u,v):u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega),\ \mathrm{dist}(M_{1},K_{c})<\delta\},\\ N_{\delta} &\triangleq \{N_{1}=(u,v):u\in H_{0}^{s}(\Omega),\ v\in H_{0}^{1}(\Omega),\ \mathrm{dist}(N_{1},K_{c}^{*}(\epsilon)<\delta\}. \nonumber \end{aligned} \end{equation}$

取$\delta>0$ 充分小使得$N_{2\delta}\subset N$. 因为$E$ 满足$\mathrm{(PS)}$ 条件, 则存在常数$\hat{\sigma}>0$, 使得

$\begin{equation} \begin{aligned} \frac{\|E_{1}'(u,v)\|_{H^{-s}(\Omega)}^{2}}{1+\|E_{1}'(u,v)\|_{H^{-s}(\Omega)}}+\frac{\|E_{2}'(u,v)\|_{H^{-1}(\Omega)}^{2}}{1+\|E_{2}'(u,v)\|_{H^{-1}(\Omega)}}\geq\frac{8\hat{\sigma}}{\delta},\ \forall(u,v)\in E^{c+\hat{\sigma}}\setminus(E^{c-\hat{\sigma}}\cup(K_{c})_{\delta}). \nonumber \end{aligned} \end{equation}$

记

$\begin{equation} \begin{aligned} H_{1}\triangleq E^{c+\hat{\sigma}}\setminus(E^{c-\hat{\sigma}}\cup N_{\delta}),\ H_{2}\triangleq E^{c+\sigma}\setminus(E^{c-\sigma}\cup N_{2\delta}),\ \sigma\in(0,\hat{\sigma}) \nonumber \end{aligned} \end{equation}$

和

$\begin{equation} \begin{aligned} \vartheta(M)\triangleq\frac{\mathrm{dist}(M,(H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega))\setminus H_{1})}{\mathrm{dist}(M,(H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega))\setminus H_{1})+\mathrm{dist}(M,(H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega))\setminus H_{2})}, \nonumber \end{aligned} \end{equation}$

其中$M=(u,v)\in H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega).$

考虑

$\begin{equation}\label{6.1} \left\{ \begin{aligned} \frac{\rm d}{{\rm d}t}\xi(t,M)&=-\frac{\vartheta(\xi(t,M))W(\xi(t,M))}{1+\|W(\xi(t,M))\|_{H^{-s}(\Omega)\times H^{-1}(\Omega)}},\\ \xi(0,M)&=M, \end{aligned} \right. \end{equation}$

其中$W(M)=M-B(M),\ \forall M\in H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega).$ 显然, 存在最大区间$[0,\infty)$, 系统(6.1) 存在唯一的解

$\begin{equation} \begin{aligned} \xi(t,M)=e^{-\theta(t,M)}M+e^{-\theta(t,M)}\int_{0}^{t}e^{-\theta(s,M)}B(\xi(s,M))\rho(\xi(s,M)) {\rm d}s,\nonumber \end{aligned} \end{equation}$

其中 $\rho(M)=\frac{\vartheta(M)}{1+\|W(M)\|_{H^{-s}(\Omega)\times H^{-1}(\Omega)}}$ 和$\theta(t,M)=\int_{0}^{t}\rho(\xi(s,M)) {\rm d}s.$ 根据文献[16], 存在一个$T>0,$ 使得$\eta(M)\triangleq \xi(T,M) $ 成立.

接下来, 应用亏格和相对亏格得到系统 (1.1) 的一个变号解序列. 对于$m \in\mathbb{N},$ 定义

$\begin{equation} \begin{aligned} c_{ m}\triangleq \inf\{c\geq0:\gamma(E ^{c}\cup D_{\epsilon };\ E ^{0}\cup D_{\epsilon },\ E ^{-1})\geq m\}.\nonumber \end{aligned} \end{equation}$

结合系统 (1.1) 的泛函

$\begin{equation} \begin{aligned} E(u,v)=&\frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}u|^{2}\!+\!|\nabla v|^{2}{\rm d}x\!-\!\frac{1}{3s+1}\int_{\Omega}|u|^{3s+1}{\rm d}x\!-\!\frac{1}{4}\int_{\Omega}|v|^{4}{\rm d}x \!-\!\frac{\beta}{\alpha+1}\int_{\Omega}|v|^{2} |u|^{\alpha+1}{\rm d}x.\nonumber \end{aligned} \end{equation}$

由Young 不等式可得

$\begin{equation} \begin{aligned} |v|^{2} |u|^{\alpha+1}\leq\frac{|v|^{2p }}{p }+\frac{|u|^{(\alpha+1)q }}{q }, \nonumber \end{aligned} \end{equation}$

其中$p,q>1,\frac{1}{p }+\frac{1}{q }=1,$ 进而有

$\begin{equation}\label{m} \begin{aligned} &E(u,v)\\=&\frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}u|^{2}+|\nabla v|^{2}{\rm d}x-\frac{1}{3s+1}\int_{\Omega}|u|^{3s+1}{\rm d}x-\frac{1}{4}\int_{\Omega}|v|^{4}{\rm d}x -\frac{\beta}{\alpha+1}\int_{\Omega}|v|^{2} |u|^{\alpha+1}{\rm d}x \\ \geq&\frac{1}{2}\int_{\Omega}|(-\Delta)^{\frac{s}{2}}u|^{2}+|\nabla v|^{2}{\rm d}x-\frac{1}{3s+1}\int_{\Omega}|u|^{3s+1}{\rm d}x-\frac{1}{4}\int_{\Omega}|v|^{4}{\rm d}x\\ &-\frac{\beta}{\alpha+1}\int_{\Omega}(\frac{|v|^{2p }}{p }+\frac{|u|^{(\alpha+1)q }}{q }) {\rm d}x\\ \geq&\frac{1}{2}(\|u\|_{H_{0}^{s}(\Omega)}^{2}+\| v\|_{H_{0}^{1}(\Omega)}^{2})-\frac{1}{3s+1}\|u\|_{H_{0}^{s}(\Omega)}^{3s+1}-\frac{1}{4}\| v\|_{H_{0}^{1}(\Omega)}^{4} -\frac{\beta}{\alpha+1} (\frac{\| v\|_{H_{0}^{1}(\Omega)}^{2p}}{p }+\frac{\|u\|_{H_{0}^{s}(\Omega)}^{(\alpha+1)q} }{q }). \end{aligned} \end{equation}$

若

$\begin{equation} \begin{aligned} 2p \geq2,\ (\alpha+1)q\geq2, \nonumber \end{aligned} \end{equation}$

即

$\begin{equation} \begin{aligned} p\geq1,\ q\geq\frac{2}{\alpha+1}, \nonumber \end{aligned} \end{equation}$

则

$\begin{equation} \begin{aligned} \frac{1}{p}+\frac{1}{q}=1 \leq1+\frac{\alpha+1}{2}. \nonumber \end{aligned} \end{equation}$

由于$\alpha>0$, 故以上式子成立. 而$s\in[\frac{3}{4},1)$, 有$3s+1>2$, 因此, 当$\|u\|_{H_{0}^{s}(\Omega)}$ 和$\|v\|_{H_{0}^{1}(\Omega)}$ 充分小, 根据 (6.2) 式有

$\begin{equation} \begin{aligned} E(u,v)>0. \nonumber \end{aligned} \end{equation}$

接下来, 将要验证对于每一个$m \in\mathbb{N},$$c_{ m}$ 都是$E $ 的临界值.

引理 6.2 如果存在$m \in\mathbb{N}$, 使得$c=c_{ m}=c_{ m+1}=\cdots=c_{ m+j},$ 则$\gamma(K_{c}^{*}(\epsilon ))\geq j+1$ 且当$m\rightarrow\infty$ 时, $c_{ m}\rightarrow+\infty.$

证 首先验证$0<c_{ 1}\leq c_{ 2}\leq\cdots<\infty.$ 根据$c_{ m}$ 的定义, 可得$c_{ 1}>0.$ 否则, 若$c_{ 1}=0,$ 从而存在一个序列$\{\delta_{i}\}\subset\mathbb{R}^{n}$, 使得当$ \delta_{i}\rightarrow0$ 时, 有

$\begin{equation}\label{6.2} \gamma(E^{\delta_{i}}\cup D_{\epsilon };\ E^{0}\cup D_{\epsilon },\ E^{-1})\geq1. \end{equation}$

若$(u,v)\in K_{0}$, 即$E(u,v)=0, E_{1}'(u,v)=E_{2}'(u,v)=0$, 则

$\begin{equation} \begin{aligned} 0&=\frac{1}{2}E_{1}'(u,v)\cdot u+\frac{1}{2}E_{2}'(u,v)\cdot v-E(u,v)\\ &=\int_{\Omega}(F(u,v)-\frac{1}{2}f_{1}(u,v)\cdot u-\frac{1}{2}f_{2}(u,v)\cdot v){\rm d}x\\ &\leq(\frac{1}{\mu}-\frac{1}{2})\|u\|_{H_{0}^{s}(\Omega)}^{2}+(\frac{1}{\mu}-\frac{1}{2})\|v\|_{H_{0}^{1}(\Omega)}^{2},\nonumber \end{aligned} \end{equation}$

即对任意的$\epsilon\in(0,\epsilon_{0}]$, $(u,v)=(0,0)\in D_{\epsilon}$, 可得$K_{0}^{*}(\epsilon)=\emptyset$. 再由引理 6.1, 存在常数$\sigma>0$ 和$\eta:E^{\sigma}\cup D_{\epsilon}\rightarrow E^{-\sigma}\cup D_{\epsilon}$, 使得

(i) $\eta(D_{\epsilon})\subset D_{\epsilon}$;

(ii) $E(\eta(M))\leq E(M),\ \forall M\in E^{\sigma}\cup D_{\epsilon}$.

由命题 6.1(i), (ii) 和$E^{-\sigma}\cup D_{\epsilon}\subset E^{0}\cup D_{\epsilon}$, 可得

$\begin{equation}\label{6.3} \begin{aligned} &\gamma(E^{\sigma}\cup D_{\epsilon};\ E^{0}\cup D_{\epsilon},\ E^{-1})\\ \leq&\gamma(E^{-\sigma}\cup D_{\epsilon};\ E^{0}\cup D_{\epsilon},\ E^{-1}) \\ \leq&\gamma(E^{0}\cup D_{\epsilon};\ E^{0}\cup D_{\epsilon},\ E^{-1}) \\ =&0. \end{aligned} \end{equation}$

取充分大的$i$ 使得$\delta_{i}<\sigma$, 根据 (6.3), (6.4) 式, 命题 6.2(i) 和$E^{\delta_{i}}\cup D_{\epsilon }\subset E^{\sigma}\cup D_{\epsilon }$, 以下成立

$\begin{equation} \begin{aligned} 1\leq\gamma(E^{\delta_{i}}\cup D_{\epsilon };E^{0}\cup D_{\epsilon },E^{-1})\leq\gamma(E^{\sigma}\cup D_{\epsilon };E^{0}\cup D_{\epsilon },E^{-1})\leq0.\nonumber \end{aligned} \end{equation}$

显然矛盾, 故$c_{1 }>0.$

接下来, 利用反证法不妨设$\gamma(K_{c}^{*}(\epsilon ))\leq j$. 取$X\setminus(E^{0}\cup D_{\epsilon })$ 中$K_{c}^{*}(\epsilon )$ 的一个对称邻域$N$, 使得$\gamma(N)=\gamma(K_{c}^{*}(\epsilon))$. 让$\delta=\sigma>0$, 类似引理 6.1. 应用命题 6.1(iv), 命题 6.2 和引理 6.1, 可得

$\begin{equation} \begin{aligned} &\gamma(E^{c+\delta}\cup D_{\epsilon};\ E^{0}\cup D_{\epsilon },\ E^{-1})\\ \leq&\gamma((E^{c+\delta}\setminus N)\cup D_{\epsilon};\ E^{0}\cup D_{\epsilon },\ E^{-1})+\gamma(K_{c}^{*}(\epsilon))\\ \leq&\gamma(E^{c-\delta}\cup D_{\epsilon };\ E^{0}\cup D_{\epsilon },\ E^{-1})+\gamma(K_{c}^{*}(\epsilon)).\nonumber \end{aligned} \end{equation}$

根据$c_{m}$ 的定义, 可以推出

$\begin{equation} \begin{aligned} m+j&\leq\gamma(E^{c_{m+j}+\delta}\cup D_{\epsilon };\ E^{0}\cup D_{\epsilon },\ E^{-1})\\ &=\gamma(E^{c_{m}+\delta}\cup D_{\epsilon };\ E^{0}\cup D_{\epsilon },\ E^{-1})\\ &\leq\gamma(E^{c_{m}-\delta}\cup D_{\epsilon };\ E^{0}\cup D_{\epsilon },\ E^{-1})+j\\ &\leq m-1+j,\nonumber \end{aligned} \end{equation}$

显然矛盾, 故$\gamma(K_{c}^{*}(\epsilon))\geq j+1 $. 结论二类似可得.

定义

$\begin{equation} \begin{aligned} \beta_{m}\triangleq \underset {W \subset H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega),\mathrm{dim}W\geq m}{\inf}\underset {M\in W}{\sup}E(M). \nonumber \end{aligned} \end{equation}$

引理 6.3 对于任意的$m\in\mathbb{N},\ c_{m}\leq\beta_{m+1}$.

证 对于每一个整数$j\geq1,$ 取$H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$ 的任意一个$m+j$ 维子空间$X_{m+j}$. 显然, $d_{ m+j}\triangleq \underset {M\in X_{m+j}}{\sup}E (M)<\infty$. 此外, 存在一个序列$\{R_{m+j}\}\subset \mathbb{R}^{+},$ 对于所有的$M\in X_{m+j}\setminus B_{R_{m+j}},$ 可得$E (M)\leq-1$, 其中$B_{R_{m+j}}\triangleq \{M\in X_{m+j}:\|M\|_{H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)}<R_{m+1}\}.$ 对于所有的$ j\geq1$, 需证$d_{ m+j}\geq c_{ m}$. 由反证法, 设存在$j_{0}\in\mathbb{N}$, 有$d_{ m+j_{0}}< c_{ m}$. 根据$c_{ m}$ 的定义, 可得$\gamma(E^{d_{ m+j_{0}}}\cup D_{\epsilon };\ E^{0}\cup D_{\epsilon },\ E^{-1})\leq m-1$, 存在$X$ 的对称闭子集$U,\ V $, 使得$X_{m+j_{0}}\subset U\cup V,\ E^{0}\cup D_{\epsilon }\subset U$ 和$\gamma(V)\leq m-1$. 此外, 存在一个连续的奇映射$h:U\rightarrow E^{0}\cup D_{\epsilon },$ 使得$ h(E^{-1})\subset E^{-1}.$ 根据 Tietze's 定理, 可以将$h$ 拓展为$H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega) $ 上的一个连续奇映射. 令

$\begin{equation} \begin{aligned} \bar{O}_{\epsilon }\triangleq \bar{D}_{\epsilon }^{+}\cap \bar{D}_{\epsilon }^{-},\quad \mathcal{O}_{m+j_{0}}\triangleq \{M\in X_{m+j_{0}}:h(M)\in C_{X}(\bar{O}_{\epsilon})\}. \nonumber \end{aligned} \end{equation}$

因为

$\begin{equation} \begin{aligned} \underset{M\in X_{m+j_{0}}\setminus B_{R_{m+j_{0}}}}{\sup}E(M)<\underset{M\in \bar{O}_{\epsilon }}{\inf}E(M)\leq\underset{M\in C_{X}(\bar{O}_{\epsilon })}{\inf}E(M)\leq \underset{M\in \mathcal{ O }_{m+j_{0}}}{\inf}E(h(M)), \nonumber \end{aligned} \end{equation}$

则有

$\begin{equation}\label{6.4} \begin{aligned} \gamma(\partial_{X_{m+j_{0}}}\mathcal{ O }_{m+j_{0}})=m+j_{0}. \end{aligned} \end{equation}$

由于$h(\partial_{X_{m+j_{0}}}\mathcal{ O }_{m+j_{0}}\cap U)\subset X_{m+j_{0}}\cap\partial C_{X}(\bar{O}_{\epsilon })\cap D_{\epsilon },$ 有

$\begin{equation}\label{6.5} \begin{aligned} \gamma(\partial_{X_{m+j_{0}}}\mathcal{ O }_{m+j_{0}}\cap U)\leq\gamma (X_{m+j_{0}}\cap\partial C_{X}(\bar{O}_{\epsilon })\cap D_{\epsilon })\leq1. \end{aligned} \end{equation}$

又由$\gamma(\partial_{X_{m+j_{0}}}\mathcal{ O }_{m+j_{0}}\cap U)\leq\gamma(V)\leq m-1,$ 结合 (6.6) 式和亏格的次可加性, 可得

$\begin{equation} \begin{aligned} \gamma(\partial_{X_{m+j_{0}}}\mathcal{ O }_{m+j_{0}})&\leq\gamma(\partial_{X_{m+j_{0}}}\mathcal{ O }_{m+j_{0}}\cap (U\cup V))\\ &\leq\gamma(\partial_{X_{m+j_{0}}}\mathcal{ O }_{m+j_{0}}\cap U)+\gamma(\partial_{X_{m+j_{0}}}\mathcal{ O }_{m+j_{0}}\cap V)\\ &\leq m.\nonumber \end{aligned} \end{equation}$

这与 (6.5) 式矛盾. 故结论成立.

定理 1.2 的证明 由引理 6.2, 系统 (1.1) 存在一列变号解$\{\pm M_{ m}\}=\{(\pm u_{ m},\pm v_{ m})\}\subset( H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega))\setminus D_{\epsilon }$, 使得当$m\rightarrow +\infty$ 时, 有

$\begin{equation}\label{6.6} \begin{aligned} E( u_{m}, v_{m})=c_{m}\rightarrow +\infty. \end{aligned} \end{equation}$

因为$\{\varphi_{m}\}$ 是$(-\Delta)^{s}$ 和$-\Delta$ 的特征向量, $X_{m}\triangleq \mathrm{span}\{(\varphi_{1},\varphi_{1}),(\varphi_{2},\varphi_{2}),\cdots,(\varphi_{m},\varphi_{m})\}$. 结合 (III), 存在常数$b_{1},\ b_{2},\ b_{3}>0$, 使得

$\begin{equation} \begin{aligned} F(\bar{\omega},\bar{\nu})\geq b_{1}|\bar{\omega}|^{\mu}+ b_{2}|\bar{\nu}|^{\mu}- b_{3},\ \forall \bar{\omega},\ \bar{\nu}\in\mathbb{R}. \nonumber \end{aligned} \end{equation}$

根据$\beta_{m}$ 的定义, 可得

$\begin{equation} \begin{aligned} \beta_{ m}&\leq\underset{(u,v)\in X_{m}}{\sup}E (u,v)\\ &\leq\underset{(u,v)\in X_{m}}{\sup}\frac{1}{2}\int_{\Omega}(|(-\Delta)^{\frac{s}{2}}u|^{2}+|\nabla v|^{2}){\rm d}x-b_{1}\int_{\Omega}|u|^{\mu}{\rm d}x-b_{2}\int_{\Omega}|v|^{\mu}{\rm d}x+b_{3}|\Omega|\\ &\triangleq\bar{\beta}_{m}.\nonumber \end{aligned} \end{equation}$

由引理 6.3, 对于每一个$m\in\mathbb{N}$, 可得

$\begin{equation}\label{6.7} \begin{aligned} E(u_{m},v_{m})\leq\beta_{m+1}\leq\bar{\beta}_{m+1}. \end{aligned} \end{equation}$

又由于$f_{1}$ 和$f_{2}$ 是连续的函数, 可知存在$\zeta>0$, 使得

$\begin{equation}\label{6.8} \begin{aligned} c_{ m}-\zeta\leq \bar{\beta}_{m}. \end{aligned} \end{equation}$

注意到$E_{1 }'(u_{ m},v_{ m})=0,\ E_{2 }'(u_{ m},v_{ m})=0$. 结合 (6.9) 式和引理 2.2, 对于任意的$m\in\mathbb{N}$, 存在$(u_{m},v_{m})\in H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega)$, 使得$(u_{m},v_{m})$ 是系统 (1.1) 的解. 另外, 根据 (6.7) 式, 可得

$\begin{equation} \begin{aligned} \underset{m\rightarrow\infty}{\lim}E(u_{m},v_{m})\geq\underset{m\rightarrow\infty}{\lim}(c_{ m}-\zeta)=\infty. \nonumber \end{aligned} \end{equation}$

即存在$n_{0}\in\mathbb{N}$, 当$m\geq n_{0}$ 时,$(u_{m},v_{m})$ 是系统 (1.1) 的一个非平凡解, 且对于所有的$m\geq n_{0}$, $(u_{m},v_{m})$ 在$\Omega$ 中变号.故可得系统 (1.1) 的一个无界变号解序列$\{(u_{m},v_{m})\}$. 定理 1.2 的第一部分证明完毕.

下面利用反证法证明第二部分. 由于 (V) 成立, 设存在$m_{0}\in\mathbb{N}$, 使得$(u_{m_{0}},v_{m_{0}})$ 有超过$m_{0}+1$ 个变号域. 不失一般性, 设$u_{m_{0}}$ 有$m_{0}+2$ 个变号域, 分别为$\Omega_{1}, \Omega_{2}, \cdots, \Omega_{m_{0}+2}$, 关于$v_{m_{0}}$ 的结论类似可得. 定义

$\begin{equation} \hat{u}_{i,m_{0}} = \begin{cases} u_{m_{0}}(x), & x\in \Omega_{i},\\ 0, & x\in \Omega\setminus\Omega_{i},\nonumber \end{cases} \end{equation}$

其中$i=1,2,\cdots,m_{0}+2.$ 根据推论 2.1, $\hat{u}_{i,m_{0}}\in H_{0}^{s}(\Omega)$. 对于$\tilde{u}_{m_{0}}=u_{m_{0}}-\sum\limits_{i=1}^{m_{0}+1}\hat{u}_{i,m_{0}}$, 有

$\begin{equation} \begin{aligned} E(u_{m_{0}},v_{m_{0}})=E(\tilde{u}_{m_{0}}, v_{m_{0} })+\sum\limits_{i=1}^{m_{0}+1}E(\hat{u}_{i,m_{0}},v_{m_{0} })=\eta_{m_{0}}>0. \nonumber \end{aligned} \end{equation}$

通过$\Omega_{1}, \Omega_{2}, \cdots,\Omega_{m_{0}+1}$ 的合理选择, 设$E(\tilde{u}_{m_{0}},v_{m_{0} })>0$, 那么

$\begin{equation} \begin{aligned} (E_{1}'(\hat{u}_{i,m_{0}},v_{m_{0}}),\hat{u}_{i,m_{0}})_{H^{-s}(\Omega),H_{0}^{s}(\Omega)}=(E_{1}'(u_{m_{0}},v_{m_{0}}),\hat{u}_{i,m_{0}})_{H^{-s}(\Omega),H_{0}^{s}(\Omega)}=0, \nonumber \end{aligned} \end{equation}$

结合$E(-\hat{u}_{i,m_{0}},-v_{m_{0}})=E(\hat{u}_{i,m_{0}},v_{m_{0}})$ 与 (5.5) 式的证明过程, 可得

$\begin{equation} \begin{aligned} E(\hat{u}_{i,m_{0}},v_{m_{0}})=\underset{(t,s)\in\mathbb{R}\times \mathbb{R}}{\sup}E(t\hat{u}_{i,m_{0}},sv_{m_{0}}). \nonumber \end{aligned} \end{equation}$

记$X_{m_{0}+1}^{1}=\mathrm{span}\{(\hat{u}_{1,m_{0}},v_{m_{0}}),\ (\hat{u}_{2,m_{0}},v_{m_{0}}),\ \cdots,\ (\hat{u}_{m_{0}+1,m_{0}},v_{m_{0}})\}$. 显然, $\mathrm{dim}X_{m_{0}+1}^{1}=m_{0}+1.$ 则

$\begin{equation} \begin{aligned} \beta_{ m_{0}+1}&= \underset{W\subset H_{0}^{s}(\Omega)\times H_{0}^{1}(\Omega),\mathrm{dim} W\geq m_{0}+1}{\inf}\underset{(u,v)\in W}{\sup}E(u,v)\\ &\leq \underset{(u,v)\in X_{m_{0}+1}^{1}}{\sup}E(u,v)\\ &=\sum\limits_{i=1}^{m_{0}+1}E(\hat{u}_{i,m_{0}},v_{m_{0}})\\ &=E(u_{m_{0}},v_{m_{0}})-E(\tilde{u}_{m_{0}},v_{m_{0}})\\ &<E(u_{m_{0}},v_{m_{0}})\\ &=c_{m_{0}}\\ &\leq \beta_{ m_{0}+1}.\nonumber \end{aligned} \end{equation}$

显然矛盾. 故定理 1.2 得证.

参考文献

Caffarelli L A, Roquejoffre J M, Sire Y.

Variational problems for free boundaries for the fractional Laplacian

J Eur Math Soc, 2010, 12(5): 1151-1179

DOI:10.4171/jems      URL     [本文引用: 1]

Barrios B, Figalli A, Ros-Oton X.

Global regularity for the free boundary in the obstacle problem for the fractional Laplacian

Amer J Math, 2018, 140(2): 415-447

DOI:10.1353/ajm.2018.0010      URL    

Jhaveri Y, Neumayer R.

Higher regularity of the free boundary in the obstacle problem for the fractional Laplacian

Adv Math, 2017, 311: 748-795

DOI:10.1016/j.aim.2017.03.006      URL    

Yang R.

Optimal regularity and nondegeneracy of a free boundary problem related to the fractional Laplacian

Arch Ration Mech Anal, 2013, 208(3): 693-723

DOI:10.1007/s00205-013-0619-7      URL    

De Silva D, Roquejoffre J M.

Regularity in a one-phase free boundary problem for the fractional Laplacian

Ann Inst H Poincaré C Anal Non Linéaire, 2012, 29(3): 335-367

DOI:10.4171/aihpc      URL    

Garofalo N, Petrosyan A, Pop C A, et al.

Regularity of the free boundary for the obstacle problem for the fractional Laplacian with drift

Ann Inst H Poincaré C Anal Non Linéaire, 2017, 34(3): 533-570

DOI:10.4171/aihpc      URL    

Kulczycki T, Wszoa J.

On the interior Bernoulli free boundary problem for the fractional Laplacian on an interval

Collect Math, 2025, 76(1): 11-34

DOI:10.1007/s13348-023-00417-5      [本文引用: 1]

Caffarelli L, Silvestre L.

An extension problem related to the fractional Laplacian

Commun Partial Differential Equations, 2007, 32(7/9): 1245-1260

DOI:10.1080/03605300600987306      URL     [本文引用: 3]

Kwaśnicki M, Mucha J.

Extension technique for complete Bernstein functions of the Laplacian operator

J Evol Equ, 2018, 18(3): 1341-1379

DOI:10.1007/s00028-018-0444-4      [本文引用: 1]

Kwaśnicki M.

Ten equivalent definitions of the fractional Laplacian operator

Fract Calc Appl Anal, 2017, 20(1): 7-51

DOI:10.1515/fca-2017-0002      URL     [本文引用: 1]

Lin T C, Wei J C.

Spikes in two coupled nonlinear Schrödinger equations

Ann Inst H Poincaré C Anal Non Linéaire, 2005, 22(4): 403-439

DOI:10.4171/aihpc      URL     [本文引用: 1]

Bartsch T, Chang K C, Wang Z Q.

On the Morse indices of sign-changing solutions for nonlinear elliptic problems

Math Z, 2000, 233(4): 655-677

DOI:10.1007/s002090050492      URL     [本文引用: 1]

Zhang Q Y, Xu B.

Multiple small solutions for some Schrödinger-Poisson systems

Nonlinear Anal, 2015, 117: 200-210

DOI:10.1016/j.na.2015.01.009      URL     [本文引用: 1]

Alves C O, Souto M A S, Soares S H M.

A sign-changing solution for the Schrödinger-Poisson equation in $\mathbb{R}^{3}$

Rocky Mountain J Math, 2017, 47(1): 1-25

[本文引用: 1]

Cui L C, Mao A.

Existence and asymptotic behavior of positive solutions to some logarithmic Schrödinger-Poisson system

Z Angew Math Phys, 2024, 75(1): 30-30

DOI:10.1007/s00033-023-02170-y     

Liu Z L, Wang Z Q.

Sign-changing solutions of nonlinear elliptic equations

Front Math, 2008, 3(2): 221-238

[本文引用: 1]

Wang J, Tian L X, Xu J X, et al.

Existence of multiple positive solutions for Schrödinger-Poisson systems with critical growth

Z Angew Math Phys, 2015, 66(5): 2441-2471

DOI:10.1007/s00033-015-0531-0      URL    

Shuai W, Wang Q F.

Existence and asymptotic behavior of sign-changing solutions for the nonlinear Schrödinger-Poisson system in $\mathbb{R}^{3}$

Z Angew Math Phys, 2015, 66(6): 3267-3282

DOI:10.1007/s00033-015-0571-5      URL     [本文引用: 1]

Chen J H, Tang X H, Luo H X.

Infinitely many solutions for fractional Schrödinger-Poisson systems with sign-changing potential

Electron J Differential Equations, 2017, 2017(1): 1072-6691

[本文引用: 1]

Guo L.

Sign-changing solutions for fractional Schrödinger-Poisson system in double-struck capital $\mathbb{R}^{3}$

Appl Anal, 2019, 98(9-12): 2085-2104

DOI:10.1080/00036811.2018.1448074      URL    

Li K X.

Existence of non-trivial solutions for nonlinear fractional Schrödinger-Poisson equations

Appl Math Lett, 2017, 72: 1-9

DOI:10.1016/j.aml.2017.03.023      URL    

Yu Y Y, Zhao F K, Zhao L G.

Positive and sign-changing least energy solutions for a fractional Schrödinger-Poisson system with critical exponent

Appl Anal, 2020, 99(13): 2229-2257

DOI:10.1080/00036811.2018.1557325      URL     [本文引用: 1]

Bartsch T, Liu Z L, Weth T.

Nodal solutions of a $p$-Laplacian equation

Int Rev Red Cross, 2005, 91(1): 129-152

[本文引用: 2]

Bartsch T, Liu Z L, Weth T.

Sign changing solutions of superlinear Schrödinger equations

Comm Partial Differential Equations, 2004, 29(1/2): 25-42

DOI:10.1081/PDE-120028842      URL     [本文引用: 2]

Liu Z L, Sun J X.

Invariant sets of descending flow in critical point theory with applications to nonlinear differential equations

J Differential Equations, 2001, 172(2): 257-299

DOI:10.1006/jdeq.2000.3867      URL     [本文引用: 5]

Rabinowitz P.

Minimax Methods in Critical Point Theory with Applications to Differential Equations

Providence, RI: American Mathematical Society, 1986

[本文引用: 4]

Brändle C, Colorado E, de Pablo A, et al.

A concave-convex elliptic problem involving the fractional Laplacian

Proc Roy Soc Edinburgh Sect A, 2013, 143(1): 39-71

DOI:10.1017/S0308210511000175      URL     [本文引用: 2]

Capella A, Dácila J, Dupaigne L, et al.

Regularity of radial extremal solutions for some nonlocal semilinear equations

Comm Partial Differential Equations, 2011, 36(8): 1353-1384

DOI:10.1080/03605302.2011.562954      URL     [本文引用: 2]

Dancer E N, Wei J C, Weth T.

A priori bounds versus multiple existence of positive solutions for a nonlinear Schrödinger system

Ann Inst H Poincaré C Anal Non Linéaire, 2010, 27(3): 953-969

DOI:10.4171/aihpc      URL     [本文引用: 1]

Chang X J, Wang Z Q.

Nodal and multiple solutions of nonlinear problems involving the fractional Laplacian

J Differential Equations, 2014, 256(8): 2965-2992

DOI:10.1016/j.jde.2014.01.027      URL     [本文引用: 4]

Heywood J G, Noussair E S, Swanson C A.

On the zeros of solutions of elliptic inequalities in bounded domains

J Differential Equations, 1978, 28(3): 345-353

DOI:10.1016/0022-0396(78)90132-8      URL     [本文引用: 1]

Cabré X, Tan J G.

Positive solutions of nonlinear problems involving the square root of the Laplacian

Adv in Math, 2010, 224(5): 2052-2093

DOI:10.1016/j.aim.2010.01.025      URL     [本文引用: 2]

Brézis H, Kato T.

Remarks on the Schrödinger operator with singular complex potentials

J Math Pures Appl (9), 1979, 58(2): 137-151

[本文引用: 1]

/

〈 〉

版权所有 © 《数学物理学报》杂志社
地址：武汉市71010号信箱(430071)　电话: 027-87199206(中文版)　027-87199087(英文版)
E-mail: actams@wipm.ac.cn
本系统由北京玛格泰克科技发展有限公司设计开发