Chong Huang

This paper investigates a deep learning-based algorithm to optimize the unmanned aerial vehicle (UAV) trajectory and reconfigurable intelligent surface (RIS) reflection coefficients in UAV-RIS-aided cell-free (CF) hybrid non-orthogonal multiple-access (NOMA)/orthogonal multiple-access (OMA) networks. The practical RIS reflection model and user grouping optimization are considered in the proposed network. A double cascade correlation network (DCCN) is proposed to optimize the RIS reflection coefficients , and based on the results from DCCN, an inverse-variance deep reinforcement learning (IV-DRL) algorithm is introduced to address the UAV trajectory optimization problem. Simulation results show that the proposed algorithms significantly improve the performance in UAV-RIS-assisted CF networks.

This paper investigates a learning-based approach autonomously and jointly optimizing the trajectory of unmanned aerial vehicle (UAV), phase shifts of reconfigurable intelligent surfaces (RIS), and aggregation weights for federated learning (FL) in wireless communications, forming an autonomous RIS-assisted UAV-enabled network. The proposed network considers practical RIS reflection models and FL transmission errors in wireless communications. To optimize the RIS phase shifts, a double cascade correlation network (DCCN) is introduced. Additionally, the deep deterministic policy gradient (DDPG) algorithm is employed to address the optimization problem of UAV trajectory and FL aggregation weights based on the results obtained from DCCN. Simulation results demonstrate the substantial improvement in FL performance within the autonomous RIS-assisted UAV-enabled network setting achieved by the proposed algorithms compared to the benchmarks.

This paper proposes a new framework for reconfigurable intelligent surface (RIS)-equipped unmanned aerial vehicles (UAVs) in free-space optical (FSO) communication. To ensure practicality, we consider atmospheric loss caused by fog, which leads to an inhomogeneous medium for laser propagation. In addition, we incorporate the pointing error loss caused by the power fraction on the photodetector (PD) into the system and derive a closed-form expression for the elliptical beam footprint in the pointing error loss. We then propose a leading angle assisted particle swarm optimization (PSO) method to efficiently optimize the numerical results of pointing error loss. Furthermore, after obtaining these numerical results as a precondition, the UAV trajectory is optimized using the proximal policy optimization (PPO) method to achieve the maximum average capacity. Numerical simulations demonstrate that the proposed optimization method achieves greater efficiency and accuracy compared to the decode-and-forward (DF) relay and deep Q-learning (DQN) methods.

—In recent years, the amalgamation of satellite communications and aerial platforms into space-air-ground integrated network (SAGINs) has emerged as an indispensable area of research for future communications due to the global coverage capacity of low Earth orbit (LEO) satellites and the flexible Deployment of aerial platforms. This paper presents a deep reinforcement learning (DRL)-based approach for the joint optimization of offloading and resource allocation in hybrid cloud and multi-access edge computing (MEC) scenarios within SAGINs. The proposed system considers the presence of multiple satellites, clouds and unmanned aerial vehicles (UAVs). The multiple tasks from ground users are modeled as directed acyclic graphs (DAGs). With the goal of reducing energy consumption and latency in MEC, we propose a novel multi-agent algorithm based on DRL that optimizes both the offloading strategy and the allocation of resources in the MEC infrastructure within SAGIN. A hybrid action algorithm is utilized to address the challenge of hybrid continuous and discrete action space in the proposed problems, and a decision-assisted DRL method is adopted to reduce the impact of unavailable actions in the training process of DRL. Through extensive simulations, the results demonstrate the efficacy of the proposed learning-based scheme, the proposed approach consistently outperforms benchmark schemes, highlighting its superior performance and potential for practical applications. Index Terms—Space-air-ground integrated networks, edge computing , resource allocation, unmanned aerial vehicle, deep reinforcement learning.

In this paper, we study the simultaneous wireless information and power transfer (SWIPT) cooperative system, where one source forwards information to one destination with the assistance of multiple relays. Each relay is equipped with a finite data buffer and a finite energy buffer storing the harvested energy by radio-frequency (RF). An optimization problem is formulated for throughput maximization of the SWIPT cooperative system, taking into consideration the strict delay constraint, dynamic channel conditions, time-varying discrete data buffer states and time-varying continuous energy buffer states. A discrete-time Markov decision process (MDP) is adopted to model the relay selection process referring to data buffer states and energy buffer states. Two deep Q-network (DQN)-based methods named invalid action penalty (IAP) and invalid action mask (IAM) are proposed. The simulation results show that the proposed IAM method can achieve better convergence and throughput performance than the IAP method.

This letter investigates machine learning approach for the joint optimal phase shift and beamforming in the reconfigurable intelligent surface (RIS) assisted multiple-input and multiple-output (MIMO) network, consisting of one source node, one RIS panel and one destination node. If individual source-to-RIS and RIS-to-destination channels are known, the joint optimization is similar to that in the traditional MIMO network, which has been well studied. However, the channel estimation for the individual channels is complicated and often inaccurate. On the other hand, while estimating the cascaded channels for the source-RIS-destination links are more accessible, the corresponding joint optimization is complicated. In this letter, we propose a novel double deep learning network model which is superior to the conventional reinforcement learning in the RIS joint optimization. Numerical simulations are given to verify the proposed algorithm.

This article proposes a novel machine-learning-based routing optimization for the multiple reconfigurable intelligent surfaces (M-RIS)-assisted multihop cooperative networks, in which a practical phase model for reconfigurable intelligent surface (RIS) with the amplitude variation based on the corresponding discrete phase shift is considered. We aim to maximize the end-to-end data rate in the proposed network by jointly optimizing the data transmission path, the passive beamforming design of RIS, and transmit power allocation. To tackle this complicated nonconvex problem, we divide it into two subtasks: 1) the passive beamforming design of the RIS and 2) joint routing and power allocation optimization. First, for the passive beamforming design of RIS, we develop a distributed learning algorithm that employs a cascade forward backpropagation network in each relay node to solve the RIS coefficients optimization problem by directly using the optimization target to train the cascade networks. This solution can avoid the curse of dimensionality of traditional reinforcement learning algorithms in the RIS optimization problem. Then, based on the result of RIS optimization, we introduce the proximal policy optimization (PPO) algorithm with the clipping method to find solutions for joint optimization of routing and power allocation via achieving the long-term benefit in the Markov decision process (MDP). Simulation results show that the proposed learning-based scheme can learn from the environment to improve its policy stability and efficiency in the iterative training process for optimizing routing and RIS and significantly outperform the benchmark schemes.